You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "AntoinePrv (via GitHub)" <gi...@apache.org> on 2025/08/08 13:32:27 UTC
[PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
AntoinePrv opened a new pull request, #47294:
URL: https://github.com/apache/arrow/pull/47294
### Rationale for this change
### What changes are included in this PR?
### Are these changes tested?
Yes.
### Are there any user-facing changes?
API changes to internal classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on PR #47294:
URL: https://github.com/apache/arrow/pull/47294#issuecomment-3167935405
Work in progress, this is currently a split of the decoder in a parser and a decoder, but is not plugged in.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #47294:
URL: https://github.com/apache/arrow/pull/47294#issuecomment-3245248171
There is a fuzz regression failure which should probably be addressed @AntoinePrv : https://github.com/apache/arrow/actions/runs/17266367915/job/48999342089?pr=47294#step:7:6933
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #47294:
URL: https://github.com/apache/arrow/pull/47294#issuecomment-3299469341
Updated benchmark results here:
<details>
```
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Non-regressions: (349)
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
benchmark baseline contender change % counters
BM_DeltaBitPackingDecode_Int64_Narrow/65536 4.802 GiB/sec 6.087 GiB/sec 26.764 {'family_index': 14, 'per_family_instance_index': 3, 'run_name': 'BM_DeltaBitPackingDecode_Int64_Narrow/65536', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 3365}
BM_DeltaBitPackingDecode_Int32_Wide/1024 1.972 GiB/sec 2.449 GiB/sec 24.222 {'family_index': 15, 'per_family_instance_index': 0, 'run_name': 'BM_DeltaBitPackingDecode_Int32_Wide/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 148102}
BM_ReadStructOfStructColumn/50 1.266 GiB/sec 1.547 GiB/sec 22.118 {'family_index': 19, 'per_family_instance_index': 2, 'run_name': 'BM_ReadStructOfStructColumn/50', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 15}
ReadFileMetadata/num_columns:100/num_row_groups:100 125.23745620257372 items/sec 150.30481130519595 items/sec 20.016 {'family_index': 0, 'per_family_instance_index': 7, 'run_name': 'ReadFileMetadata/num_columns:100/num_row_groups:100', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 34, 'data_size': 260000.0, 'file_size': 1431533.0}
BM_ArrowBinaryDict/DecodeArrow_Dense/32768 241.412 MiB/sec 287.865 MiB/sec 19.242 {'family_index': 23, 'per_family_instance_index': 2, 'run_name': 'BM_ArrowBinaryDict/DecodeArrow_Dense/32768', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 422}
BM_ArrowBinaryPlain/DecodeArrowNonNull_Dense/1024 425.661 MiB/sec 507.194 MiB/sec 19.154 {'family_index': 18, 'per_family_instance_index': 0, 'run_name': 'BM_ArrowBinaryPlain/DecodeArrowNonNull_Dense/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 21879}
BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:4096/null_in_ten_thousand:100 31.705 MiB/sec 37.739 MiB/sec 19.030 {'family_index': 31, 'per_family_instance_index': 5, 'run_name': 'BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:4096/null_in_ten_thousand:100', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 20286}
BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:16384/null_in_ten_thousand:1 98.431 MiB/sec 116.935 MiB/sec 18.799 {'family_index': 34, 'per_family_instance_index': 2, 'run_name': 'BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:16384/null_in_ten_thousand:1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 14448}
BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:65536/null_in_ten_thousand:1 86.626 MiB/sec 102.327 MiB/sec 18.126 {'family_index': 34, 'per_family_instance_index': 3, 'run_name': 'BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:65536/null_in_ten_thousand:1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 3102}
BM_ReadColumn<true,Int32Type>/1/1 1.707 GiB/sec 2.015 GiB/sec 18.090 {'family_index': 1, 'per_family_instance_index': 2, 'run_name': 'BM_ReadColumn<true,Int32Type>/1/1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 10}
BM_ByteStreamSplitDecode_Double_Scalar/65536 3.317 GiB/sec 3.905 GiB/sec 17.737 {'family_index': 7, 'per_family_instance_index': 1, 'run_name': 'BM_ByteStreamSplitDecode_Double_Scalar/65536', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 2109}
BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:16384/null_in_ten_thousand:5000 15.796 MiB/sec 18.536 MiB/sec 17.343 {'family_index': 31, 'per_family_instance_index': 14, 'run_name': 'BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:16384/null_in_ten_thousand:5000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 2579}
BM_ReadBinaryViewColumn/null_probability:99/unique_values:32 1.005 GiB/sec 1.163 GiB/sec 15.710 {'family_index': 15, 'per_family_instance_index': 4, 'run_name': 'BM_ReadBinaryViewColumn/null_probability:99/unique_values:32', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 4}
BM_ArrowBinaryPlain/DecodeArrow_Dict/65536 146.054 MiB/sec 168.885 MiB/sec 15.632 {'family_index': 19, 'per_family_instance_index': 3, 'run_name': 'BM_ArrowBinaryPlain/DecodeArrow_Dict/65536', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 115}
BM_ReadStructOfStructColumn/99 2.313 GiB/sec 2.665 GiB/sec 15.211 {'family_index': 19, 'per_family_instance_index': 3, 'run_name': 'BM_ReadStructOfStructColumn/99', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 31}
BM_ReadStructColumn/50 1.081 GiB/sec 1.239 GiB/sec 14.594 {'family_index': 18, 'per_family_instance_index': 2, 'run_name': 'BM_ReadStructColumn/50', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 26}
BM_ByteStreamSplitDecode_FLBA_Generic<7>/1024 3.395 GiB/sec 3.887 GiB/sec 14.499 {'family_index': 3, 'per_family_instance_index': 0, 'run_name': 'BM_ByteStreamSplitDecode_FLBA_Generic<7>/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 148315}
BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:65536/null_in_ten_thousand:1 19.130 MiB/sec 21.866 MiB/sec 14.299 {'family_index': 31, 'per_family_instance_index': 3, 'run_name': 'BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:65536/null_in_ten_thousand:1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 677}
BM_ReadBinaryViewColumnDeltaByteArray/null_probability:1/unique_values:-1 1.396 GiB/sec 1.594 GiB/sec 14.133 {'family_index': 17, 'per_family_instance_index': 1, 'run_name': 'BM_ReadBinaryViewColumnDeltaByteArray/null_probability:1/unique_values:-1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 2}
BM_ReadColumnIndex<DoubleType>/num_pages:64 206.992 MiB/sec 235.539 MiB/sec 13.791 {'family_index': 2, 'per_family_instance_index': 1, 'run_name': 'BM_ReadColumnIndex<DoubleType>/num_pages:64', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 52424}
BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:65536/null_in_ten_thousand:10000 330.878 MiB/sec 376.257 MiB/sec 13.715 {'family_index': 34, 'per_family_instance_index': 19, 'run_name': 'BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:65536/null_in_ten_thousand:10000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 12938}
BM_DeltaBitPackingDecode_Int64_Wide/4096 4.297 GiB/sec 4.883 GiB/sec 13.646 {'family_index': 16, 'per_family_instance_index': 1, 'run_name': 'BM_DeltaBitPackingDecode_Int64_Wide/4096', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 36230}
BM_ReadStructColumn/99 2.776 GiB/sec 3.152 GiB/sec 13.568 {'family_index': 18, 'per_family_instance_index': 3, 'run_name': 'BM_ReadStructColumn/99', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 66}
BM_ReadListColumn/99 1.342 GiB/sec 1.519 GiB/sec 13.189 {'family_index': 21, 'per_family_instance_index': 3, 'run_name': 'BM_ReadListColumn/99', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 49}
BM_ByteStreamSplitDecode_Int16_Sse2/1024 47.387 GiB/sec 53.603 GiB/sec 13.116 {'family_index': 8, 'per_family_instance_index': 0, 'run_name': 'BM_ByteStreamSplitDecode_Int16_Sse2/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 6810813}
BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:1024/null_in_ten_thousand:100 30.905 MiB/sec 34.806 MiB/sec 12.624 {'family_index': 31, 'per_family_instance_index': 4, 'run_name': 'BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:1024/null_in_ten_thousand:100', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 67875}
BM_ByteStreamSplitDecode_Float_Sse2/1024 16.383 GiB/sec 18.397 GiB/sec 12.295 {'family_index': 9, 'per_family_instance_index': 0, 'run_name': 'BM_ByteStreamSplitDecode_Float_Sse2/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 1135427}
BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:16384/null_in_ten_thousand:1 21.054 MiB/sec 23.577 MiB/sec 11.984 {'family_index': 31, 'per_family_instance_index': 2, 'run_name': 'BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:16384/null_in_ten_thousand:1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 2986}
BM_DeltaBitPackingDecode_Int64_Wide/1024 4.188 GiB/sec 4.680 GiB/sec 11.747 {'family_index': 16, 'per_family_instance_index': 0, 'run_name': 'BM_DeltaBitPackingDecode_Int64_Wide/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 163247}
BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:16384/null_in_ten_thousand:5000 17.635 MiB/sec 19.673 MiB/sec 11.556 {'family_index': 34, 'per_family_instance_index': 14, 'run_name': 'BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:16384/null_in_ten_thousand:5000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 2366}
BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:65536/null_in_ten_thousand:1000 16.714 MiB/sec 18.638 MiB/sec 11.509 {'family_index': 34, 'per_family_instance_index': 11, 'run_name': 'BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:65536/null_in_ten_thousand:1000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 645}
BM_ReadColumn<true,Int32Type>/25/5 858.421 MiB/sec 952.653 MiB/sec 10.977 {'family_index': 1, 'per_family_instance_index': 4, 'run_name': 'BM_ReadColumn<true,Int32Type>/25/5', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 5}
BM_ReadMultipleRowGroups 2.231 GiB/sec 2.470 GiB/sec 10.714 {'family_index': 25, 'per_family_instance_index': 0, 'run_name': 'BM_ReadMultipleRowGroups', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 9}
BM_ReadBinaryViewColumn/null_probability:50/unique_values:-1 881.249 MiB/sec 974.177 MiB/sec 10.545 {'family_index': 15, 'per_family_instance_index': 6, 'run_name': 'BM_ReadBinaryViewColumn/null_probability:50/unique_values:-1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 2}
BM_ReadStructColumn/0 5.492 GiB/sec 6.062 GiB/sec 10.388 {'family_index': 18, 'per_family_instance_index': 0, 'run_name': 'BM_ReadStructColumn/0', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 128}
BM_ReadListOfStructColumn/50 585.053 MiB/sec 645.676 MiB/sec 10.362 {'family_index': 22, 'per_family_instance_index': 2, 'run_name': 'BM_ReadListOfStructColumn/50', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 14}
BM_ReadInt64Column<Repetition::OPTIONAL, Compression::ZSTD>/1024/16 533.080 MiB/sec 587.484 MiB/sec 10.206 {'family_index': 14, 'per_family_instance_index': 0, 'run_name': 'BM_ReadInt64Column<Repetition::OPTIONAL, Compression::ZSTD>/1024/16', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 15276, 'compression_ratio': 2.390428946600525}
BM_ArrowBinaryPlain/DecodeArrow_Dict/1024 168.719 MiB/sec 185.352 MiB/sec 9.858 {'family_index': 19, 'per_family_instance_index': 0, 'run_name': 'BM_ArrowBinaryPlain/DecodeArrow_Dict/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 8332}
BM_ArrowBinaryDict/DecodeArrow_Dict/65536 147.801 MiB/sec 161.938 MiB/sec 9.565 {'family_index': 25, 'per_family_instance_index': 3, 'run_name': 'BM_ArrowBinaryDict/DecodeArrow_Dict/65536', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 115}
RecordReaderReadAndSkipRecords/Repetition:2/BatchSize:10000/LevelsPerPage:1000000 612.425 MiB/sec 669.961 MiB/sec 9.395 {'family_index': 4, 'per_family_instance_index': 8, 'run_name': 'RecordReaderReadAndSkipRecords/Repetition:2/BatchSize:10000/LevelsPerPage:1000000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 6}
BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:65536/null_in_ten_thousand:5000 14.346 MiB/sec 15.647 MiB/sec 9.073 {'family_index': 31, 'per_family_instance_index': 15, 'run_name': 'BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:65536/null_in_ten_thousand:5000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 470}
BM_ReadStructOfListColumn/50 471.756 MiB/sec 513.596 MiB/sec 8.869 {'family_index': 20, 'per_family_instance_index': 2, 'run_name': 'BM_ReadStructOfListColumn/50', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 10}
BM_ReadStructColumn/1 1.633 GiB/sec 1.778 GiB/sec 8.854 {'family_index': 18, 'per_family_instance_index': 1, 'run_name': 'BM_ReadStructColumn/1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 33}
BM_ReadInt64Column<Repetition::OPTIONAL>/1024/1024 4.963 GiB/sec 5.384 GiB/sec 8.470 {'family_index': 1, 'per_family_instance_index': 1, 'run_name': 'BM_ReadInt64Column<Repetition::OPTIONAL>/1024/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 155550, 'compression_ratio': 0.9909277851699528}
BM_ReadInt64Column<Repetition::REQUIRED>/1024/16 3.324 GiB/sec 3.605 GiB/sec 8.461 {'family_index': 0, 'per_family_instance_index': 0, 'run_name': 'BM_ReadInt64Column<Repetition::REQUIRED>/1024/16', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 120616, 'compression_ratio': 0.9917675544794189}
BM_ReadListColumn/1 976.874 MiB/sec 1.033 GiB/sec 8.287 {'family_index': 21, 'per_family_instance_index': 1, 'run_name': 'BM_ReadListColumn/1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 33}
BM_ArrowBinaryDict/DecodeArrow_Dict/32768 152.868 MiB/sec 165.490 MiB/sec 8.257 {'family_index': 25, 'per_family_instance_index': 2, 'run_name': 'BM_ArrowBinaryDict/DecodeArrow_Dict/32768', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 221}
BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:16384/null_in_ten_thousand:100 40.854 MiB/sec 44.224 MiB/sec 8.248 {'family_index': 34, 'per_family_instance_index': 6, 'run_name': 'BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:16384/null_in_ten_thousand:100', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 5942}
BM_ReadInt64Column<Repetition::OPTIONAL, Compression::LZ4>/1024/1024 2.039 GiB/sec 2.206 GiB/sec 8.206 {'family_index': 9, 'per_family_instance_index': 1, 'run_name': 'BM_ReadInt64Column<Repetition::OPTIONAL, Compression::LZ4>/1024/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 62994, 'compression_ratio': 1.6028174525533163}
BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:65536/null_in_ten_thousand:100 19.510 MiB/sec 21.102 MiB/sec 8.159 {'family_index': 31, 'per_family_instance_index': 7, 'run_name': 'BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:65536/null_in_ten_thousand:100', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 727}
BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:4096/null_in_ten_thousand:1 1.028 GiB/sec 1.109 GiB/sec 7.885 {'family_index': 34, 'per_family_instance_index': 1, 'run_name': 'BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:4096/null_in_ten_thousand:1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 622392}
BM_DeltaBitPackingDecode_Int64_Fixed/32768 7.291 GiB/sec 7.860 GiB/sec 7.811 {'family_index': 12, 'per_family_instance_index': 2, 'run_name': 'BM_DeltaBitPackingDecode_Int64_Fixed/32768', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 9125}
BM_ReadInt64Column<Repetition::REQUIRED>/65536/1024 54.815 GiB/sec 58.994 GiB/sec 7.624 {'family_index': 0, 'per_family_instance_index': 2, 'run_name': 'BM_ReadInt64Column<Repetition::REQUIRED>/65536/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 33269, 'compression_ratio': 0.9998684102624739}
BM_ArrowBinaryViewPlain/DecodeArrow_Dense/4096 272.694 MiB/sec 292.966 MiB/sec 7.434 {'family_index': 21, 'per_family_instance_index': 1, 'run_name': 'BM_ArrowBinaryViewPlain/DecodeArrow_Dense/4096', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 3478}
BM_DeltaBitPackingDecode_Int64_Wide/65536 4.490 GiB/sec 4.821 GiB/sec 7.381 {'family_index': 16, 'per_family_instance_index': 3, 'run_name': 'BM_DeltaBitPackingDecode_Int64_Wide/65536', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 2690}
BM_DeltaBitPackingDecode_Int32_Wide/4096 2.347 GiB/sec 2.519 GiB/sec 7.359 {'family_index': 15, 'per_family_instance_index': 1, 'run_name': 'BM_DeltaBitPackingDecode_Int32_Wide/4096', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 40179}
BM_DeltaBitPackingDecode_Int32_Wide/65536 2.382 GiB/sec 2.557 GiB/sec 7.343 {'family_index': 15, 'per_family_instance_index': 3, 'run_name': 'BM_DeltaBitPackingDecode_Int32_Wide/65536', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 2700}
BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:1024/null_in_ten_thousand:1000 31.754 MiB/sec 34.030 MiB/sec 7.166 {'family_index': 31, 'per_family_instance_index': 8, 'run_name': 'BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:1024/null_in_ten_thousand:1000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 74710}
BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:1024/null_in_ten_thousand:10000 163.945 MiB/sec 175.648 MiB/sec 7.138 {'family_index': 34, 'per_family_instance_index': 16, 'run_name': 'BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:1024/null_in_ten_thousand:10000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 379784}
BM_ReadInt64Column<Repetition::OPTIONAL, Compression::SNAPPY>/65536/1024 640.804 MiB/sec 686.327 MiB/sec 7.104 {'family_index': 4, 'per_family_instance_index': 2, 'run_name': 'BM_ReadInt64Column<Repetition::OPTIONAL, Compression::SNAPPY>/65536/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 280, 'compression_ratio': 1.622243468951412}
BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:4096/null_in_ten_thousand:5000 28.285 MiB/sec 30.211 MiB/sec 6.808 {'family_index': 34, 'per_family_instance_index': 13, 'run_name': 'BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:4096/null_in_ten_thousand:5000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 13713}
BM_DeltaBitPackingDecode_Int32_Wide/32768 2.382 GiB/sec 2.544 GiB/sec 6.781 {'family_index': 15, 'per_family_instance_index': 2, 'run_name': 'BM_DeltaBitPackingDecode_Int32_Wide/32768', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 5489}
BM_ReadColumnIndex<DoubleType>/num_pages:8 129.526 MiB/sec 138.297 MiB/sec 6.771 {'family_index': 2, 'per_family_instance_index': 0, 'run_name': 'BM_ReadColumnIndex<DoubleType>/num_pages:8', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 217882}
BM_ReadBinaryColumn/null_probability:50/unique_values:32 513.068 MiB/sec 547.617 MiB/sec 6.734 {'family_index': 14, 'per_family_instance_index': 3, 'run_name': 'BM_ReadBinaryColumn/null_probability:50/unique_values:32', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 1}
BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:4096/null_in_ten_thousand:1000 34.219 MiB/sec 36.522 MiB/sec 6.731 {'family_index': 31, 'per_family_instance_index': 9, 'run_name': 'BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:4096/null_in_ten_thousand:1000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 19600}
BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:65536/null_in_ten_thousand:1000 18.478 MiB/sec 19.720 MiB/sec 6.719 {'family_index': 31, 'per_family_instance_index': 11, 'run_name': 'BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:65536/null_in_ten_thousand:1000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 642}
BM_ReadColumn<false,Int32Type>/-1/1 7.046 GiB/sec 7.511 GiB/sec 6.612 {'family_index': 0, 'per_family_instance_index': 0, 'run_name': 'BM_ReadColumn<false,Int32Type>/-1/1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 42}
BM_ReadColumn<false,Int32Type>/-1/10 7.616 GiB/sec 8.120 GiB/sec 6.611 {'family_index': 0, 'per_family_instance_index': 1, 'run_name': 'BM_ReadColumn<false,Int32Type>/-1/10', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 53}
BM_ArrowBinaryPlain/DecodeArrow_Dict/32768 164.763 MiB/sec 175.566 MiB/sec 6.557 {'family_index': 19, 'per_family_instance_index': 2, 'run_name': 'BM_ArrowBinaryPlain/DecodeArrow_Dict/32768', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 251}
BM_ReadBinaryViewColumnDeltaByteArray/null_probability:50/unique_values:-1 923.664 MiB/sec 982.050 MiB/sec 6.321 {'family_index': 17, 'per_family_instance_index': 2, 'run_name': 'BM_ReadBinaryViewColumnDeltaByteArray/null_probability:50/unique_values:-1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 2}
BM_ByteStreamSplitDecode_Double_Scalar/1024 3.819 GiB/sec 4.057 GiB/sec 6.240 {'family_index': 7, 'per_family_instance_index': 0, 'run_name': 'BM_ByteStreamSplitDecode_Double_Scalar/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 140142}
BM_ByteStreamSplitDecode_Float_Generic/65536 16.154 GiB/sec 17.157 GiB/sec 6.208 {'family_index': 0, 'per_family_instance_index': 1, 'run_name': 'BM_ByteStreamSplitDecode_Float_Generic/65536', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 19148}
BM_ArrowBinaryPlain/DecodeArrow_Dict/4096 173.929 MiB/sec 184.468 MiB/sec 6.060 {'family_index': 19, 'per_family_instance_index': 1, 'run_name': 'BM_ArrowBinaryPlain/DecodeArrow_Dict/4096', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 1920}
BM_ReadStructOfListColumn/0 750.289 MiB/sec 794.821 MiB/sec 5.935 {'family_index': 20, 'per_family_instance_index': 0, 'run_name': 'BM_ReadStructOfListColumn/0', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 17}
BM_ReadBinaryColumn/null_probability:99/unique_values:32 610.192 MiB/sec 646.358 MiB/sec 5.927 {'family_index': 14, 'per_family_instance_index': 4, 'run_name': 'BM_ReadBinaryColumn/null_probability:99/unique_values:32', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 4}
BM_ArrowBinaryPlain/DecodeArrowNonNull_Dict/1024 177.265 MiB/sec 187.712 MiB/sec 5.894 {'family_index': 20, 'per_family_instance_index': 0, 'run_name': 'BM_ArrowBinaryPlain/DecodeArrowNonNull_Dict/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 7158}
BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:16384/null_in_ten_thousand:1000 19.567 MiB/sec 20.708 MiB/sec 5.832 {'family_index': 34, 'per_family_instance_index': 10, 'run_name': 'BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:16384/null_in_ten_thousand:1000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 2796}
BM_ArrowBinaryViewDict/DecodeArrowNonNull_Dense/1024 255.861 MiB/sec 270.285 MiB/sec 5.637 {'family_index': 28, 'per_family_instance_index': 0, 'run_name': 'BM_ArrowBinaryViewDict/DecodeArrowNonNull_Dense/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 12465}
BM_ArrowBinaryDict/DecodeArrow_Dict/4096 166.653 MiB/sec 176.020 MiB/sec 5.621 {'family_index': 25, 'per_family_instance_index': 1, 'run_name': 'BM_ArrowBinaryDict/DecodeArrow_Dict/4096', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 2057}
BM_DeltaBitPackingDecode_Int64_Fixed/65536 7.704 GiB/sec 8.132 GiB/sec 5.568 {'family_index': 12, 'per_family_instance_index': 3, 'run_name': 'BM_DeltaBitPackingDecode_Int64_Fixed/65536', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 4361}
BM_ArrowBinaryDict/DecodeArrow_Dense/1024 265.005 MiB/sec 279.682 MiB/sec 5.538 {'family_index': 23, 'per_family_instance_index': 0, 'run_name': 'BM_ArrowBinaryDict/DecodeArrow_Dense/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 12893}
BM_ReadColumn<true,Int32Type>/0/1 2.705 GiB/sec 2.855 GiB/sec 5.538 {'family_index': 1, 'per_family_instance_index': 1, 'run_name': 'BM_ReadColumn<true,Int32Type>/0/1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 20}
BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:65536/null_in_ten_thousand:5000 14.540 MiB/sec 15.346 MiB/sec 5.537 {'family_index': 34, 'per_family_instance_index': 15, 'run_name': 'BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:65536/null_in_ten_thousand:5000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 529}
BM_ReadColumnIndex<Int64Type>/num_pages:1024 254.636 MiB/sec 268.481 MiB/sec 5.437 {'family_index': 1, 'per_family_instance_index': 3, 'run_name': 'BM_ReadColumnIndex<Int64Type>/num_pages:1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 3789}
BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:4096/null_in_ten_thousand:5000 34.035 MiB/sec 35.877 MiB/sec 5.411 {'family_index': 31, 'per_family_instance_index': 13, 'run_name': 'BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:4096/null_in_ten_thousand:5000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 19475}
BM_ReadColumnByteStreamSplit<true,Float16LogicalType>/null_probability:50 483.101 MiB/sec 509.182 MiB/sec 5.399 {'family_index': 13, 'per_family_instance_index': 2, 'run_name': 'BM_ReadColumnByteStreamSplit<true,Float16LogicalType>/null_probability:50', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 7}
BM_ArrowBinaryDict/DecodeArrow_Dense/65536 271.642 MiB/sec 286.163 MiB/sec 5.346 {'family_index': 23, 'per_family_instance_index': 3, 'run_name': 'BM_ArrowBinaryDict/DecodeArrow_Dense/65536', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 176}
BM_ByteStreamSplitDecode_Int16_Scalar/1024 3.856 GiB/sec 4.056 GiB/sec 5.187 {'family_index': 5, 'per_family_instance_index': 0, 'run_name': 'BM_ByteStreamSplitDecode_Int16_Scalar/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 567250}
BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:16384/null_in_ten_thousand:1000 20.476 MiB/sec 21.535 MiB/sec 5.173 {'family_index': 31, 'per_family_instance_index': 10, 'run_name': 'BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:16384/null_in_ten_thousand:1000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 2935}
BM_ArrowBinaryDict/DecodeArrowNonNull_Dict/65536 156.009 MiB/sec 164.045 MiB/sec 5.151 {'family_index': 26, 'per_family_instance_index': 3, 'run_name': 'BM_ArrowBinaryDict/DecodeArrowNonNull_Dict/65536', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 117}
BM_ArrowBinaryPlain/DecodeArrowNonNull_Dense/32768 439.196 MiB/sec 461.538 MiB/sec 5.087 {'family_index': 18, 'per_family_instance_index': 2, 'run_name': 'BM_ArrowBinaryPlain/DecodeArrowNonNull_Dense/32768', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 691}
BM_ArrowBinaryViewDict/DecodeArrow_Dense/65536 265.656 MiB/sec 278.999 MiB/sec 5.023 {'family_index': 27, 'per_family_instance_index': 3, 'run_name': 'BM_ArrowBinaryViewDict/DecodeArrow_Dense/65536', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 201}
BM_DeltaBitPackingDecode_Int64_Narrow/1024 5.391 GiB/sec 5.661 GiB/sec 5.017 {'family_index': 14, 'per_family_instance_index': 0, 'run_name': 'BM_DeltaBitPackingDecode_Int64_Narrow/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 192743}
BM_ArrowBinaryDict/DecodeArrowNonNull_Dict/1024 167.112 MiB/sec 175.303 MiB/sec 4.901 {'family_index': 26, 'per_family_instance_index': 0, 'run_name': 'BM_ArrowBinaryDict/DecodeArrowNonNull_Dict/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 8066}
BM_ArrowBinaryPlain/DecodeArrow_Dense/65536 439.476 MiB/sec 461.006 MiB/sec 4.899 {'family_index': 17, 'per_family_instance_index': 3, 'run_name': 'BM_ArrowBinaryPlain/DecodeArrow_Dense/65536', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 339}
BM_ByteStreamSplitDecode_FLBA_Generic<16>/65536 3.671 GiB/sec 3.849 GiB/sec 4.847 {'family_index': 4, 'per_family_instance_index': 1, 'run_name': 'BM_ByteStreamSplitDecode_FLBA_Generic<16>/65536', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 1059}
BM_DeltaBitPackingDecode_Int64_Wide/32768 4.633 GiB/sec 4.857 GiB/sec 4.835 {'family_index': 16, 'per_family_instance_index': 2, 'run_name': 'BM_DeltaBitPackingDecode_Int64_Wide/32768', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 5323}
BM_ReadColumnIndex<DoubleType>/num_pages:512 253.761 MiB/sec 265.684 MiB/sec 4.699 {'family_index': 2, 'per_family_instance_index': 2, 'run_name': 'BM_ReadColumnIndex<DoubleType>/num_pages:512', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 6615}
BM_ByteStreamSplitDecode_Float_Scalar/65536 3.874 GiB/sec 4.055 GiB/sec 4.678 {'family_index': 6, 'per_family_instance_index': 1, 'run_name': 'BM_ByteStreamSplitDecode_Float_Scalar/65536', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 4476}
BM_ReadInt64Column<Repetition::REQUIRED, Compression::LZ4, Encoding::BYTE_STREAM_SPLIT>/65536/1024 5.499 GiB/sec 5.755 GiB/sec 4.666 {'family_index': 11, 'per_family_instance_index': 2, 'run_name': 'BM_ReadInt64Column<Repetition::REQUIRED, Compression::LZ4, Encoding::BYTE_STREAM_SPLIT>/65536/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 3155, 'compression_ratio': 2.6369850267325887}
BM_ReadInt64Column<Repetition::OPTIONAL, Compression::SNAPPY>/1024/1024 571.909 MiB/sec 598.457 MiB/sec 4.642 {'family_index': 4, 'per_family_instance_index': 1, 'run_name': 'BM_ReadInt64Column<Repetition::OPTIONAL, Compression::SNAPPY>/1024/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 16553, 'compression_ratio': 1.5882124854594803}
BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:16384/null_in_ten_thousand:100 22.367 MiB/sec 23.400 MiB/sec 4.618 {'family_index': 31, 'per_family_instance_index': 6, 'run_name': 'BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:16384/null_in_ten_thousand:100', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 3208}
BM_ReadListColumn/50 652.391 MiB/sec 681.968 MiB/sec 4.534 {'family_index': 21, 'per_family_instance_index': 2, 'run_name': 'BM_ReadListColumn/50', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 23}
BM_ReadListOfListColumn/1 851.673 MiB/sec 890.194 MiB/sec 4.523 {'family_index': 23, 'per_family_instance_index': 1, 'run_name': 'BM_ReadListOfListColumn/1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 29}
BM_ArrowBinaryPlain/DecodeArrowNonNull_Dense/4096 460.562 MiB/sec 481.279 MiB/sec 4.498 {'family_index': 18, 'per_family_instance_index': 1, 'run_name': 'BM_ArrowBinaryPlain/DecodeArrowNonNull_Dense/4096', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 5527}
BM_ReadStructOfListColumn/99 996.816 MiB/sec 1.017 GiB/sec 4.488 {'family_index': 20, 'per_family_instance_index': 3, 'run_name': 'BM_ReadStructOfListColumn/99', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 24}
BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:1024/null_in_ten_thousand:5000 25.858 MiB/sec 27.014 MiB/sec 4.473 {'family_index': 34, 'per_family_instance_index': 12, 'run_name': 'BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:1024/null_in_ten_thousand:5000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 63685}
BM_ByteStreamSplitDecode_Double_Sse2/65536 8.297 GiB/sec 8.661 GiB/sec 4.393 {'family_index': 10, 'per_family_instance_index': 1, 'run_name': 'BM_ByteStreamSplitDecode_Double_Sse2/65536', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 4843}
BM_ReadStructOfListColumn/1 700.565 MiB/sec 730.952 MiB/sec 4.337 {'family_index': 20, 'per_family_instance_index': 1, 'run_name': 'BM_ReadStructOfListColumn/1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 15}
BM_ArrowBinaryPlain/DecodeArrow_Dense/4096 468.106 MiB/sec 487.816 MiB/sec 4.211 {'family_index': 17, 'per_family_instance_index': 1, 'run_name': 'BM_ArrowBinaryPlain/DecodeArrow_Dense/4096', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 5716}
BM_ReadColumnIndex<FLBAType>/num_pages:1024 259.200 MiB/sec 269.993 MiB/sec 4.164 {'family_index': 3, 'per_family_instance_index': 3, 'run_name': 'BM_ReadColumnIndex<FLBAType>/num_pages:1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 3871}
BM_ReadBinaryViewColumn/null_probability:1/unique_values:32 865.671 MiB/sec 901.705 MiB/sec 4.163 {'family_index': 15, 'per_family_instance_index': 2, 'run_name': 'BM_ReadBinaryViewColumn/null_probability:1/unique_values:32', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 1}
BM_ByteStreamSplitDecode_Float_Scalar/1024 3.904 GiB/sec 4.063 GiB/sec 4.081 {'family_index': 6, 'per_family_instance_index': 0, 'run_name': 'BM_ByteStreamSplitDecode_Float_Scalar/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 281209}
BM_ArrowBinaryPlain/DecodeArrowNonNull_Dict/4096 178.997 MiB/sec 186.187 MiB/sec 4.017 {'family_index': 20, 'per_family_instance_index': 1, 'run_name': 'BM_ArrowBinaryPlain/DecodeArrowNonNull_Dict/4096', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 2147}
BM_ArrowBinaryViewPlain/DecodeArrowNonNull_Dense/32768 282.386 MiB/sec 293.638 MiB/sec 3.985 {'family_index': 22, 'per_family_instance_index': 2, 'run_name': 'BM_ArrowBinaryViewPlain/DecodeArrowNonNull_Dense/32768', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 435}
RecordReaderReadRecords/Repetition:0/BatchSize:1000/ReadDense:1 33.688 GiB/sec 35.009 GiB/sec 3.921 {'family_index': 3, 'per_family_instance_index': 0, 'run_name': 'RecordReaderReadRecords/Repetition:0/BatchSize:1000/ReadDense:1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 1901}
BM_ReadBinaryViewColumnDeltaByteArray/null_probability:99/unique_values:-1 1.139 GiB/sec 1.184 GiB/sec 3.873 {'family_index': 17, 'per_family_instance_index': 3, 'run_name': 'BM_ReadBinaryViewColumnDeltaByteArray/null_probability:99/unique_values:-1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 4}
BM_ArrowBinaryViewPlain/DecodeArrowNonNull_Dense/65536 285.089 MiB/sec 296.017 MiB/sec 3.833 {'family_index': 22, 'per_family_instance_index': 3, 'run_name': 'BM_ArrowBinaryViewPlain/DecodeArrowNonNull_Dense/65536', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 210}
BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:4096/null_in_ten_thousand:1 156.412 MiB/sec 162.392 MiB/sec 3.823 {'family_index': 31, 'per_family_instance_index': 1, 'run_name': 'BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:4096/null_in_ten_thousand:1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 90789}
BM_ByteStreamSplitDecode_FLBA_Generic<2>/65536 55.455 GiB/sec 57.532 GiB/sec 3.745 {'family_index': 2, 'per_family_instance_index': 1, 'run_name': 'BM_ByteStreamSplitDecode_FLBA_Generic<2>/65536', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 121441}
BM_ReadBinaryViewColumn/null_probability:1/unique_values:-1 1.549 GiB/sec 1.607 GiB/sec 3.742 {'family_index': 15, 'per_family_instance_index': 5, 'run_name': 'BM_ReadBinaryViewColumn/null_probability:1/unique_values:-1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 2}
BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:65536/null_in_ten_thousand:100 30.279 MiB/sec 31.399 MiB/sec 3.700 {'family_index': 34, 'per_family_instance_index': 7, 'run_name': 'BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:65536/null_in_ten_thousand:100', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 1088}
BM_ArrowBinaryDict/DecodeArrow_Dict/1024 168.792 MiB/sec 175.023 MiB/sec 3.691 {'family_index': 25, 'per_family_instance_index': 0, 'run_name': 'BM_ArrowBinaryDict/DecodeArrow_Dict/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 8057}
BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:16384/null_in_ten_thousand:10000 341.889 MiB/sec 354.481 MiB/sec 3.683 {'family_index': 34, 'per_family_instance_index': 18, 'run_name': 'BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:16384/null_in_ten_thousand:10000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 49316}
BM_ArrowBinaryDict/DecodeArrowNonNull_Dense/1024 273.737 MiB/sec 283.609 MiB/sec 3.606 {'family_index': 24, 'per_family_instance_index': 0, 'run_name': 'BM_ArrowBinaryDict/DecodeArrowNonNull_Dense/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 13065}
BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:4096/null_in_ten_thousand:10000 281.710 MiB/sec 291.805 MiB/sec 3.584 {'family_index': 34, 'per_family_instance_index': 17, 'run_name': 'BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:4096/null_in_ten_thousand:10000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 154150}
BM_ReadInt64Column<Repetition::REQUIRED, Compression::LZ4>/1024/1024 1.978 GiB/sec 2.048 GiB/sec 3.555 {'family_index': 8, 'per_family_instance_index': 1, 'run_name': 'BM_ReadInt64Column<Repetition::REQUIRED, Compression::LZ4>/1024/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 72303, 'compression_ratio': 1.6053301979227905}
BM_ReadBinaryColumnDeltaByteArray/null_probability:50/unique_values:-1 635.403 MiB/sec 657.613 MiB/sec 3.495 {'family_index': 16, 'per_family_instance_index': 2, 'run_name': 'BM_ReadBinaryColumnDeltaByteArray/null_probability:50/unique_values:-1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 1}
BM_ReadInt64Column<Repetition::REPEATED, Compression::SNAPPY>/65536/1024 781.750 MiB/sec 808.704 MiB/sec 3.448 {'family_index': 5, 'per_family_instance_index': 2, 'run_name': 'BM_ReadInt64Column<Repetition::REPEATED, Compression::SNAPPY>/65536/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 290, 'compression_ratio': 1.6222033137888892}
BM_ReadBinaryViewColumn/null_probability:50/unique_values:32 729.222 MiB/sec 753.923 MiB/sec 3.387 {'family_index': 15, 'per_family_instance_index': 3, 'run_name': 'BM_ReadBinaryViewColumn/null_probability:50/unique_values:32', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 1}
BM_ArrowBinaryPlain/DecodeArrowNonNull_Dense/65536 443.049 MiB/sec 458.018 MiB/sec 3.379 {'family_index': 18, 'per_family_instance_index': 3, 'run_name': 'BM_ArrowBinaryPlain/DecodeArrowNonNull_Dense/65536', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 303}
ReadFileMetadata/num_columns:100/num_row_groups:1000 12.968771595706526 items/sec 13.405732735717823 items/sec 3.369 {'family_index': 0, 'per_family_instance_index': 8, 'run_name': 'ReadFileMetadata/num_columns:100/num_row_groups:1000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 3, 'data_size': 2600000.0, 'file_size': 14626870.0}
BM_ArrowBinaryPlain/DecodeArrowNonNull_Dict/65536 164.762 MiB/sec 170.293 MiB/sec 3.357 {'family_index': 20, 'per_family_instance_index': 3, 'run_name': 'BM_ArrowBinaryPlain/DecodeArrowNonNull_Dict/65536', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 126}
BM_ReadInt64Column<Repetition::REQUIRED, Compression::SNAPPY>/65536/1024 529.350 MiB/sec 547.079 MiB/sec 3.349 {'family_index': 3, 'per_family_instance_index': 2, 'run_name': 'BM_ReadInt64Column<Repetition::REQUIRED, Compression::SNAPPY>/65536/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 301, 'compression_ratio': 1.6223037054227585}
BM_DeltaBitPackingDecode_Int64_Narrow/32768 5.876 GiB/sec 6.071 GiB/sec 3.318 {'family_index': 14, 'per_family_instance_index': 2, 'run_name': 'BM_DeltaBitPackingDecode_Int64_Narrow/32768', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 6773}
BM_ByteStreamSplitDecode_FLBA_Generic<16>/1024 3.877 GiB/sec 4.006 GiB/sec 3.312 {'family_index': 4, 'per_family_instance_index': 0, 'run_name': 'BM_ByteStreamSplitDecode_FLBA_Generic<16>/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 71130}
BM_ArrowBinaryViewDict/DecodeArrowNonNull_Dense/4096 265.267 MiB/sec 273.993 MiB/sec 3.290 {'family_index': 28, 'per_family_instance_index': 1, 'run_name': 'BM_ArrowBinaryViewDict/DecodeArrowNonNull_Dense/4096', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 3133}
BM_ArrowBinaryViewDict/DecodeArrowNonNull_Dense/65536 265.028 MiB/sec 273.689 MiB/sec 3.268 {'family_index': 28, 'per_family_instance_index': 3, 'run_name': 'BM_ArrowBinaryViewDict/DecodeArrowNonNull_Dense/65536', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 199}
BM_ReadColumnPlain<true,Int32Type>/null_probability:1 1.970 GiB/sec 2.033 GiB/sec 3.234 {'family_index': 9, 'per_family_instance_index': 1, 'run_name': 'BM_ReadColumnPlain<true,Int32Type>/null_probability:1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 14}
BM_ReadStructOfStructColumn/1 1.455 GiB/sec 1.501 GiB/sec 3.207 {'family_index': 19, 'per_family_instance_index': 1, 'run_name': 'BM_ReadStructOfStructColumn/1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 17}
BM_ArrowBinaryPlain/DecodeArrow_Dense/1024 492.716 MiB/sec 508.233 MiB/sec 3.149 {'family_index': 17, 'per_family_instance_index': 0, 'run_name': 'BM_ArrowBinaryPlain/DecodeArrow_Dense/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 23737}
BM_ReadColumn<false,Int32Type>/-1/50 7.875 GiB/sec 8.122 GiB/sec 3.129 {'family_index': 0, 'per_family_instance_index': 2, 'run_name': 'BM_ReadColumn<false,Int32Type>/-1/50', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 55}
BM_ArrowBinaryDict/DecodeArrowNonNull_Dict/32768 160.452 MiB/sec 165.458 MiB/sec 3.120 {'family_index': 26, 'per_family_instance_index': 2, 'run_name': 'BM_ArrowBinaryDict/DecodeArrowNonNull_Dict/32768', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 240}
BM_ReadInt64Column<Repetition::REQUIRED, Compression::ZSTD>/65536/1024 643.570 MiB/sec 662.155 MiB/sec 2.888 {'family_index': 13, 'per_family_instance_index': 2, 'run_name': 'BM_ReadInt64Column<Repetition::REQUIRED, Compression::ZSTD>/65536/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 368, 'compression_ratio': 2.2907176406335337}
BM_ByteStreamSplitDecode_Int16_Scalar/65536 3.879 GiB/sec 3.990 GiB/sec 2.854 {'family_index': 5, 'per_family_instance_index': 1, 'run_name': 'BM_ByteStreamSplitDecode_Int16_Scalar/65536', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 8922}
BM_DecodeArrowBooleanRle/DecodeArrow/32768 179.594 MiB/sec 184.671 MiB/sec 2.827 {'family_index': 29, 'per_family_instance_index': 2, 'run_name': 'BM_DecodeArrowBooleanRle/DecodeArrow/32768', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 12967}
BM_ReadColumnIndex<DoubleType>/num_pages:1024 261.621 MiB/sec 268.919 MiB/sec 2.790 {'family_index': 2, 'per_family_instance_index': 3, 'run_name': 'BM_ReadColumnIndex<DoubleType>/num_pages:1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 3747}
BM_ByteStreamSplitDecode_Float_Generic/1024 16.471 GiB/sec 16.931 GiB/sec 2.789 {'family_index': 0, 'per_family_instance_index': 0, 'run_name': 'BM_ByteStreamSplitDecode_Float_Generic/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 1231133}
RecordReaderSkipRecords/Repetition:0/BatchSize:1000 54.866 GiB/sec 56.381 GiB/sec 2.762 {'family_index': 2, 'per_family_instance_index': 0, 'run_name': 'RecordReaderSkipRecords/Repetition:0/BatchSize:1000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 3167}
BM_ReadOffsetIndex/num_pages:64 209.519 MiB/sec 215.288 MiB/sec 2.753 {'family_index': 0, 'per_family_instance_index': 1, 'run_name': 'BM_ReadOffsetIndex/num_pages:64', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 80730}
BM_ArrowBinaryDict/DecodeArrowNonNull_Dict/4096 172.410 MiB/sec 177.146 MiB/sec 2.747 {'family_index': 26, 'per_family_instance_index': 1, 'run_name': 'BM_ArrowBinaryDict/DecodeArrowNonNull_Dict/4096', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 2079}
BM_ArrowBinaryViewDict/DecodeArrow_Dense/4096 265.850 MiB/sec 273.094 MiB/sec 2.725 {'family_index': 27, 'per_family_instance_index': 1, 'run_name': 'BM_ArrowBinaryViewDict/DecodeArrow_Dense/4096', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 3097}
BM_ReadInt64Column<Repetition::REQUIRED, Compression::ZSTD, Encoding::BYTE_STREAM_SPLIT>/65536/1024 4.796 GiB/sec 4.926 GiB/sec 2.708 {'family_index': 16, 'per_family_instance_index': 2, 'run_name': 'BM_ReadInt64Column<Repetition::REQUIRED, Compression::ZSTD, Encoding::BYTE_STREAM_SPLIT>/65536/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 2747, 'compression_ratio': 3.1956309732787207}
BM_ReadInt64Column<Repetition::REQUIRED, Compression::ZSTD, Encoding::BYTE_STREAM_SPLIT>/1024/1024 1.030 GiB/sec 1.057 GiB/sec 2.701 {'family_index': 16, 'per_family_instance_index': 1, 'run_name': 'BM_ReadInt64Column<Repetition::REQUIRED, Compression::ZSTD, Encoding::BYTE_STREAM_SPLIT>/1024/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 38407, 'compression_ratio': 2.737053123955897}
BM_DecodeArrowBooleanRle/DecodeArrowNonNull/65536 181.028 MiB/sec 185.876 MiB/sec 2.678 {'family_index': 30, 'per_family_instance_index': 3, 'run_name': 'BM_DecodeArrowBooleanRle/DecodeArrowNonNull/65536', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 6527}
BM_DecodeArrowBooleanRle/DecodeArrow/65536 181.297 MiB/sec 186.085 MiB/sec 2.641 {'family_index': 29, 'per_family_instance_index': 3, 'run_name': 'BM_DecodeArrowBooleanRle/DecodeArrow/65536', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 6502}
BM_ByteStreamSplitDecode_Double_Generic/1024 11.652 GiB/sec 11.957 GiB/sec 2.614 {'family_index': 1, 'per_family_instance_index': 0, 'run_name': 'BM_ByteStreamSplitDecode_Double_Generic/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 427780}
BM_ByteStreamSplitDecode_Float_Sse2/65536 17.518 GiB/sec 17.969 GiB/sec 2.575 {'family_index': 9, 'per_family_instance_index': 1, 'run_name': 'BM_ByteStreamSplitDecode_Float_Sse2/65536', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 19843}
BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:1024/null_in_ten_thousand:1 113.609 MiB/sec 116.531 MiB/sec 2.572 {'family_index': 31, 'per_family_instance_index': 0, 'run_name': 'BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:1024/null_in_ten_thousand:1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 254806}
BM_DeltaBitPackingDecode_Int32_Narrow/1024 2.504 GiB/sec 2.567 GiB/sec 2.501 {'family_index': 13, 'per_family_instance_index': 0, 'run_name': 'BM_DeltaBitPackingDecode_Int32_Narrow/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 184410}
BM_ReadBinaryViewColumn/null_probability:0/unique_values:-1 1.145 GiB/sec 1.173 GiB/sec 2.481 {'family_index': 15, 'per_family_instance_index': 1, 'run_name': 'BM_ReadBinaryViewColumn/null_probability:0/unique_values:-1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 1}
BM_ReadBinaryViewColumn/null_probability:99/unique_values:-1 1.147 GiB/sec 1.176 GiB/sec 2.465 {'family_index': 15, 'per_family_instance_index': 7, 'run_name': 'BM_ReadBinaryViewColumn/null_probability:99/unique_values:-1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 4}
BM_ReadOffsetIndex/num_pages:512 244.001 MiB/sec 250.004 MiB/sec 2.460 {'family_index': 0, 'per_family_instance_index': 2, 'run_name': 'BM_ReadOffsetIndex/num_pages:512', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 11027}
BM_ByteStreamSplitDecode_Int16_Sse2/65536 50.479 GiB/sec 51.692 GiB/sec 2.403 {'family_index': 8, 'per_family_instance_index': 1, 'run_name': 'BM_ByteStreamSplitDecode_Int16_Sse2/65536', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 117232}
BM_DeltaBitPackingDecode_Int64_Fixed/4096 7.625 GiB/sec 7.808 GiB/sec 2.402 {'family_index': 12, 'per_family_instance_index': 1, 'run_name': 'BM_DeltaBitPackingDecode_Int64_Fixed/4096', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 67201}
BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:1024/null_in_ten_thousand:1000 31.562 MiB/sec 32.314 MiB/sec 2.384 {'family_index': 34, 'per_family_instance_index': 8, 'run_name': 'BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:1024/null_in_ten_thousand:1000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 64490}
BM_ReadInt64Column<Repetition::REQUIRED, Compression::SNAPPY>/1024/16 458.784 MiB/sec 469.709 MiB/sec 2.381 {'family_index': 3, 'per_family_instance_index': 0, 'run_name': 'BM_ReadInt64Column<Repetition::REQUIRED, Compression::SNAPPY>/1024/16', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 16636, 'compression_ratio': 1.5875968992248062}
BM_ReadListOfStructColumn/0 899.909 MiB/sec 921.036 MiB/sec 2.348 {'family_index': 22, 'per_family_instance_index': 0, 'run_name': 'BM_ReadListOfStructColumn/0', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 20}
BM_ReadColumnIndex<ByteArrayType>/num_pages:64 267.003 MiB/sec 273.269 MiB/sec 2.346 {'family_index': 4, 'per_family_instance_index': 1, 'run_name': 'BM_ReadColumnIndex<ByteArrayType>/num_pages:64', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 60794}
BM_ReadColumnIndex<Int64Type>/num_pages:8 134.841 MiB/sec 137.956 MiB/sec 2.310 {'family_index': 1, 'per_family_instance_index': 0, 'run_name': 'BM_ReadColumnIndex<Int64Type>/num_pages:8', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 235945}
BM_DecodeArrowBooleanPlain/DecodeArrowNonNull/4096 1.081 GiB/sec 1.106 GiB/sec 2.304 {'family_index': 33, 'per_family_instance_index': 1, 'run_name': 'BM_DecodeArrowBooleanPlain/DecodeArrowNonNull/4096', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 639338}
BM_ReadBinaryColumn/null_probability:1/unique_values:-1 1.057 GiB/sec 1.082 GiB/sec 2.299 {'family_index': 14, 'per_family_instance_index': 5, 'run_name': 'BM_ReadBinaryColumn/null_probability:1/unique_values:-1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 2}
BM_ReadInt64Column<Repetition::OPTIONAL, Compression::SNAPPY>/1024/16 516.334 MiB/sec 528.175 MiB/sec 2.293 {'family_index': 4, 'per_family_instance_index': 0, 'run_name': 'BM_ReadInt64Column<Repetition::OPTIONAL, Compression::SNAPPY>/1024/16', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 15086, 'compression_ratio': 1.5882124854594803}
BM_ReadInt64Column<Repetition::REPEATED, Compression::LZ4>/1024/1024 2.573 GiB/sec 2.632 GiB/sec 2.274 {'family_index': 10, 'per_family_instance_index': 1, 'run_name': 'BM_ReadInt64Column<Repetition::REPEATED, Compression::LZ4>/1024/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 62783, 'compression_ratio': 1.601564027370479}
BM_ArrowBinaryDict/DecodeArrow_Dense/4096 279.599 MiB/sec 285.888 MiB/sec 2.249 {'family_index': 23, 'per_family_instance_index': 1, 'run_name': 'BM_ArrowBinaryDict/DecodeArrow_Dense/4096', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 3380}
BM_ArrowBinaryViewPlain/DecodeArrowNonNull_Dense/1024 284.069 MiB/sec 290.402 MiB/sec 2.229 {'family_index': 22, 'per_family_instance_index': 0, 'run_name': 'BM_ArrowBinaryViewPlain/DecodeArrowNonNull_Dense/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 13590}
BM_ReadInt64Column<Repetition::REQUIRED, Compression::LZ4, Encoding::BYTE_STREAM_SPLIT>/1024/1024 1.687 GiB/sec 1.723 GiB/sec 2.102 {'family_index': 11, 'per_family_instance_index': 1, 'run_name': 'BM_ReadInt64Column<Repetition::REQUIRED, Compression::LZ4, Encoding::BYTE_STREAM_SPLIT>/1024/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 61780, 'compression_ratio': 2.5672203071137574}
BM_DecodeArrowBooleanRle/DecodeArrowNonNull/32768 180.178 MiB/sec 183.946 MiB/sec 2.091 {'family_index': 30, 'per_family_instance_index': 2, 'run_name': 'BM_DecodeArrowBooleanRle/DecodeArrowNonNull/32768', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 12844}
BM_ArrowBinaryDict/DecodeArrowNonNull_Dense/65536 277.296 MiB/sec 282.944 MiB/sec 2.037 {'family_index': 24, 'per_family_instance_index': 3, 'run_name': 'BM_ArrowBinaryDict/DecodeArrowNonNull_Dense/65536', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 206}
BM_ByteStreamSplitDecode_Double_Sse2/1024 8.667 GiB/sec 8.843 GiB/sec 2.031 {'family_index': 10, 'per_family_instance_index': 0, 'run_name': 'BM_ByteStreamSplitDecode_Double_Sse2/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 318662}
BM_ReadInt64Column<Repetition::REQUIRED, Compression::LZ4>/1024/16 1.702 GiB/sec 1.737 GiB/sec 2.021 {'family_index': 8, 'per_family_instance_index': 0, 'run_name': 'BM_ReadInt64Column<Repetition::REQUIRED, Compression::LZ4>/1024/16', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 62080, 'compression_ratio': 1.6053301979227905}
BM_ReadInt64Column<Repetition::REQUIRED, Compression::SNAPPY, Encoding::BYTE_STREAM_SPLIT>/1024/16 1.797 GiB/sec 1.833 GiB/sec 2.012 {'family_index': 6, 'per_family_instance_index': 0, 'run_name': 'BM_ReadInt64Column<Repetition::REQUIRED, Compression::SNAPPY, Encoding::BYTE_STREAM_SPLIT>/1024/16', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 63679, 'compression_ratio': 2.302417088251827}
ReadFileMetadata/num_columns:1/num_row_groups:1000 1.293K items/sec 1.319K items/sec 2.009 {'family_index': 0, 'per_family_instance_index': 2, 'run_name': 'ReadFileMetadata/num_columns:1/num_row_groups:1000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 361, 'data_size': 26000.0, 'file_size': 154142.0}
BM_ReadInt64Column<Repetition::REQUIRED, Compression::LZ4>/65536/1024 3.288 GiB/sec 3.353 GiB/sec 1.983 {'family_index': 8, 'per_family_instance_index': 2, 'run_name': 'BM_ReadInt64Column<Repetition::REQUIRED, Compression::LZ4>/65536/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 1887, 'compression_ratio': 1.696000414062601}
ReadFileMetadata/num_columns:1/num_row_groups:1 287.861K items/sec 293.530K items/sec 1.969 {'family_index': 0, 'per_family_instance_index': 0, 'run_name': 'ReadFileMetadata/num_columns:1/num_row_groups:1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 80328, 'data_size': 26.0, 'file_size': 238.0}
BM_DeltaBitPackingDecode_Int64_Narrow/4096 5.850 GiB/sec 5.961 GiB/sec 1.888 {'family_index': 14, 'per_family_instance_index': 1, 'run_name': 'BM_DeltaBitPackingDecode_Int64_Narrow/4096', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 53333}
BM_ReadInt64Column<Repetition::REPEATED, Compression::SNAPPY>/1024/1024 696.895 MiB/sec 709.846 MiB/sec 1.858 {'family_index': 5, 'per_family_instance_index': 1, 'run_name': 'BM_ReadInt64Column<Repetition::REPEATED, Compression::SNAPPY>/1024/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 16583, 'compression_ratio': 1.5869817900038745}
BM_ReadInt64Column<Repetition::REQUIRED, Compression::ZSTD>/1024/16 518.228 MiB/sec 527.803 MiB/sec 1.848 {'family_index': 13, 'per_family_instance_index': 0, 'run_name': 'BM_ReadInt64Column<Repetition::REQUIRED, Compression::ZSTD>/1024/16', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 18566, 'compression_ratio': 2.398126463700234}
BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:16384/null_in_ten_thousand:10000 4.226 GiB/sec 4.301 GiB/sec 1.783 {'family_index': 31, 'per_family_instance_index': 18, 'run_name': 'BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:16384/null_in_ten_thousand:10000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 615243}
ReadFileMetadata/num_columns:10/num_row_groups:1 52.446K items/sec 53.377K items/sec 1.776 {'family_index': 0, 'per_family_instance_index': 3, 'run_name': 'ReadFileMetadata/num_columns:10/num_row_groups:1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 14698, 'data_size': 260.0, 'file_size': 1606.0}
BM_ReadListOfStructColumn/1 756.957 MiB/sec 770.276 MiB/sec 1.760 {'family_index': 22, 'per_family_instance_index': 1, 'run_name': 'BM_ReadListOfStructColumn/1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 18}
BM_ReadInt64Column<Repetition::REQUIRED, Compression::ZSTD, Encoding::BYTE_STREAM_SPLIT>/1024/16 921.615 MiB/sec 937.349 MiB/sec 1.707 {'family_index': 16, 'per_family_instance_index': 0, 'run_name': 'BM_ReadInt64Column<Repetition::REQUIRED, Compression::ZSTD, Encoding::BYTE_STREAM_SPLIT>/1024/16', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 33086, 'compression_ratio': 2.737053123955897}
BM_ArrowBinaryViewDict/DecodeArrow_Dense/1024 260.031 MiB/sec 264.441 MiB/sec 1.696 {'family_index': 27, 'per_family_instance_index': 0, 'run_name': 'BM_ArrowBinaryViewDict/DecodeArrow_Dense/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 12340}
BM_DecodeArrowBooleanRle/DecodeArrow/1024 115.145 MiB/sec 117.094 MiB/sec 1.693 {'family_index': 29, 'per_family_instance_index': 0, 'run_name': 'BM_DecodeArrowBooleanRle/DecodeArrow/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 263094}
BM_ArrowBinaryViewDict/DecodeArrowNonNull_Dense/32768 271.754 MiB/sec 276.303 MiB/sec 1.674 {'family_index': 28, 'per_family_instance_index': 2, 'run_name': 'BM_ArrowBinaryViewDict/DecodeArrowNonNull_Dense/32768', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 407}
BM_ReadColumn<false,Int64Type>/-1/50 7.864 GiB/sec 7.992 GiB/sec 1.624 {'family_index': 2, 'per_family_instance_index': 2, 'run_name': 'BM_ReadColumn<false,Int64Type>/-1/50', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 29}
BM_ReadBinaryColumnDeltaByteArray/null_probability:0/unique_values:-1 1.185 GiB/sec 1.205 GiB/sec 1.618 {'family_index': 16, 'per_family_instance_index': 0, 'run_name': 'BM_ReadBinaryColumnDeltaByteArray/null_probability:0/unique_values:-1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 2}
BM_ReadBinaryColumnDeltaByteArray/null_probability:1/unique_values:-1 1.080 GiB/sec 1.097 GiB/sec 1.600 {'family_index': 16, 'per_family_instance_index': 1, 'run_name': 'BM_ReadBinaryColumnDeltaByteArray/null_probability:1/unique_values:-1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 2}
BM_ReadColumnIndex<Int64Type>/num_pages:64 231.673 MiB/sec 235.358 MiB/sec 1.591 {'family_index': 1, 'per_family_instance_index': 1, 'run_name': 'BM_ReadColumnIndex<Int64Type>/num_pages:64', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 53063}
BM_ByteStreamSplitDecode_Double_Generic/65536 11.437 GiB/sec 11.615 GiB/sec 1.552 {'family_index': 1, 'per_family_instance_index': 1, 'run_name': 'BM_ByteStreamSplitDecode_Double_Generic/65536', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 6568}
BM_DeltaBitPackingDecode_Int64_Fixed/1024 6.805 GiB/sec 6.910 GiB/sec 1.552 {'family_index': 12, 'per_family_instance_index': 0, 'run_name': 'BM_DeltaBitPackingDecode_Int64_Fixed/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 249620}
BM_ArrowBinaryViewPlain/DecodeArrowNonNull_Dense/4096 286.566 MiB/sec 290.997 MiB/sec 1.546 {'family_index': 22, 'per_family_instance_index': 1, 'run_name': 'BM_ArrowBinaryViewPlain/DecodeArrowNonNull_Dense/4096', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 3493}
BM_ReadColumnIndex<Int64Type>/num_pages:512 261.897 MiB/sec 265.894 MiB/sec 1.526 {'family_index': 1, 'per_family_instance_index': 2, 'run_name': 'BM_ReadColumnIndex<Int64Type>/num_pages:512', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 7532}
BM_ReadColumn<false,Int64Type>/-1/1 7.993 GiB/sec 8.114 GiB/sec 1.510 {'family_index': 2, 'per_family_instance_index': 0, 'run_name': 'BM_ReadColumn<false,Int64Type>/-1/1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 28}
BM_ByteStreamSplitDecode_FLBA_Generic<7>/65536 3.806 GiB/sec 3.863 GiB/sec 1.501 {'family_index': 3, 'per_family_instance_index': 1, 'run_name': 'BM_ByteStreamSplitDecode_FLBA_Generic<7>/65536', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 2491}
BM_ReadOffsetIndex/num_pages:1024 249.583 MiB/sec 253.313 MiB/sec 1.494 {'family_index': 0, 'per_family_instance_index': 3, 'run_name': 'BM_ReadOffsetIndex/num_pages:1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 5516}
BM_ReadBinaryColumnDeltaByteArray/null_probability:99/unique_values:-1 644.030 MiB/sec 653.651 MiB/sec 1.494 {'family_index': 16, 'per_family_instance_index': 3, 'run_name': 'BM_ReadBinaryColumnDeltaByteArray/null_probability:99/unique_values:-1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 4}
BM_ReadInt64Column<Repetition::OPTIONAL, Compression::ZSTD>/65536/1024 806.954 MiB/sec 818.815 MiB/sec 1.470 {'family_index': 14, 'per_family_instance_index': 2, 'run_name': 'BM_ReadInt64Column<Repetition::OPTIONAL, Compression::ZSTD>/65536/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 362, 'compression_ratio': 2.2905174861836213}
BM_ReadColumnIndex<FLBAType>/num_pages:512 267.381 MiB/sec 271.305 MiB/sec 1.468 {'family_index': 3, 'per_family_instance_index': 2, 'run_name': 'BM_ReadColumnIndex<FLBAType>/num_pages:512', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 7642}
BM_ReadBinaryColumn/null_probability:0/unique_values:-1 850.617 MiB/sec 862.807 MiB/sec 1.433 {'family_index': 14, 'per_family_instance_index': 1, 'run_name': 'BM_ReadBinaryColumn/null_probability:0/unique_values:-1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 1}
BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:1024/null_in_ten_thousand:1 310.201 MiB/sec 314.609 MiB/sec 1.421 {'family_index': 34, 'per_family_instance_index': 0, 'run_name': 'BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:1024/null_in_ten_thousand:1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 721099}
BM_ReadInt64Column<Repetition::REQUIRED, Compression::SNAPPY, Encoding::BYTE_STREAM_SPLIT>/1024/1024 2.404 GiB/sec 2.438 GiB/sec 1.419 {'family_index': 6, 'per_family_instance_index': 1, 'run_name': 'BM_ReadInt64Column<Repetition::REQUIRED, Compression::SNAPPY, Encoding::BYTE_STREAM_SPLIT>/1024/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 80883, 'compression_ratio': 2.302417088251827}
BM_ArrowBinaryDict/DecodeArrowNonNull_Dense/32768 284.534 MiB/sec 288.487 MiB/sec 1.389 {'family_index': 24, 'per_family_instance_index': 2, 'run_name': 'BM_ArrowBinaryDict/DecodeArrowNonNull_Dense/32768', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 424}
BM_ReadColumnIndex<FLBAType>/num_pages:64 235.350 MiB/sec 238.577 MiB/sec 1.371 {'family_index': 3, 'per_family_instance_index': 1, 'run_name': 'BM_ReadColumnIndex<FLBAType>/num_pages:64', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 51275}
BM_ReadListOfStructColumn/99 1.128 GiB/sec 1.144 GiB/sec 1.369 {'family_index': 22, 'per_family_instance_index': 3, 'run_name': 'BM_ReadListOfStructColumn/99', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 27}
BM_DecodeArrowBooleanRle/DecodeArrow/4096 161.300 MiB/sec 163.472 MiB/sec 1.346 {'family_index': 29, 'per_family_instance_index': 1, 'run_name': 'BM_DecodeArrowBooleanRle/DecodeArrow/4096', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 92048}
BM_ReadColumnIndex<ByteArrayType>/num_pages:1024 318.491 MiB/sec 322.742 MiB/sec 1.335 {'family_index': 4, 'per_family_instance_index': 3, 'run_name': 'BM_ReadColumnIndex<ByteArrayType>/num_pages:1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 4558}
BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:65536/null_in_ten_thousand:10000 12.068 GiB/sec 12.227 GiB/sec 1.319 {'family_index': 31, 'per_family_instance_index': 19, 'run_name': 'BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:65536/null_in_ten_thousand:10000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 446543}
BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:4096/null_in_ten_thousand:100 54.140 MiB/sec 54.848 MiB/sec 1.308 {'family_index': 34, 'per_family_instance_index': 5, 'run_name': 'BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:4096/null_in_ten_thousand:100', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 31199}
ReadLevels_BitPack/MaxLevel:1/NumLevels:8096/BatchSize:1024/LevelRepeatCount:7 7.859 GiB/sec 7.961 GiB/sec 1.296 {'family_index': 6, 'per_family_instance_index': 1, 'run_name': 'ReadLevels_BitPack/MaxLevel:1/NumLevels:8096/BatchSize:1024/LevelRepeatCount:7', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 157097}
BM_ReadInt64Column<Repetition::REPEATED, Compression::LZ4>/65536/1024 4.415 GiB/sec 4.472 GiB/sec 1.293 {'family_index': 10, 'per_family_instance_index': 2, 'run_name': 'BM_ReadInt64Column<Repetition::REPEATED, Compression::LZ4>/65536/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 1686, 'compression_ratio': 1.695929094761358}
BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:1024/null_in_ten_thousand:100 42.561 MiB/sec 43.091 MiB/sec 1.246 {'family_index': 34, 'per_family_instance_index': 4, 'run_name': 'BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:1024/null_in_ten_thousand:100', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 99050}
BM_ReadInt64Column<Repetition::REQUIRED, Compression::SNAPPY>/1024/1024 481.007 MiB/sec 486.876 MiB/sec 1.220 {'family_index': 3, 'per_family_instance_index': 1, 'run_name': 'BM_ReadInt64Column<Repetition::REQUIRED, Compression::SNAPPY>/1024/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 17287, 'compression_ratio': 1.5875968992248062}
BM_ArrowBinaryViewDict/DecodeArrow_Dense/32768 267.287 MiB/sec 270.417 MiB/sec 1.171 {'family_index': 27, 'per_family_instance_index': 2, 'run_name': 'BM_ArrowBinaryViewDict/DecodeArrow_Dense/32768', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 402}
BM_ReadInt64Column<Repetition::OPTIONAL, Compression::SNAPPY, Encoding::BYTE_STREAM_SPLIT>/1024/1024 2.774 GiB/sec 2.806 GiB/sec 1.164 {'family_index': 7, 'per_family_instance_index': 1, 'run_name': 'BM_ReadInt64Column<Repetition::OPTIONAL, Compression::SNAPPY, Encoding::BYTE_STREAM_SPLIT>/1024/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 82330, 'compression_ratio': 2.3017701601573477}
BM_DecodeArrowBooleanRle/DecodeArrowNonNull/4096 160.789 MiB/sec 162.645 MiB/sec 1.154 {'family_index': 30, 'per_family_instance_index': 1, 'run_name': 'BM_DecodeArrowBooleanRle/DecodeArrowNonNull/4096', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 92463}
ReadFileMetadata/num_columns:1/num_row_groups:100 12.530K items/sec 12.667K items/sec 1.101 {'family_index': 0, 'per_family_instance_index': 1, 'run_name': 'ReadFileMetadata/num_columns:1/num_row_groups:100', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 3557, 'data_size': 2600.0, 'file_size': 15086.0}
BM_ArrowBinaryDict/DecodeArrowNonNull_Dense/4096 283.755 MiB/sec 286.848 MiB/sec 1.090 {'family_index': 24, 'per_family_instance_index': 1, 'run_name': 'BM_ArrowBinaryDict/DecodeArrowNonNull_Dense/4096', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 3363}
BM_ReadColumnIndex<FLBAType>/num_pages:8 133.693 MiB/sec 135.149 MiB/sec 1.089 {'family_index': 3, 'per_family_instance_index': 0, 'run_name': 'BM_ReadColumnIndex<FLBAType>/num_pages:8', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 229168}
ReadFileMetadata/num_columns:10/num_row_groups:1000 147.3626175249572 items/sec 148.96179850914265 items/sec 1.085 {'family_index': 0, 'per_family_instance_index': 5, 'run_name': 'ReadFileMetadata/num_columns:10/num_row_groups:1000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 40, 'data_size': 260000.0, 'file_size': 1434553.0}
ReadFileMetadata/num_columns:1000/num_row_groups:100 12.98194299033613 items/sec 13.120842642557482 items/sec 1.070 {'family_index': 0, 'per_family_instance_index': 10, 'run_name': 'ReadFileMetadata/num_columns:1000/num_row_groups:100', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 4, 'data_size': 2600000.0, 'file_size': 14725637.0}
BM_ReadColumn<true,Int64Type>/99/0 3.867 GiB/sec 3.908 GiB/sec 1.066 {'family_index': 3, 'per_family_instance_index': 12, 'run_name': 'BM_ReadColumn<true,Int64Type>/99/0', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 14}
BM_ArrowBinaryPlain/DecodeArrowNonNull_Dict/32768 172.952 MiB/sec 174.786 MiB/sec 1.061 {'family_index': 20, 'per_family_instance_index': 2, 'run_name': 'BM_ArrowBinaryPlain/DecodeArrowNonNull_Dict/32768', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 253}
BM_ReadInt64Column<Repetition::OPTIONAL, Compression::LZ4, Encoding::BYTE_STREAM_SPLIT>/65536/1024 2.850 GiB/sec 2.879 GiB/sec 1.000 {'family_index': 12, 'per_family_instance_index': 2, 'run_name': 'BM_ReadInt64Column<Repetition::OPTIONAL, Compression::LZ4, Encoding::BYTE_STREAM_SPLIT>/65536/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 1308, 'compression_ratio': 2.637794324813846}
BM_ReadInt64Column<Repetition::OPTIONAL, Compression::LZ4>/65536/1024 3.755 GiB/sec 3.792 GiB/sec 0.971 {'family_index': 9, 'per_family_instance_index': 2, 'run_name': 'BM_ReadInt64Column<Repetition::OPTIONAL, Compression::LZ4>/65536/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 1721, 'compression_ratio': 1.6959620106165836}
BM_ReadInt64Column<Repetition::REPEATED, Compression::ZSTD>/65536/1024 963.554 MiB/sec 972.896 MiB/sec 0.970 {'family_index': 15, 'per_family_instance_index': 2, 'run_name': 'BM_ReadInt64Column<Repetition::REPEATED, Compression::ZSTD>/65536/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 356, 'compression_ratio': 2.290447440389337}
BM_ReadInt64Column<Repetition::OPTIONAL, Compression::LZ4, Encoding::BYTE_STREAM_SPLIT>/1024/1024 1.966 GiB/sec 1.985 GiB/sec 0.968 {'family_index': 12, 'per_family_instance_index': 1, 'run_name': 'BM_ReadInt64Column<Repetition::OPTIONAL, Compression::LZ4, Encoding::BYTE_STREAM_SPLIT>/1024/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 57801, 'compression_ratio': 2.561601000625391}
BM_DeltaBitPackingDecode_Int32_Narrow/4096 2.545 GiB/sec 2.568 GiB/sec 0.923 {'family_index': 13, 'per_family_instance_index': 1, 'run_name': 'BM_DeltaBitPackingDecode_Int32_Narrow/4096', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 47042}
BM_ReadInt64Column<Repetition::REQUIRED, Compression::SNAPPY, Encoding::BYTE_STREAM_SPLIT>/65536/1024 2.020 GiB/sec 2.039 GiB/sec 0.900 {'family_index': 6, 'per_family_instance_index': 2, 'run_name': 'BM_ReadInt64Column<Repetition::REQUIRED, Compression::SNAPPY, Encoding::BYTE_STREAM_SPLIT>/65536/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 1152, 'compression_ratio': 2.4723685389443504}
BM_ReadBinaryViewColumnDeltaByteArray/null_probability:0/unique_values:-1 1.802 GiB/sec 1.818 GiB/sec 0.896 {'family_index': 17, 'per_family_instance_index': 0, 'run_name': 'BM_ReadBinaryViewColumnDeltaByteArray/null_probability:0/unique_values:-1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 2}
BM_ReadColumn<true,Int64Type>/99/50 3.891 GiB/sec 3.925 GiB/sec 0.876 {'family_index': 3, 'per_family_instance_index': 11, 'run_name': 'BM_ReadColumn<true,Int64Type>/99/50', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 10}
ReadLevels_BitPack/MaxLevel:3/NumLevels:8096/BatchSize:1024/LevelRepeatCount:7 8.377 GiB/sec 8.446 GiB/sec 0.829 {'family_index': 6, 'per_family_instance_index': 6, 'run_name': 'ReadLevels_BitPack/MaxLevel:3/NumLevels:8096/BatchSize:1024/LevelRepeatCount:7', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 157556}
ReadFileMetadata/num_columns:10/num_row_groups:100 1.486K items/sec 1.499K items/sec 0.814 {'family_index': 0, 'per_family_instance_index': 4, 'run_name': 'ReadFileMetadata/num_columns:10/num_row_groups:100', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 409, 'data_size': 26000.0, 'file_size': 143052.0}
BM_ArrowBinaryViewPlain/DecodeArrow_Dense/65536 292.074 MiB/sec 294.447 MiB/sec 0.813 {'family_index': 21, 'per_family_instance_index': 3, 'run_name': 'BM_ArrowBinaryViewPlain/DecodeArrow_Dense/65536', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 217}
ReadLevels_BitPack/MaxLevel:3/NumLevels:8096/BatchSize:1024/LevelRepeatCount:1 8.371 GiB/sec 8.438 GiB/sec 0.802 {'family_index': 6, 'per_family_instance_index': 4, 'run_name': 'ReadLevels_BitPack/MaxLevel:3/NumLevels:8096/BatchSize:1024/LevelRepeatCount:1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 160705}
BM_ReadColumn<true,Int32Type>/10/10 978.874 MiB/sec 986.432 MiB/sec 0.772 {'family_index': 1, 'per_family_instance_index': 3, 'run_name': 'BM_ReadColumn<true,Int32Type>/10/10', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 7}
BM_ReadColumn<false,Int64Type>/-1/10 7.916 GiB/sec 7.973 GiB/sec 0.723 {'family_index': 2, 'per_family_instance_index': 1, 'run_name': 'BM_ReadColumn<false,Int64Type>/-1/10', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 29}
BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:4096/null_in_ten_thousand:1000 34.710 MiB/sec 34.946 MiB/sec 0.679 {'family_index': 34, 'per_family_instance_index': 9, 'run_name': 'BM_DecodeArrowBooleanPlain/DecodeArrowWithNull/num_values:4096/null_in_ten_thousand:1000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 19651}
BM_ArrowBinaryPlain/DecodeArrow_Dense/32768 462.152 MiB/sec 465.020 MiB/sec 0.621 {'family_index': 17, 'per_family_instance_index': 2, 'run_name': 'BM_ArrowBinaryPlain/DecodeArrow_Dense/32768', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 689}
BM_DecodeArrowBooleanPlain/DecodeArrowNonNull/65536 6.669 GiB/sec 6.707 GiB/sec 0.570 {'family_index': 33, 'per_family_instance_index': 3, 'run_name': 'BM_DecodeArrowBooleanPlain/DecodeArrowNonNull/65536', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 241757}
BM_ReadColumn<false,DoubleType>/-1/20 8.047 GiB/sec 8.092 GiB/sec 0.564 {'family_index': 4, 'per_family_instance_index': 1, 'run_name': 'BM_ReadColumn<false,DoubleType>/-1/20', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 29}
BM_ReadListOfListColumn/50 595.144 MiB/sec 598.204 MiB/sec 0.514 {'family_index': 23, 'per_family_instance_index': 2, 'run_name': 'BM_ReadListOfListColumn/50', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 21}
BM_ReadColumnByteStreamSplit<true,Float16LogicalType>/null_probability:0 1.632 GiB/sec 1.640 GiB/sec 0.506 {'family_index': 13, 'per_family_instance_index': 0, 'run_name': 'BM_ReadColumnByteStreamSplit<true,Float16LogicalType>/null_probability:0', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 23}
BM_ReadColumnPlain<true,Int32Type>/null_probability:99 2.329 GiB/sec 2.340 GiB/sec 0.493 {'family_index': 9, 'per_family_instance_index': 3, 'run_name': 'BM_ReadColumnPlain<true,Int32Type>/null_probability:99', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 17}
BM_ReadInt64Column<Repetition::REPEATED, Compression::ZSTD>/1024/1024 774.747 MiB/sec 778.268 MiB/sec 0.455 {'family_index': 15, 'per_family_instance_index': 1, 'run_name': 'BM_ReadInt64Column<Repetition::REPEATED, Compression::ZSTD>/1024/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 18444, 'compression_ratio': 2.3841676367869615}
BM_ReadColumnIndex<ByteArrayType>/num_pages:512 313.753 MiB/sec 315.024 MiB/sec 0.405 {'family_index': 4, 'per_family_instance_index': 2, 'run_name': 'BM_ReadColumnIndex<ByteArrayType>/num_pages:512', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 9027}
BM_DecodeArrowBooleanPlain/DecodeArrow/32768 4.969 GiB/sec 4.989 GiB/sec 0.400 {'family_index': 32, 'per_family_instance_index': 2, 'run_name': 'BM_DecodeArrowBooleanPlain/DecodeArrow/32768', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 366039}
BM_ReadBinaryColumn/null_probability:50/unique_values:-1 615.560 MiB/sec 617.976 MiB/sec 0.392 {'family_index': 14, 'per_family_instance_index': 6, 'run_name': 'BM_ReadBinaryColumn/null_probability:50/unique_values:-1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 1}
BM_ReadInt64Column<Repetition::OPTIONAL, Compression::SNAPPY, Encoding::BYTE_STREAM_SPLIT>/65536/1024 2.319 GiB/sec 2.326 GiB/sec 0.307 {'family_index': 7, 'per_family_instance_index': 2, 'run_name': 'BM_ReadInt64Column<Repetition::OPTIONAL, Compression::SNAPPY, Encoding::BYTE_STREAM_SPLIT>/65536/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 1059, 'compression_ratio': 2.474258720038509}
BM_ReadColumn<true,Int32Type>/99/50 2.316 GiB/sec 2.323 GiB/sec 0.303 {'family_index': 1, 'per_family_instance_index': 7, 'run_name': 'BM_ReadColumn<true,Int32Type>/99/50', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 17}
BM_ArrowBinaryViewPlain/DecodeArrow_Dense/1024 290.038 MiB/sec 290.596 MiB/sec 0.192 {'family_index': 21, 'per_family_instance_index': 0, 'run_name': 'BM_ArrowBinaryViewPlain/DecodeArrow_Dense/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 13884}
BM_ReadColumn<true,Int64Type>/5/5 2.015 GiB/sec 2.018 GiB/sec 0.168 {'family_index': 3, 'per_family_instance_index': 2, 'run_name': 'BM_ReadColumn<true,Int64Type>/5/5', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 7}
BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:1024/null_in_ten_thousand:5000 31.340 MiB/sec 31.388 MiB/sec 0.152 {'family_index': 31, 'per_family_instance_index': 12, 'run_name': 'BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:1024/null_in_ten_thousand:5000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 70826}
BM_ReadColumnPlain<false,Int32Type>/null_probability:-1 8.086 GiB/sec 8.097 GiB/sec 0.139 {'family_index': 8, 'per_family_instance_index': 0, 'run_name': 'BM_ReadColumnPlain<false,Int32Type>/null_probability:-1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 58}
ReadLevels_Rle/MaxLevel:3/NumLevels:8096/BatchSize:2048/LevelRepeatCount:1 6.584 GiB/sec 6.590 GiB/sec 0.105 {'family_index': 5, 'per_family_instance_index': 5, 'run_name': 'ReadLevels_Rle/MaxLevel:3/NumLevels:8096/BatchSize:2048/LevelRepeatCount:1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 121940}
BM_ReadBinaryColumn/null_probability:0/unique_values:32 623.627 MiB/sec 624.019 MiB/sec 0.063 {'family_index': 14, 'per_family_instance_index': 0, 'run_name': 'BM_ReadBinaryColumn/null_probability:0/unique_values:32', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 1}
BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:1024/null_in_ten_thousand:10000 328.421 MiB/sec 328.611 MiB/sec 0.058 {'family_index': 31, 'per_family_instance_index': 16, 'run_name': 'BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:1024/null_in_ten_thousand:10000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 689095}
BM_ReadColumn<true,Int64Type>/25/10 1.650 GiB/sec 1.650 GiB/sec 0.004 {'family_index': 3, 'per_family_instance_index': 4, 'run_name': 'BM_ReadColumn<true,Int64Type>/25/10', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 6}
RecordReaderReadAndSkipRecords/Repetition:0/BatchSize:10000/LevelsPerPage:1000000 15.261 GiB/sec 15.261 GiB/sec 0.002 {'family_index': 4, 'per_family_instance_index': 2, 'run_name': 'RecordReaderReadAndSkipRecords/Repetition:0/BatchSize:10000/LevelsPerPage:1000000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 71}
BM_ReadInt64Column<Repetition::REPEATED, Compression::SNAPPY>/1024/16 604.230 MiB/sec 603.727 MiB/sec -0.083 {'family_index': 5, 'per_family_instance_index': 0, 'run_name': 'BM_ReadInt64Column<Repetition::REPEATED, Compression::SNAPPY>/1024/16', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 14718, 'compression_ratio': 1.5869817900038745}
BM_ReadListColumn/0 1.123 GiB/sec 1.122 GiB/sec -0.116 {'family_index': 21, 'per_family_instance_index': 0, 'run_name': 'BM_ReadListColumn/0', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 40}
BM_ReadColumn<false,DoubleType>/-1/0 8.087 GiB/sec 8.074 GiB/sec -0.164 {'family_index': 4, 'per_family_instance_index': 0, 'run_name': 'BM_ReadColumn<false,DoubleType>/-1/0', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 29}
ReadFileMetadata/num_columns:1000/num_row_groups:1 625.9066167928996 items/sec 624.7840612275301 items/sec -0.179 {'family_index': 0, 'per_family_instance_index': 9, 'run_name': 'ReadFileMetadata/num_columns:1000/num_row_groups:1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 175, 'data_size': 26000.0, 'file_size': 159245.0}
BM_ReadColumnByteStreamSplit<false,Float16LogicalType>/null_probability:-1 7.540 GiB/sec 7.513 GiB/sec -0.363 {'family_index': 12, 'per_family_instance_index': 0, 'run_name': 'BM_ReadColumnByteStreamSplit<false,Float16LogicalType>/null_probability:-1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 107}
BM_DecodeArrowBooleanRle/DecodeArrowNonNull/1024 117.622 MiB/sec 117.124 MiB/sec -0.424 {'family_index': 30, 'per_family_instance_index': 0, 'run_name': 'BM_DecodeArrowBooleanRle/DecodeArrowNonNull/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 267954}
BM_ReadColumn<true,Int64Type>/10/5 1.733 GiB/sec 1.725 GiB/sec -0.466 {'family_index': 3, 'per_family_instance_index': 3, 'run_name': 'BM_ReadColumn<true,Int64Type>/10/5', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 6}
BM_ReadInt64Column<Repetition::OPTIONAL, Compression::ZSTD, Encoding::BYTE_STREAM_SPLIT>/1024/1024 1.258 GiB/sec 1.252 GiB/sec -0.480 {'family_index': 17, 'per_family_instance_index': 1, 'run_name': 'BM_ReadInt64Column<Repetition::OPTIONAL, Compression::ZSTD, Encoding::BYTE_STREAM_SPLIT>/1024/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 37075, 'compression_ratio': 2.7379679144385025}
BM_ByteStreamSplitDecode_FLBA_Generic<2>/1024 62.410 GiB/sec 62.102 GiB/sec -0.494 {'family_index': 2, 'per_family_instance_index': 0, 'run_name': 'BM_ByteStreamSplitDecode_FLBA_Generic<2>/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 9241419}
BM_ReadInt64Column<Repetition::REQUIRED>/1024/1024 5.023 GiB/sec 4.998 GiB/sec -0.502 {'family_index': 0, 'per_family_instance_index': 1, 'run_name': 'BM_ReadInt64Column<Repetition::REQUIRED>/1024/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 181545, 'compression_ratio': 0.9917675544794189}
BM_ReadColumn<true,Int32Type>/99/0 2.341 GiB/sec 2.328 GiB/sec -0.548 {'family_index': 1, 'per_family_instance_index': 8, 'run_name': 'BM_ReadColumn<true,Int32Type>/99/0', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 17}
BM_ReadColumn<true,Int64Type>/30/10 1.678 GiB/sec 1.669 GiB/sec -0.564 {'family_index': 3, 'per_family_instance_index': 5, 'run_name': 'BM_ReadColumn<true,Int64Type>/30/10', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 6}
RecordReaderReadAndSkipRecords/Repetition:1/BatchSize:10/LevelsPerPage:80000 160.786 MiB/sec 159.701 MiB/sec -0.675 {'family_index': 4, 'per_family_instance_index': 3, 'run_name': 'RecordReaderReadAndSkipRecords/Repetition:1/BatchSize:10/LevelsPerPage:80000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 17}
BM_ReadColumn<true,Int64Type>/35/10 1.686 GiB/sec 1.675 GiB/sec -0.680 {'family_index': 3, 'per_family_instance_index': 6, 'run_name': 'BM_ReadColumn<true,Int64Type>/35/10', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 6}
ReadFileMetadata/num_columns:100/num_row_groups:1 5.896K items/sec 5.855K items/sec -0.708 {'family_index': 0, 'per_family_instance_index': 6, 'run_name': 'ReadFileMetadata/num_columns:100/num_row_groups:1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 1654, 'data_size': 2600.0, 'file_size': 15470.0}
BM_ReadInt64Column<Repetition::OPTIONAL, Compression::ZSTD>/1024/1024 660.354 MiB/sec 655.662 MiB/sec -0.710 {'family_index': 14, 'per_family_instance_index': 1, 'run_name': 'BM_ReadInt64Column<Repetition::OPTIONAL, Compression::ZSTD>/1024/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 18963, 'compression_ratio': 2.390428946600525}
BM_ReadColumn<true,Int64Type>/75/1 2.064 GiB/sec 2.049 GiB/sec -0.717 {'family_index': 3, 'per_family_instance_index': 10, 'run_name': 'BM_ReadColumn<true,Int64Type>/75/1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 6}
RecordReaderReadAndSkipRecords/Repetition:2/BatchSize:10/LevelsPerPage:80000 147.076 MiB/sec 145.960 MiB/sec -0.759 {'family_index': 4, 'per_family_instance_index': 6, 'run_name': 'RecordReaderReadAndSkipRecords/Repetition:2/BatchSize:10/LevelsPerPage:80000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 15}
BM_ReadColumn<false,BooleanType>/1/20 65.606 MiB/sec 65.101 MiB/sec -0.770 {'family_index': 6, 'per_family_instance_index': 1, 'run_name': 'BM_ReadColumn<false,BooleanType>/1/20', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 15}
BM_ReadBinaryColumn/null_probability:1/unique_values:32 745.839 MiB/sec 739.997 MiB/sec -0.783 {'family_index': 14, 'per_family_instance_index': 2, 'run_name': 'BM_ReadBinaryColumn/null_probability:1/unique_values:32', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 1}
BM_ReadOffsetIndex/num_pages:8 130.515 MiB/sec 129.434 MiB/sec -0.828 {'family_index': 0, 'per_family_instance_index': 0, 'run_name': 'BM_ReadOffsetIndex/num_pages:8', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 412704}
BM_ReadColumnPlain<true,Float16LogicalType>/null_probability:0 1.655 GiB/sec 1.640 GiB/sec -0.916 {'family_index': 11, 'per_family_instance_index': 0, 'run_name': 'BM_ReadColumnPlain<true,Float16LogicalType>/null_probability:0', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 24}
BM_ReadBinaryViewColumn/null_probability:0/unique_values:32 924.357 MiB/sec 915.609 MiB/sec -0.946 {'family_index': 15, 'per_family_instance_index': 0, 'run_name': 'BM_ReadBinaryViewColumn/null_probability:0/unique_values:32', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 1}
BM_ReadColumnPlain<false,Float16LogicalType>/null_probability:-1 7.679 GiB/sec 7.605 GiB/sec -0.963 {'family_index': 10, 'per_family_instance_index': 0, 'run_name': 'BM_ReadColumnPlain<false,Float16LogicalType>/null_probability:-1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 108}
BM_DeltaBitPackingDecode_Int32_Fixed/32768 3.536 GiB/sec 3.501 GiB/sec -0.999 {'family_index': 11, 'per_family_instance_index': 2, 'run_name': 'BM_DeltaBitPackingDecode_Int32_Fixed/32768', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 6977}
BM_ReadColumnPlain<true,Float16LogicalType>/null_probability:50 517.500 MiB/sec 512.243 MiB/sec -1.016 {'family_index': 11, 'per_family_instance_index': 2, 'run_name': 'BM_ReadColumnPlain<true,Float16LogicalType>/null_probability:50', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 7}
BM_DecodeArrowBooleanPlain/DecodeArrow/1024 319.366 MiB/sec 315.979 MiB/sec -1.060 {'family_index': 32, 'per_family_instance_index': 0, 'run_name': 'BM_DecodeArrowBooleanPlain/DecodeArrow/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 734807}
ReadLevels_Rle/MaxLevel:1/NumLevels:8096/BatchSize:1024/LevelRepeatCount:1024 14.356 GiB/sec 14.196 GiB/sec -1.110 {'family_index': 5, 'per_family_instance_index': 2, 'run_name': 'ReadLevels_Rle/MaxLevel:1/NumLevels:8096/BatchSize:1024/LevelRepeatCount:1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 265253}
BM_ReadInt64Column<Repetition::OPTIONAL>/65536/1024 28.222 GiB/sec 27.904 GiB/sec -1.128 {'family_index': 1, 'per_family_instance_index': 2, 'run_name': 'BM_ReadInt64Column<Repetition::OPTIONAL>/65536/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 12887, 'compression_ratio': 0.9998531557216824}
BM_ReadListOfListColumn/99 1.250 GiB/sec 1.235 GiB/sec -1.151 {'family_index': 23, 'per_family_instance_index': 3, 'run_name': 'BM_ReadListOfListColumn/99', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 44}
BM_ReadColumn<true,BooleanType>/5/10 33.048 MiB/sec 32.665 MiB/sec -1.159 {'family_index': 7, 'per_family_instance_index': 1, 'run_name': 'BM_ReadColumn<true,BooleanType>/5/10', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 7}
BM_ReadInt64Column<Repetition::REPEATED>/1024/1024 6.017 GiB/sec 5.946 GiB/sec -1.184 {'family_index': 2, 'per_family_instance_index': 1, 'run_name': 'BM_ReadInt64Column<Repetition::REPEATED>/1024/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 147796, 'compression_ratio': 0.9900894367899444}
BM_ReadColumn<true,Int64Type>/50/1 1.769 GiB/sec 1.748 GiB/sec -1.190 {'family_index': 3, 'per_family_instance_index': 9, 'run_name': 'BM_ReadColumn<true,Int64Type>/50/1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 6}
ReadLevels_Rle/MaxLevel:1/NumLevels:8096/BatchSize:2048/LevelRepeatCount:1 5.849 GiB/sec 5.772 GiB/sec -1.318 {'family_index': 5, 'per_family_instance_index': 3, 'run_name': 'ReadLevels_Rle/MaxLevel:1/NumLevels:8096/BatchSize:2048/LevelRepeatCount:1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 107494}
BM_ReadColumn<true,Int64Type>/-1/0 1.569 GiB/sec 1.548 GiB/sec -1.337 {'family_index': 3, 'per_family_instance_index': 0, 'run_name': 'BM_ReadColumn<true,Int64Type>/-1/0', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 6}
RecordReaderReadRecords/Repetition:1/BatchSize:1000/ReadDense:1 2.986 GiB/sec 2.946 GiB/sec -1.352 {'family_index': 3, 'per_family_instance_index': 2, 'run_name': 'RecordReaderReadRecords/Repetition:1/BatchSize:1000/ReadDense:1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 329}
BM_ReadInt64Column<Repetition::REQUIRED, Compression::LZ4, Encoding::BYTE_STREAM_SPLIT>/1024/16 1.371 GiB/sec 1.352 GiB/sec -1.353 {'family_index': 11, 'per_family_instance_index': 0, 'run_name': 'BM_ReadInt64Column<Repetition::REQUIRED, Compression::LZ4, Encoding::BYTE_STREAM_SPLIT>/1024/16', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 50032, 'compression_ratio': 2.5672203071137574}
BM_ReadColumn<true,DoubleType>/-1/0 1.540 GiB/sec 1.518 GiB/sec -1.482 {'family_index': 5, 'per_family_instance_index': 0, 'run_name': 'BM_ReadColumn<true,DoubleType>/-1/0', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 6}
BM_ReadColumnPlain<true,Float16LogicalType>/null_probability:100 1.795 GiB/sec 1.768 GiB/sec -1.549 {'family_index': 11, 'per_family_instance_index': 4, 'run_name': 'BM_ReadColumnPlain<true,Float16LogicalType>/null_probability:100', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 26}
ReadLevels_Rle/MaxLevel:1/NumLevels:8096/BatchSize:1024/LevelRepeatCount:1 5.175 GiB/sec 5.087 GiB/sec -1.694 {'family_index': 5, 'per_family_instance_index': 0, 'run_name': 'ReadLevels_Rle/MaxLevel:1/NumLevels:8096/BatchSize:1024/LevelRepeatCount:1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 95649}
BM_DecodeArrowBooleanPlain/DecodeArrow/4096 1.097 GiB/sec 1.078 GiB/sec -1.723 {'family_index': 32, 'per_family_instance_index': 1, 'run_name': 'BM_DecodeArrowBooleanPlain/DecodeArrow/4096', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 640816}
BM_ReadColumn<true,DoubleType>/25/25 1.660 GiB/sec 1.630 GiB/sec -1.790 {'family_index': 5, 'per_family_instance_index': 2, 'run_name': 'BM_ReadColumn<true,DoubleType>/25/25', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 6}
RecordReaderReadAndSkipRecords/Repetition:2/BatchSize:100/LevelsPerPage:80000 445.857 MiB/sec 437.591 MiB/sec -1.854 {'family_index': 4, 'per_family_instance_index': 7, 'run_name': 'RecordReaderReadAndSkipRecords/Repetition:2/BatchSize:100/LevelsPerPage:80000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 45}
BM_ReadIndividualRowGroups 2.525 GiB/sec 2.476 GiB/sec -1.953 {'family_index': 24, 'per_family_instance_index': 0, 'run_name': 'BM_ReadIndividualRowGroups', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 9}
BM_ReadInt64Column<Repetition::REQUIRED, Compression::ZSTD>/1024/1024 546.807 MiB/sec 535.980 MiB/sec -1.980 {'family_index': 13, 'per_family_instance_index': 1, 'run_name': 'BM_ReadInt64Column<Repetition::REQUIRED, Compression::ZSTD>/1024/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 19575, 'compression_ratio': 2.398126463700234}
BM_DecodeArrowBooleanPlain/DecodeArrowNonNull/32768 4.875 GiB/sec 4.776 GiB/sec -2.041 {'family_index': 33, 'per_family_instance_index': 2, 'run_name': 'BM_DecodeArrowBooleanPlain/DecodeArrowNonNull/32768', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 359896}
ReadLevels_BitPack/MaxLevel:3/NumLevels:8096/BatchSize:2048/LevelRepeatCount:1 8.511 GiB/sec 8.335 GiB/sec -2.069 {'family_index': 6, 'per_family_instance_index': 5, 'run_name': 'ReadLevels_BitPack/MaxLevel:3/NumLevels:8096/BatchSize:2048/LevelRepeatCount:1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 158567}
BM_ReadColumn<true,Int64Type>/50/50 1.783 GiB/sec 1.746 GiB/sec -2.109 {'family_index': 3, 'per_family_instance_index': 8, 'run_name': 'BM_ReadColumn<true,Int64Type>/50/50', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 6}
BM_ReadColumn<true,DoubleType>/10/50 1.747 GiB/sec 1.708 GiB/sec -2.262 {'family_index': 5, 'per_family_instance_index': 1, 'run_name': 'BM_ReadColumn<true,DoubleType>/10/50', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 6}
BM_ReadColumnIndex<ByteArrayType>/num_pages:8 149.451 MiB/sec 145.858 MiB/sec -2.404 {'family_index': 4, 'per_family_instance_index': 0, 'run_name': 'BM_ReadColumnIndex<ByteArrayType>/num_pages:8', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 240103}
BM_ReadColumn<true,Int64Type>/45/25 1.731 GiB/sec 1.687 GiB/sec -2.532 {'family_index': 3, 'per_family_instance_index': 7, 'run_name': 'BM_ReadColumn<true,Int64Type>/45/25', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 6}
BM_DeltaBitPackingDecode_Int32_Fixed/65536 3.477 GiB/sec 3.386 GiB/sec -2.627 {'family_index': 11, 'per_family_instance_index': 3, 'run_name': 'BM_DeltaBitPackingDecode_Int32_Fixed/65536', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 4132}
ReadLevels_BitPack/MaxLevel:1/NumLevels:8096/BatchSize:1024/LevelRepeatCount:1024 8.477 GiB/sec 8.245 GiB/sec -2.737 {'family_index': 6, 'per_family_instance_index': 2, 'run_name': 'ReadLevels_BitPack/MaxLevel:1/NumLevels:8096/BatchSize:1024/LevelRepeatCount:1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 156081}
BM_ReadMultipleRowGroupsGenerator 2.545 GiB/sec 2.473 GiB/sec -2.823 {'family_index': 26, 'per_family_instance_index': 0, 'run_name': 'BM_ReadMultipleRowGroupsGenerator', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 9}
BM_ReadColumnPlain<true,Int32Type>/null_probability:0 2.899 GiB/sec 2.817 GiB/sec -2.854 {'family_index': 9, 'per_family_instance_index': 0, 'run_name': 'BM_ReadColumnPlain<true,Int32Type>/null_probability:0', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 21}
BM_ReadInt64Column<Repetition::OPTIONAL, Compression::ZSTD, Encoding::BYTE_STREAM_SPLIT>/1024/16 1.004 GiB/sec 997.163 MiB/sec -2.998 {'family_index': 17, 'per_family_instance_index': 0, 'run_name': 'BM_ReadInt64Column<Repetition::OPTIONAL, Compression::ZSTD, Encoding::BYTE_STREAM_SPLIT>/1024/16', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 29380, 'compression_ratio': 2.7379679144385025}
ReadLevels_Rle/MaxLevel:3/NumLevels:8096/BatchSize:1024/LevelRepeatCount:1 6.081 GiB/sec 5.894 GiB/sec -3.075 {'family_index': 5, 'per_family_instance_index': 4, 'run_name': 'ReadLevels_Rle/MaxLevel:3/NumLevels:8096/BatchSize:1024/LevelRepeatCount:1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 112579}
BM_ReadInt64Column<Repetition::REPEATED, Compression::ZSTD>/1024/16 681.907 MiB/sec 660.270 MiB/sec -3.173 {'family_index': 15, 'per_family_instance_index': 0, 'run_name': 'BM_ReadInt64Column<Repetition::REPEATED, Compression::ZSTD>/1024/16', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 16187, 'compression_ratio': 2.3841676367869615}
BM_ReadStructOfStructColumn/0 6.090 GiB/sec 5.891 GiB/sec -3.259 {'family_index': 19, 'per_family_instance_index': 0, 'run_name': 'BM_ReadStructOfStructColumn/0', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 70}
BM_ReadColumnPlain<true,Int32Type>/null_probability:100 3.311 GiB/sec 3.202 GiB/sec -3.301 {'family_index': 9, 'per_family_instance_index': 4, 'run_name': 'BM_ReadColumnPlain<true,Int32Type>/null_probability:100', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 24}
BM_ArrowBinaryViewPlain/DecodeArrow_Dense/32768 293.862 MiB/sec 283.562 MiB/sec -3.505 {'family_index': 21, 'per_family_instance_index': 2, 'run_name': 'BM_ArrowBinaryViewPlain/DecodeArrow_Dense/32768', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 438}
BM_ReadColumnByteStreamSplit<true,Float16LogicalType>/null_probability:1 1.126 GiB/sec 1.086 GiB/sec -3.513 {'family_index': 13, 'per_family_instance_index': 1, 'run_name': 'BM_ReadColumnByteStreamSplit<true,Float16LogicalType>/null_probability:1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 16}
BM_ReadColumn<true,Int32Type>/50/0 1017.803 MiB/sec 982.000 MiB/sec -3.518 {'family_index': 1, 'per_family_instance_index': 6, 'run_name': 'BM_ReadColumn<true,Int32Type>/50/0', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 7}
RecordReaderReadAndSkipRecords/Repetition:1/BatchSize:10000/LevelsPerPage:1000000 997.760 MiB/sec 961.547 MiB/sec -3.629 {'family_index': 4, 'per_family_instance_index': 5, 'run_name': 'RecordReaderReadAndSkipRecords/Repetition:1/BatchSize:10000/LevelsPerPage:1000000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 9}
BM_ReadColumnPlain<true,Int32Type>/null_probability:50 1014.733 MiB/sec 977.545 MiB/sec -3.665 {'family_index': 9, 'per_family_instance_index': 2, 'run_name': 'BM_ReadColumnPlain<true,Int32Type>/null_probability:50', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 7}
BM_DeltaBitPackingDecode_Int32_Fixed/1024 3.293 GiB/sec 3.173 GiB/sec -3.669 {'family_index': 11, 'per_family_instance_index': 0, 'run_name': 'BM_DeltaBitPackingDecode_Int32_Fixed/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 244157}
BM_ReadColumn<true,BooleanType>/-1/1 25.668 MiB/sec 24.706 MiB/sec -3.747 {'family_index': 7, 'per_family_instance_index': 0, 'run_name': 'BM_ReadColumn<true,BooleanType>/-1/1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 6}
BM_ReadColumn<true,Int64Type>/1/1 3.116 GiB/sec 2.999 GiB/sec -3.778 {'family_index': 3, 'per_family_instance_index': 1, 'run_name': 'BM_ReadColumn<true,Int64Type>/1/1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 11}
BM_ReadColumn<true,Int32Type>/50/50 1022.767 MiB/sec 983.956 MiB/sec -3.795 {'family_index': 1, 'per_family_instance_index': 5, 'run_name': 'BM_ReadColumn<true,Int32Type>/50/50', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 7}
BM_DeltaBitPackingDecode_Int32_Fixed/4096 3.573 GiB/sec 3.436 GiB/sec -3.820 {'family_index': 11, 'per_family_instance_index': 1, 'run_name': 'BM_DeltaBitPackingDecode_Int32_Fixed/4096', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 66086}
BM_ReadColumnPlain<true,Float16LogicalType>/null_probability:1 1.138 GiB/sec 1.094 GiB/sec -3.877 {'family_index': 11, 'per_family_instance_index': 1, 'run_name': 'BM_ReadColumnPlain<true,Float16LogicalType>/null_probability:1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 16}
RecordReaderReadAndSkipRecords/Repetition:0/BatchSize:1000/LevelsPerPage:80000 32.487 GiB/sec 31.210 GiB/sec -3.930 {'family_index': 4, 'per_family_instance_index': 1, 'run_name': 'RecordReaderReadAndSkipRecords/Repetition:0/BatchSize:1000/LevelsPerPage:80000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 1962}
BM_ReadBinaryColumn/null_probability:99/unique_values:-1 629.802 MiB/sec 603.617 MiB/sec -4.158 {'family_index': 14, 'per_family_instance_index': 7, 'run_name': 'BM_ReadBinaryColumn/null_probability:99/unique_values:-1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 4}
BM_ReadColumnPlain<true,Float16LogicalType>/null_probability:99 1.263 GiB/sec 1.208 GiB/sec -4.339 {'family_index': 11, 'per_family_instance_index': 3, 'run_name': 'BM_ReadColumnPlain<true,Float16LogicalType>/null_probability:99', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 18}
BM_ReadInt64Column<Repetition::REPEATED>/65536/1024 29.670 GiB/sec 28.359 GiB/sec -4.419 {'family_index': 2, 'per_family_instance_index': 2, 'run_name': 'BM_ReadInt64Column<Repetition::REPEATED>/65536/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 10984, 'compression_ratio': 0.9998379016463471}
ColumnReaderReadBatchInt32/Repetition:0/BatchSize:1000 65.692 GiB/sec 62.761 GiB/sec -4.462 {'family_index': 1, 'per_family_instance_index': 0, 'run_name': 'ColumnReaderReadBatchInt32/Repetition:0/BatchSize:1000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 3845}
RecordReaderReadRecords/Repetition:1/BatchSize:1000/ReadDense:0 537.623 MiB/sec 512.850 MiB/sec -4.608 {'family_index': 3, 'per_family_instance_index': 3, 'run_name': 'RecordReaderReadRecords/Repetition:1/BatchSize:1000/ReadDense:0', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 58}
ColumnReaderReadBatchInt32/Repetition:2/BatchSize:1000 1.734 GiB/sec 1.652 GiB/sec -4.699 {'family_index': 1, 'per_family_instance_index': 2, 'run_name': 'ColumnReaderReadBatchInt32/Repetition:2/BatchSize:1000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 180}
RecordReaderSkipRecords/Repetition:1/BatchSize:1000 3.932 GiB/sec 3.746 GiB/sec -4.744 {'family_index': 2, 'per_family_instance_index': 1, 'run_name': 'RecordReaderSkipRecords/Repetition:1/BatchSize:1000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 434}
ColumnReaderSkipInt32/Repetition:0/BatchSize:1000 53.198 GiB/sec 50.656 GiB/sec -4.779 {'family_index': 0, 'per_family_instance_index': 0, 'run_name': 'ColumnReaderSkipInt32/Repetition:0/BatchSize:1000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 2853}
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Regressions: (30)
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
benchmark baseline contender change % counters
RecordReaderReadAndSkipRecords/Repetition:1/BatchSize:1000/LevelsPerPage:80000 965.796 MiB/sec 917.336 MiB/sec -5.018 {'family_index': 4, 'per_family_instance_index': 4, 'run_name': 'RecordReaderReadAndSkipRecords/Repetition:1/BatchSize:1000/LevelsPerPage:80000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 105}
RecordReaderReadRecords/Repetition:0/BatchSize:1000/ReadDense:0 34.992 GiB/sec 33.223 GiB/sec -5.057 {'family_index': 3, 'per_family_instance_index': 1, 'run_name': 'RecordReaderReadRecords/Repetition:0/BatchSize:1000/ReadDense:0', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 2067}
BM_ReadColumn<true,Int32Type>/-1/0 861.380 MiB/sec 817.652 MiB/sec -5.077 {'family_index': 1, 'per_family_instance_index': 0, 'run_name': 'BM_ReadColumn<true,Int32Type>/-1/0', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 6}
ReadLevels_BitPack/MaxLevel:1/NumLevels:8096/BatchSize:1024/LevelRepeatCount:1 8.504 GiB/sec 8.069 GiB/sec -5.113 {'family_index': 6, 'per_family_instance_index': 0, 'run_name': 'ReadLevels_BitPack/MaxLevel:1/NumLevels:8096/BatchSize:1024/LevelRepeatCount:1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 158692}
BM_ReadListOfListColumn/0 979.297 MiB/sec 928.942 MiB/sec -5.142 {'family_index': 23, 'per_family_instance_index': 0, 'run_name': 'BM_ReadListOfListColumn/0', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 34}
BM_ReadInt64Column<Repetition::OPTIONAL, Compression::LZ4, Encoding::BYTE_STREAM_SPLIT>/1024/16 1.420 GiB/sec 1.344 GiB/sec -5.378 {'family_index': 12, 'per_family_instance_index': 0, 'run_name': 'BM_ReadInt64Column<Repetition::OPTIONAL, Compression::LZ4, Encoding::BYTE_STREAM_SPLIT>/1024/16', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 41811, 'compression_ratio': 2.561601000625391}
ColumnReaderReadBatchInt32/Repetition:1/BatchSize:1000 3.990 GiB/sec 3.769 GiB/sec -5.541 {'family_index': 1, 'per_family_instance_index': 1, 'run_name': 'ColumnReaderReadBatchInt32/Repetition:1/BatchSize:1000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 440}
BM_ReadColumnByteStreamSplit<true,Float16LogicalType>/null_probability:99 1.264 GiB/sec 1.193 GiB/sec -5.571 {'family_index': 13, 'per_family_instance_index': 3, 'run_name': 'BM_ReadColumnByteStreamSplit<true,Float16LogicalType>/null_probability:99', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 18}
BM_ReadColumnByteStreamSplit<true,Float16LogicalType>/null_probability:100 1.814 GiB/sec 1.713 GiB/sec -5.597 {'family_index': 13, 'per_family_instance_index': 4, 'run_name': 'BM_ReadColumnByteStreamSplit<true,Float16LogicalType>/null_probability:100', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 26}
BM_DecodeArrowBooleanPlain/DecodeArrow/65536 6.708 GiB/sec 6.310 GiB/sec -5.934 {'family_index': 32, 'per_family_instance_index': 3, 'run_name': 'BM_DecodeArrowBooleanPlain/DecodeArrow/65536', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 245712}
BM_ReadInt64Column<Repetition::OPTIONAL, Compression::LZ4>/1024/16 1.609 GiB/sec 1.502 GiB/sec -6.628 {'family_index': 9, 'per_family_instance_index': 0, 'run_name': 'BM_ReadInt64Column<Repetition::OPTIONAL, Compression::LZ4>/1024/16', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 47357, 'compression_ratio': 1.6028174525533163}
ReadLevels_BitPack/MaxLevel:1/NumLevels:8096/BatchSize:2048/LevelRepeatCount:1 8.538 GiB/sec 7.964 GiB/sec -6.729 {'family_index': 6, 'per_family_instance_index': 3, 'run_name': 'ReadLevels_BitPack/MaxLevel:1/NumLevels:8096/BatchSize:2048/LevelRepeatCount:1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 157596}
ColumnReaderSkipInt32/Repetition:2/BatchSize:1000 1.675 GiB/sec 1.555 GiB/sec -7.165 {'family_index': 0, 'per_family_instance_index': 2, 'run_name': 'ColumnReaderSkipInt32/Repetition:2/BatchSize:1000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 170}
ColumnReaderSkipInt32/Repetition:1/BatchSize:1000 3.808 GiB/sec 3.533 GiB/sec -7.204 {'family_index': 0, 'per_family_instance_index': 1, 'run_name': 'ColumnReaderSkipInt32/Repetition:1/BatchSize:1000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 420}
BM_DecodeArrowBooleanPlain/DecodeArrowNonNull/1024 313.922 MiB/sec 291.032 MiB/sec -7.292 {'family_index': 33, 'per_family_instance_index': 0, 'run_name': 'BM_DecodeArrowBooleanPlain/DecodeArrowNonNull/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 719054}
BM_ReadColumn<false,BooleanType>/-1/0 187.612 MiB/sec 173.859 MiB/sec -7.331 {'family_index': 6, 'per_family_instance_index': 0, 'run_name': 'BM_ReadColumn<false,BooleanType>/-1/0', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 42}
BM_ReadInt64Column<Repetition::OPTIONAL, Compression::ZSTD, Encoding::BYTE_STREAM_SPLIT>/65536/1024 5.151 GiB/sec 4.770 GiB/sec -7.391 {'family_index': 17, 'per_family_instance_index': 2, 'run_name': 'BM_ReadInt64Column<Repetition::OPTIONAL, Compression::ZSTD, Encoding::BYTE_STREAM_SPLIT>/65536/1024', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 2367, 'compression_ratio': 3.1924592180335756}
RecordReaderReadRecords/Repetition:2/BatchSize:1000/ReadDense:0 395.323 MiB/sec 365.173 MiB/sec -7.626 {'family_index': 3, 'per_family_instance_index': 5, 'run_name': 'RecordReaderReadRecords/Repetition:2/BatchSize:1000/ReadDense:0', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 40}
BM_ReadInt64Column<Repetition::OPTIONAL, Compression::SNAPPY, Encoding::BYTE_STREAM_SPLIT>/1024/16 1.811 GiB/sec 1.672 GiB/sec -7.706 {'family_index': 7, 'per_family_instance_index': 0, 'run_name': 'BM_ReadInt64Column<Repetition::OPTIONAL, Compression::SNAPPY, Encoding::BYTE_STREAM_SPLIT>/1024/16', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 53029, 'compression_ratio': 2.3017701601573477}
BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:4096/null_in_ten_thousand:10000 1.215 GiB/sec 1.115 GiB/sec -8.176 {'family_index': 31, 'per_family_instance_index': 17, 'run_name': 'BM_DecodeArrowBooleanRle/DecodeArrowWithNull/num_values:4096/null_in_ten_thousand:10000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 723190}
RecordReaderReadAndSkipRecords/Repetition:0/BatchSize:10/LevelsPerPage:80000 1.044 GiB/sec 980.830 MiB/sec -8.248 {'family_index': 4, 'per_family_instance_index': 0, 'run_name': 'RecordReaderReadAndSkipRecords/Repetition:0/BatchSize:10/LevelsPerPage:80000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 64}
BM_DeltaBitPackingDecode_Int32_Narrow/65536 2.774 GiB/sec 2.535 GiB/sec -8.612 {'family_index': 13, 'per_family_instance_index': 3, 'run_name': 'BM_DeltaBitPackingDecode_Int32_Narrow/65536', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 3193}
RecordReaderReadRecords/Repetition:2/BatchSize:1000/ReadDense:1 1.342 GiB/sec 1.225 GiB/sec -8.729 {'family_index': 3, 'per_family_instance_index': 4, 'run_name': 'RecordReaderReadRecords/Repetition:2/BatchSize:1000/ReadDense:1', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 137}
ReadLevels_Rle/MaxLevel:3/NumLevels:8096/BatchSize:1024/LevelRepeatCount:7 1.922 GiB/sec 1.747 GiB/sec -9.116 {'family_index': 5, 'per_family_instance_index': 6, 'run_name': 'ReadLevels_Rle/MaxLevel:3/NumLevels:8096/BatchSize:1024/LevelRepeatCount:7', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 35112}
RecordReaderSkipRecords/Repetition:2/BatchSize:1000 1.350 GiB/sec 1.225 GiB/sec -9.262 {'family_index': 2, 'per_family_instance_index': 2, 'run_name': 'RecordReaderSkipRecords/Repetition:2/BatchSize:1000', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 140}
BM_ReadInt64Column<Repetition::REPEATED, Compression::LZ4>/1024/16 1.775 GiB/sec 1.560 GiB/sec -12.112 {'family_index': 10, 'per_family_instance_index': 0, 'run_name': 'BM_ReadInt64Column<Repetition::REPEATED, Compression::LZ4>/1024/16', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 42355, 'compression_ratio': 1.601564027370479}
BM_DeltaBitPackingDecode_Int32_Narrow/32768 2.775 GiB/sec 2.421 GiB/sec -12.750 {'family_index': 13, 'per_family_instance_index': 2, 'run_name': 'BM_DeltaBitPackingDecode_Int32_Narrow/32768', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 6402}
ReadLevels_Rle/MaxLevel:1/NumLevels:8096/BatchSize:1024/LevelRepeatCount:7 1.860 GiB/sec 1.591 GiB/sec -14.488 {'family_index': 5, 'per_family_instance_index': 1, 'run_name': 'ReadLevels_Rle/MaxLevel:1/NumLevels:8096/BatchSize:1024/LevelRepeatCount:7', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 34512}
BM_ReadInt64Column<Repetition::REPEATED>/1024/16 2.730 GiB/sec 2.330 GiB/sec -14.643 {'family_index': 2, 'per_family_instance_index': 0, 'run_name': 'BM_ReadInt64Column<Repetition::REPEATED>/1024/16', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 60733, 'compression_ratio': 0.9900894367899444}
BM_ReadInt64Column<Repetition::OPTIONAL>/1024/16 2.955 GiB/sec 2.509 GiB/sec -15.079 {'family_index': 1, 'per_family_instance_index': 0, 'run_name': 'BM_ReadInt64Column<Repetition::OPTIONAL>/1024/16', 'repetitions': 1, 'repetition_index': 0, 'threads': 1, 'iterations': 83619, 'compression_ratio': 0.9909277851699528}
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2319452696
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
Review Comment:
Looks like the iteration pattern may be more idiomatic than having to return some `ControlFlow` enum. Would it be slower?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2319495572
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -84,32 +85,278 @@ namespace util {
/// (total 26 bytes, 1 byte overhead)
//
+class RleRun {
+ public:
+ using byte = uint8_t;
+ /// Enough space to store a 64bit value
+ using raw_data_storage = std::array<byte, 8>;
+ using raw_data_const_pointer = const byte*;
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr RleRun() noexcept = default;
+ constexpr RleRun(RleRun const&) noexcept = default;
+ constexpr RleRun(RleRun&&) noexcept = default;
+
+ explicit RleRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr RleRun& operator=(RleRun const&) noexcept = default;
+ constexpr RleRun& operator=(RleRun&&) noexcept = default;
+
+ /// The number of repeated values in this run.
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ /// A pointer to the repeated value raw bytes.
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ /// The number of bytes used for the raw repeated value.
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The repeated value raw bytes stored inside the class
+ raw_data_storage data_ = {};
+ /// The number of time the value is repeated
+ values_count_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+class BitPackedRun {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// According to the Parquet thrift definition the page size can be written into an
+ /// int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr BitPackedRun() noexcept = default;
+ constexpr BitPackedRun(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun(BitPackedRun&&) noexcept = default;
+
+ constexpr BitPackedRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr BitPackedRun& operator=(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun& operator=(BitPackedRun&&) noexcept = default;
+
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Number of values in this run.
+ raw_data_size_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+/// A parser that emits either a ``BitPackedRun`` or a ``RleRun``.
+class RleBitPackedParser {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// By Parquet thrift definition the page size can be written into an int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+ /// The different types of runs emitted by the parser
+ using dynamic_run_type = std::variant<RleRun, BitPackedRun>;
+
+ constexpr RleBitPackedParser() noexcept = default;
+
+ constexpr RleBitPackedParser(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr void Reset(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width_) noexcept;
+
+ /// Get the current run with a small parsing cost without advancing the iteration.
+ [[nodiscard]] std::optional<dynamic_run_type> Peek() const;
+
+ /// Move to the next run.
+ [[nodiscard]] bool Advance();
+
+ /// Advance and return the current run.
+ [[nodiscard]] std::optional<dynamic_run_type> Next();
+
+ /// Whether there is still runs to iterate over.
+ ///
+ /// WARN: Due to lack of proper error handling, iteration with Next and Peek could
+ /// return not data while the parser is not exhausted.
+ /// This is how one can check for errors.
+ [[nodiscard]] bool Exhausted() const;
+
+ /// Enum to return from an ``Parse`` handler.
+ ///
+ /// Since a callback has no way to know when to stop, the handler must return
+ /// a value indicating to the ``Parse`` function whether to stop or continue.
+ enum class ControlFlow {
+ Continue,
+ Break,
+ };
+
+ /// A callback approach to parsing.
+ ///
+ /// This approach is used to reduce the number of dynamic lookups involved with using a
+ /// variant.
+ ///
+ /// The handler must be of the form
+ /// ```cpp`
+ /// struct Handler {
+ /// ControlFlow OnBitPackedRun(BitPackedRun run);
+ ///
+ /// ControlFlow OnRleRun(RleRun run);
+ /// };
+ /// ```
+ template <typename Handler>
+ void Parse(Handler&& handler);
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Size in bytes of the run.
+ raw_data_size_type data_size_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+
+ /// Run the handler on the run read and return the number of values read.
+ /// Does not advance the parser.
+ template <typename Handler>
+ std::pair<raw_data_size_type, ControlFlow> PeekImpl(Handler&&) const;
+};
+
/// Decoder class for RLE encoded data.
+template <typename T>
class RleDecoder {
public:
- /// Create a decoder object. buffer/buffer_len is the decoded data.
- /// bit_width is the width of each value (before encoding).
- RleDecoder(const uint8_t* buffer, int buffer_len, int bit_width)
- : bit_reader_(buffer, buffer_len),
- bit_width_(bit_width),
- current_value_(0),
- repeat_count_(0),
- literal_count_(0) {
- ARROW_DCHECK_GE(bit_width_, 0);
- ARROW_DCHECK_LE(bit_width_, 64);
- }
+ /// The type in which the data should be decoded.
+ using value_type = T;
+ /// The type of run that can be decoded.
+ using run_type = RleRun;
+ using values_count_type = run_type::values_count_type;
- RleDecoder() : bit_width_(-1) {}
+ constexpr RleDecoder() noexcept = default;
- void Reset(const uint8_t* buffer, int buffer_len, int bit_width) {
- ARROW_DCHECK_GE(bit_width, 0);
- ARROW_DCHECK_LE(bit_width, 64);
- bit_reader_.Reset(buffer, buffer_len);
- bit_width_ = bit_width;
- current_value_ = 0;
- repeat_count_ = 0;
- literal_count_ = 0;
- }
+ explicit RleDecoder(run_type const& run) noexcept;
+
+ void Reset(run_type const& run) noexcept;
+
+ /// Return the number of values that can be advanced.
+ [[nodiscard]] values_count_type Remaining() const;
+
+ /// Return the repeated value of this decoder.
+ [[nodiscard]] constexpr value_type Value() const;
+
+ /// Try to advance by as many values as provided.
+ /// Return the number of values skipped.
+ [[nodiscard]] values_count_type Advance(values_count_type batch_size);
+
+ /// Get the next value and return false if there are no more.
+ [[nodiscard]] constexpr bool Get(value_type* out_value);
+
+ /// Get a batch of values return the number of decoded elements.
+ [[nodiscard]] values_count_type GetBatch(value_type* out, values_count_type batch_size);
+
+ private:
+ value_type value_ = {};
+ values_count_type remaining_count_ = 0;
+
+ static_assert(std::is_integral_v<value_type>,
+ "This class makes assumptions about integer endianness and padding");
+};
+
+/// Decoder class for Bit packing encoded data.
+template <typename T>
+class BitPackedDecoder {
+ public:
+ /// The type in which the data should be decoded.
+ using value_type = T;
+ /// The type of run that can be decoded.
+ using run_type = BitPackedRun;
+ using values_count_type = run_type::values_count_type;
+ using bit_size_type = run_type::bit_size_type;
+
+ BitPackedDecoder() noexcept = default;
+
+ explicit BitPackedDecoder(run_type const& run) noexcept;
+
+ void Reset(run_type const& run) noexcept;
+
+ /// Return the number of values that can be advanced.
+ [[nodiscard]] constexpr values_count_type Remaining() const;
+
+ /// Return the size in bit in which each encoded value is written.
+ [[nodiscard]] constexpr bit_size_type ValueBitWidth() const;
+
+ /// Try to advance by as many values as provided.
+ /// Return the number of values skipped.
+ [[nodiscard]] values_count_type Advance(values_count_type batch_size);
+
+ /// Get the next value and return false if there are no more.
+ [[nodiscard]] bool Get(value_type* out_value);
+
+ /// Get a batch of values return the number of decoded elements.
+ [[nodiscard]] values_count_type GetBatch(value_type* out, values_count_type batch_size);
+
+ private:
+ ::arrow::bit_util::BitReader bit_reader_ = {};
+ bit_size_type value_bit_width_ = 0;
+ values_count_type remaining_count_ = 0;
+
+ static_assert(std::is_integral_v<value_type>,
+ "This class makes assumptions about integer endianness and padding");
+};
+
+/// Decoder class for RLE encoded data.
+template <typename T>
+class RleBitPackedDecoder {
+ public:
+ /// The type in which the data should be decoded.
+ using value_type = T;
+ using byte = RleBitPackedParser::byte;
+ using raw_data_const_pointer = RleBitPackedParser::raw_data_const_pointer;
+ using raw_data_size_type = RleBitPackedParser::raw_data_size_type;
+ using bit_size_type = RleBitPackedParser::bit_size_type;
+ using dynamic_run_type = RleBitPackedParser::dynamic_run_type;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+
+ RleBitPackedDecoder() noexcept = default;
+
+ /// Create a decoder object.
+ ///
+ /// data and data_size are the raw bytes to decode.
+ /// value_bit_width is the size in bits of each encoded value.
+ RleBitPackedDecoder(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept;
+
+ void Reset(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width_) noexcept;
+
+ /// Whether there is still runs to iterate over.
+ ///
+ /// WARN: Due to lack of proper error handling, iteration with Get methods could return
+ /// no data while the parser is not exhausted.
+ /// This is how one can check for errors.
+ [[nodiscard]] bool Exhausted() const;
Review Comment:
Nit, but we would use `snake_case` for such cheap properties.
```suggestion
[[nodiscard]] bool exhausted() const;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2355590946
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -84,32 +85,278 @@ namespace util {
/// (total 26 bytes, 1 byte overhead)
//
+class RleRun {
+ public:
+ using byte = uint8_t;
+ /// Enough space to store a 64bit value
+ using raw_data_storage = std::array<byte, 8>;
+ using raw_data_const_pointer = const byte*;
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr RleRun() noexcept = default;
Review Comment:
I still believe the `RleRun` is not trivial. It requires partially filling the local array (and also opportunities for some checks). It acts as a cheap read-only run with an interface uniform with `BitPackedRun`.
Now with the function definition within the class and reduced number of aliases, it reads clearly within one screen height. I don't think exposing the member variable would do more than save a few lines.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2349481944
##########
cpp/src/parquet/decoder.cc:
##########
@@ -1898,7 +1900,7 @@ class RleBooleanDecoder : public TypedDecoderImpl<BooleanType>, public BooleanDe
}
private:
- std::shared_ptr<::arrow::util::RleDecoder> decoder_;
+ std::shared_ptr<::arrow::util::RleBitPackedDecoder<bool>> decoder_;
Review Comment:
EDIT: removing the pointer seems to be a slight regression. Perhaps since `sizeof(decoder_) == 64` (mostly driven by the `BitReader` in the `BitPackedRunDecoder`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2348873945
##########
cpp/src/parquet/decoder.cc:
##########
@@ -1898,7 +1900,7 @@ class RleBooleanDecoder : public TypedDecoderImpl<BooleanType>, public BooleanDe
}
private:
- std::shared_ptr<::arrow::util::RleDecoder> decoder_;
+ std::shared_ptr<::arrow::util::RleBitPackedDecoder<bool>> decoder_;
Review Comment:
Actually, we can remove the delayed initialization altogether since an default-constructed decoder is now valid.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2359203120
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -82,86 +81,400 @@ namespace util {
/// 200 ints = 25 groups of 8
/// <varint((25 << 1) | 1)> <25 bytes of values, bitpacked>
/// (total 26 bytes, 1 byte overhead)
-//
-/// Decoder class for RLE encoded data.
-class RleDecoder {
+/// The type for an encoded Rle of BitPacked run size, between 1 and 2^31-1 as per Parquet
+/// spec.
+/// This is also pragmatically used for other integer used in the Rle and BitPacked runs
+/// and decoder to avoid conversions.
+/// It can therefore be referred to as a "typical" size for Rle and BitPacked logic.
+using rle_size_t = int32_t;
+
+template <typename T>
+class RleRunDecoder;
+
+class RleRun {
public:
- /// Create a decoder object. buffer/buffer_len is the decoded data.
- /// bit_width is the width of each value (before encoding).
- RleDecoder(const uint8_t* buffer, int buffer_len, int bit_width)
- : bit_reader_(buffer, buffer_len),
- bit_width_(bit_width),
- current_value_(0),
- repeat_count_(0),
- literal_count_(0) {
- ARROW_DCHECK_GE(bit_width_, 0);
- ARROW_DCHECK_LE(bit_width_, 64);
+ /// The decoder class used to decode a single run in the given type.
+ template <typename T>
+ using DecoderType = RleRunDecoder<T>;
+
+ constexpr RleRun() noexcept = default;
+
+ explicit RleRun(const uint8_t* data, rle_size_t values_count,
+ rle_size_t value_bit_width) noexcept
+ : values_count_(values_count), value_bit_width_(value_bit_width) {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_GE(values_count, 0);
+ std::copy(data, data + raw_data_size(), data_.begin());
}
- RleDecoder() : bit_width_(-1) {}
+ /// The number of repeated values in this run.
+ constexpr rle_size_t values_count() const noexcept { return values_count_; }
- void Reset(const uint8_t* buffer, int buffer_len, int bit_width) {
- ARROW_DCHECK_GE(bit_width, 0);
- ARROW_DCHECK_LE(bit_width, 64);
- bit_reader_.Reset(buffer, buffer_len);
- bit_width_ = bit_width;
- current_value_ = 0;
- repeat_count_ = 0;
- literal_count_ = 0;
+ /// The size in bits of each encoded value.
+ constexpr rle_size_t values_bit_width() const noexcept { return value_bit_width_; }
+
+ /// A pointer to the repeated value raw bytes.
+ constexpr const uint8_t* raw_data_ptr() const noexcept { return data_.data(); }
+
+ /// The number of bytes used for the raw repeated value.
+ constexpr rle_size_t raw_data_size() const noexcept {
+ auto out = bit_util::BytesForBits(value_bit_width_);
+ ARROW_DCHECK_LE(out, std::numeric_limits<rle_size_t>::max());
+ return static_cast<rle_size_t>(out);
}
- /// Gets the next value. Returns false if there are no more.
+ private:
+ /// The repeated value raw bytes stored inside the class with enough space to store
+ /// up to a 64 bit value.
+ std::array<uint8_t, 8> data_ = {};
+ /// The number of time the value is repeated.
+ rle_size_t values_count_ = 0;
+ /// The size in bit of a packed value in the run.
+ rle_size_t value_bit_width_ = 0;
Review Comment:
I'm curious, why does this have to be a member of `RleRun`? The value bit width is constant for the entire RleBitPacked-encoded data.
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -82,86 +81,400 @@ namespace util {
/// 200 ints = 25 groups of 8
/// <varint((25 << 1) | 1)> <25 bytes of values, bitpacked>
/// (total 26 bytes, 1 byte overhead)
-//
-/// Decoder class for RLE encoded data.
-class RleDecoder {
+/// The type for an encoded Rle of BitPacked run size, between 1 and 2^31-1 as per Parquet
+/// spec.
+/// This is also pragmatically used for other integer used in the Rle and BitPacked runs
+/// and decoder to avoid conversions.
+/// It can therefore be referred to as a "typical" size for Rle and BitPacked logic.
+using rle_size_t = int32_t;
+
+template <typename T>
+class RleRunDecoder;
+
+class RleRun {
public:
- /// Create a decoder object. buffer/buffer_len is the decoded data.
- /// bit_width is the width of each value (before encoding).
- RleDecoder(const uint8_t* buffer, int buffer_len, int bit_width)
- : bit_reader_(buffer, buffer_len),
- bit_width_(bit_width),
- current_value_(0),
- repeat_count_(0),
- literal_count_(0) {
- ARROW_DCHECK_GE(bit_width_, 0);
- ARROW_DCHECK_LE(bit_width_, 64);
+ /// The decoder class used to decode a single run in the given type.
+ template <typename T>
+ using DecoderType = RleRunDecoder<T>;
+
+ constexpr RleRun() noexcept = default;
+
+ explicit RleRun(const uint8_t* data, rle_size_t values_count,
+ rle_size_t value_bit_width) noexcept
+ : values_count_(values_count), value_bit_width_(value_bit_width) {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_GE(values_count, 0);
+ std::copy(data, data + raw_data_size(), data_.begin());
}
- RleDecoder() : bit_width_(-1) {}
+ /// The number of repeated values in this run.
+ constexpr rle_size_t values_count() const noexcept { return values_count_; }
- void Reset(const uint8_t* buffer, int buffer_len, int bit_width) {
- ARROW_DCHECK_GE(bit_width, 0);
- ARROW_DCHECK_LE(bit_width, 64);
- bit_reader_.Reset(buffer, buffer_len);
- bit_width_ = bit_width;
- current_value_ = 0;
- repeat_count_ = 0;
- literal_count_ = 0;
+ /// The size in bits of each encoded value.
+ constexpr rle_size_t values_bit_width() const noexcept { return value_bit_width_; }
+
+ /// A pointer to the repeated value raw bytes.
+ constexpr const uint8_t* raw_data_ptr() const noexcept { return data_.data(); }
+
+ /// The number of bytes used for the raw repeated value.
+ constexpr rle_size_t raw_data_size() const noexcept {
+ auto out = bit_util::BytesForBits(value_bit_width_);
+ ARROW_DCHECK_LE(out, std::numeric_limits<rle_size_t>::max());
+ return static_cast<rle_size_t>(out);
}
- /// Gets the next value. Returns false if there are no more.
+ private:
+ /// The repeated value raw bytes stored inside the class with enough space to store
+ /// up to a 64 bit value.
+ std::array<uint8_t, 8> data_ = {};
+ /// The number of time the value is repeated.
+ rle_size_t values_count_ = 0;
+ /// The size in bit of a packed value in the run.
+ rle_size_t value_bit_width_ = 0;
+};
+
+template <typename T>
+class BitPackedRunDecoder;
+
+class BitPackedRun {
+ public:
+ /// The decoder class used to decode a single run in the given type.
+ template <typename T>
+ using DecoderType = BitPackedRunDecoder<T>;
+
+ constexpr BitPackedRun() noexcept = default;
+
+ constexpr BitPackedRun(const uint8_t* data, rle_size_t values_count,
+ rle_size_t value_bit_width) noexcept
+ : data_(data), values_count_(values_count), value_bit_width_(value_bit_width) {
+ ARROW_CHECK_GE(value_bit_width_, 0);
+ ARROW_CHECK_GE(values_count_, 0);
+ }
+
+ constexpr rle_size_t values_count() const noexcept { return values_count_; }
+
+ /// The size in bits of each encoded value.
+ constexpr rle_size_t values_bit_width() const noexcept { return value_bit_width_; }
+
+ constexpr const uint8_t* raw_data_ptr() const noexcept { return data_; }
+
+ constexpr rle_size_t raw_data_size() const noexcept {
+ auto out = bit_util::BytesForBits(static_cast<int64_t>(value_bit_width_) *
+ static_cast<int64_t>(values_count_));
+ ARROW_CHECK_LE(out, std::numeric_limits<rle_size_t>::max());
+ return static_cast<rle_size_t>(out);
+ }
+
+ private:
+ /// The pointer to the beginning of the run
+ const uint8_t* data_ = nullptr;
+ /// Number of values in this run.
+ rle_size_t values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ rle_size_t value_bit_width_ = 0;
+};
+
+/// A parser that emits either a ``BitPackedRun`` or a ``RleRun``.
+class RleBitPackedParser {
+ public:
+ /// The different types of runs emitted by the parser
+ using dynamic_run_type = std::variant<RleRun, BitPackedRun>;
+
+ constexpr RleBitPackedParser() noexcept = default;
+
+ constexpr RleBitPackedParser(const uint8_t* data, rle_size_t data_size,
+ rle_size_t value_bit_width) noexcept
+ : data_(data), data_size_(data_size), value_bit_width_(value_bit_width) {}
+
+ constexpr void Reset(const uint8_t* data, rle_size_t data_size,
+ rle_size_t value_bit_width) noexcept {
+ *this = {data, data_size, value_bit_width};
+ }
+
+ /// Whether there is still runs to iterate over.
+ ///
+ /// WARN: Due to simplistic error handling, iteration with Next and Peek could
+ /// fail to return data while the parser is not exhausted.
+ /// This is how one can check for errors.
+ bool exhausted() const { return data_size_ == 0; }
+
+ /// Enum to return from an ``Parse`` handler.
+ ///
+ /// Since a callback has no way to know when to stop, the handler must return
+ /// a value indicating to the ``Parse`` function whether to stop or continue.
+ enum class ControlFlow {
+ Continue,
+ Break,
+ };
+
+ /// A callback approach to parsing.
+ ///
+ /// This approach is used to reduce the number of dynamic lookups involved with using a
+ /// variant.
+ ///
+ /// The handler must be of the form
+ /// ```cpp
+ /// struct Handler {
+ /// ControlFlow OnBitPackedRun(BitPackedRun run);
+ ///
+ /// ControlFlow OnRleRun(RleRun run);
+ /// };
+ /// ```
+ template <typename Handler>
+ void Parse(Handler&& handler);
+
+ private:
+ /// The pointer to the beginning of the run
+ const uint8_t* data_ = nullptr;
+ /// Size in bytes of the run.
+ rle_size_t data_size_ = 0;
+ /// The size in bit of a packed value in the run
+ rle_size_t value_bit_width_ = 0;
Review Comment:
This could probably be `const`, if we don't care about this type being movable. Not sure it would make a difference performance-wise...
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +612,663 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/************************
+ * RleBitPackedParser *
+ ************************/
+
+template <typename Handler>
+void RleBitPackedParser::Parse(Handler&& handler) {
+ while (!exhausted()) {
+ auto [read, control] = PeekImpl(handler);
+ data_ += read;
+ data_size_ -= read;
+ if (ARROW_PREDICT_FALSE(control == ControlFlow::Break)) {
+ break;
+ }
+ }
+}
+
+namespace internal {
+/// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+} // namespace internal
+
+template <typename Handler>
+auto RleBitPackedParser::PeekImpl(Handler&& handler) const
+ -> std::pair<rle_size_t, ControlFlow> {
+ ARROW_DCHECK(!exhausted());
+
+ constexpr auto kMaxSize = bit_util::kMaxLEB128ByteLenFor<uint32_t>;
+ uint32_t run_len_type = 0;
+ const auto header_bytes = bit_util::ParseLeadingLEB128(data_, kMaxSize, &run_len_type);
+
+ if (ARROW_PREDICT_FALSE(header_bytes == 0)) {
+ // Malfomrmed LEB128 data
+ return {};
+ }
+
+ const bool is_bit_packed = run_len_type & 1;
+ const uint32_t count = run_len_type >> 1;
+ if (is_bit_packed) {
+ constexpr auto kMaxCount = bit_util::CeilDiv(internal::max_size_for_v<rle_size_t>, 8);
+ if (ARROW_PREDICT_FALSE(count == 0 || count > kMaxCount)) {
+ // Illegal number of encoded values
+ return {0, ControlFlow::Break};
+ }
+
+ const auto values_count = static_cast<rle_size_t>(count * 8);
+ ARROW_DCHECK_LT(count, internal::max_size_for_v<rle_size_t>);
+ // Count Already divided by 8
+ const auto bytes_read =
+ header_bytes + static_cast<rle_size_t>(count) * value_bit_width_;
+
+ auto control = handler.OnBitPackedRun(
+ BitPackedRun(data_ + header_bytes, values_count, value_bit_width_));
+
+ return {bytes_read, control};
+ }
+
+ if (ARROW_PREDICT_FALSE(
+ count == 0 ||
+ count > static_cast<uint32_t>(std::numeric_limits<rle_size_t>::max()))) {
+ // Illegal number of encoded values
+ return {0, ControlFlow::Break};
+ }
+
+ const auto values_count = static_cast<rle_size_t>(count);
+ const auto value_bytes = bit_util::BytesForBits(value_bit_width_);
+ ARROW_DCHECK_LT(value_bytes, internal::max_size_for_v<rle_size_t>);
+ const auto bytes_read = header_bytes + static_cast<rle_size_t>(value_bytes);
+
+ auto control =
+ handler.OnRleRun(RleRun(data_ + header_bytes, values_count, value_bit_width_));
+
+ return {bytes_read, control};
+}
+
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+template <typename Callable>
+void RleBitPackedDecoder<T>::ParseWithCallable(Callable&& func) {
+ struct {
+ Callable func;
+ auto OnBitPackedRun(BitPackedRun run) { return func(std::move(run)); }
+ auto OnRleRun(RleRun run) { return func(std::move(run)); }
+ } handler{std::move(func)};
+
+ parser_.Parse(std::move(handler));
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, rle_size_t batch_size)
+ -> rle_size_t {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ rle_size_t values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(run_remaining() > 0)) {
+ const auto read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(run_remaining() == 0);
}
+ ParseWithCallable([&](auto run) {
+ using RunDecoder = typename decltype(run)::template DecoderType<value_type>;
+
+ ARROW_DCHECK_LT(values_read, batch_size);
+ RunDecoder decoder(run);
+ const auto read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
Review Comment:
We can just check `read == 0` I think.
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -82,86 +81,400 @@ namespace util {
/// 200 ints = 25 groups of 8
/// <varint((25 << 1) | 1)> <25 bytes of values, bitpacked>
/// (total 26 bytes, 1 byte overhead)
-//
-/// Decoder class for RLE encoded data.
-class RleDecoder {
+/// The type for an encoded Rle of BitPacked run size, between 1 and 2^31-1 as per Parquet
+/// spec.
+/// This is also pragmatically used for other integer used in the Rle and BitPacked runs
+/// and decoder to avoid conversions.
+/// It can therefore be referred to as a "typical" size for Rle and BitPacked logic.
+using rle_size_t = int32_t;
+
+template <typename T>
+class RleRunDecoder;
+
+class RleRun {
public:
- /// Create a decoder object. buffer/buffer_len is the decoded data.
- /// bit_width is the width of each value (before encoding).
- RleDecoder(const uint8_t* buffer, int buffer_len, int bit_width)
- : bit_reader_(buffer, buffer_len),
- bit_width_(bit_width),
- current_value_(0),
- repeat_count_(0),
- literal_count_(0) {
- ARROW_DCHECK_GE(bit_width_, 0);
- ARROW_DCHECK_LE(bit_width_, 64);
+ /// The decoder class used to decode a single run in the given type.
+ template <typename T>
+ using DecoderType = RleRunDecoder<T>;
+
+ constexpr RleRun() noexcept = default;
+
+ explicit RleRun(const uint8_t* data, rle_size_t values_count,
+ rle_size_t value_bit_width) noexcept
+ : values_count_(values_count), value_bit_width_(value_bit_width) {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_GE(values_count, 0);
+ std::copy(data, data + raw_data_size(), data_.begin());
}
- RleDecoder() : bit_width_(-1) {}
+ /// The number of repeated values in this run.
+ constexpr rle_size_t values_count() const noexcept { return values_count_; }
- void Reset(const uint8_t* buffer, int buffer_len, int bit_width) {
- ARROW_DCHECK_GE(bit_width, 0);
- ARROW_DCHECK_LE(bit_width, 64);
- bit_reader_.Reset(buffer, buffer_len);
- bit_width_ = bit_width;
- current_value_ = 0;
- repeat_count_ = 0;
- literal_count_ = 0;
+ /// The size in bits of each encoded value.
+ constexpr rle_size_t values_bit_width() const noexcept { return value_bit_width_; }
+
+ /// A pointer to the repeated value raw bytes.
+ constexpr const uint8_t* raw_data_ptr() const noexcept { return data_.data(); }
+
+ /// The number of bytes used for the raw repeated value.
+ constexpr rle_size_t raw_data_size() const noexcept {
+ auto out = bit_util::BytesForBits(value_bit_width_);
+ ARROW_DCHECK_LE(out, std::numeric_limits<rle_size_t>::max());
+ return static_cast<rle_size_t>(out);
}
- /// Gets the next value. Returns false if there are no more.
+ private:
+ /// The repeated value raw bytes stored inside the class with enough space to store
+ /// up to a 64 bit value.
+ std::array<uint8_t, 8> data_ = {};
+ /// The number of time the value is repeated.
+ rle_size_t values_count_ = 0;
+ /// The size in bit of a packed value in the run.
+ rle_size_t value_bit_width_ = 0;
+};
+
+template <typename T>
+class BitPackedRunDecoder;
+
+class BitPackedRun {
+ public:
+ /// The decoder class used to decode a single run in the given type.
+ template <typename T>
+ using DecoderType = BitPackedRunDecoder<T>;
+
+ constexpr BitPackedRun() noexcept = default;
+
+ constexpr BitPackedRun(const uint8_t* data, rle_size_t values_count,
+ rle_size_t value_bit_width) noexcept
+ : data_(data), values_count_(values_count), value_bit_width_(value_bit_width) {
+ ARROW_CHECK_GE(value_bit_width_, 0);
+ ARROW_CHECK_GE(values_count_, 0);
+ }
+
+ constexpr rle_size_t values_count() const noexcept { return values_count_; }
+
+ /// The size in bits of each encoded value.
+ constexpr rle_size_t values_bit_width() const noexcept { return value_bit_width_; }
+
+ constexpr const uint8_t* raw_data_ptr() const noexcept { return data_; }
+
+ constexpr rle_size_t raw_data_size() const noexcept {
+ auto out = bit_util::BytesForBits(static_cast<int64_t>(value_bit_width_) *
+ static_cast<int64_t>(values_count_));
+ ARROW_CHECK_LE(out, std::numeric_limits<rle_size_t>::max());
+ return static_cast<rle_size_t>(out);
+ }
+
+ private:
+ /// The pointer to the beginning of the run
+ const uint8_t* data_ = nullptr;
+ /// Number of values in this run.
+ rle_size_t values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ rle_size_t value_bit_width_ = 0;
Review Comment:
Same question of course.
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +612,663 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/************************
+ * RleBitPackedParser *
+ ************************/
+
+template <typename Handler>
+void RleBitPackedParser::Parse(Handler&& handler) {
+ while (!exhausted()) {
+ auto [read, control] = PeekImpl(handler);
+ data_ += read;
+ data_size_ -= read;
+ if (ARROW_PREDICT_FALSE(control == ControlFlow::Break)) {
+ break;
+ }
+ }
+}
+
+namespace internal {
+/// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+} // namespace internal
+
+template <typename Handler>
+auto RleBitPackedParser::PeekImpl(Handler&& handler) const
+ -> std::pair<rle_size_t, ControlFlow> {
+ ARROW_DCHECK(!exhausted());
+
+ constexpr auto kMaxSize = bit_util::kMaxLEB128ByteLenFor<uint32_t>;
+ uint32_t run_len_type = 0;
+ const auto header_bytes = bit_util::ParseLeadingLEB128(data_, kMaxSize, &run_len_type);
+
+ if (ARROW_PREDICT_FALSE(header_bytes == 0)) {
+ // Malfomrmed LEB128 data
+ return {};
Review Comment:
Let's spell `{0, ControlFlow::Break}` explicitly?
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +612,663 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/************************
+ * RleBitPackedParser *
+ ************************/
+
+template <typename Handler>
+void RleBitPackedParser::Parse(Handler&& handler) {
+ while (!exhausted()) {
+ auto [read, control] = PeekImpl(handler);
+ data_ += read;
+ data_size_ -= read;
+ if (ARROW_PREDICT_FALSE(control == ControlFlow::Break)) {
+ break;
+ }
+ }
+}
+
+namespace internal {
+/// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+} // namespace internal
+
+template <typename Handler>
+auto RleBitPackedParser::PeekImpl(Handler&& handler) const
+ -> std::pair<rle_size_t, ControlFlow> {
+ ARROW_DCHECK(!exhausted());
+
+ constexpr auto kMaxSize = bit_util::kMaxLEB128ByteLenFor<uint32_t>;
+ uint32_t run_len_type = 0;
+ const auto header_bytes = bit_util::ParseLeadingLEB128(data_, kMaxSize, &run_len_type);
+
+ if (ARROW_PREDICT_FALSE(header_bytes == 0)) {
+ // Malfomrmed LEB128 data
+ return {};
+ }
+
+ const bool is_bit_packed = run_len_type & 1;
+ const uint32_t count = run_len_type >> 1;
+ if (is_bit_packed) {
+ constexpr auto kMaxCount = bit_util::CeilDiv(internal::max_size_for_v<rle_size_t>, 8);
+ if (ARROW_PREDICT_FALSE(count == 0 || count > kMaxCount)) {
+ // Illegal number of encoded values
+ return {0, ControlFlow::Break};
+ }
+
+ const auto values_count = static_cast<rle_size_t>(count * 8);
+ ARROW_DCHECK_LT(count, internal::max_size_for_v<rle_size_t>);
Review Comment:
Do you mean `values_count` here?
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +612,663 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/************************
+ * RleBitPackedParser *
+ ************************/
+
+template <typename Handler>
+void RleBitPackedParser::Parse(Handler&& handler) {
+ while (!exhausted()) {
+ auto [read, control] = PeekImpl(handler);
+ data_ += read;
+ data_size_ -= read;
+ if (ARROW_PREDICT_FALSE(control == ControlFlow::Break)) {
+ break;
+ }
+ }
+}
+
+namespace internal {
+/// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+} // namespace internal
+
+template <typename Handler>
+auto RleBitPackedParser::PeekImpl(Handler&& handler) const
+ -> std::pair<rle_size_t, ControlFlow> {
+ ARROW_DCHECK(!exhausted());
+
+ constexpr auto kMaxSize = bit_util::kMaxLEB128ByteLenFor<uint32_t>;
+ uint32_t run_len_type = 0;
+ const auto header_bytes = bit_util::ParseLeadingLEB128(data_, kMaxSize, &run_len_type);
+
+ if (ARROW_PREDICT_FALSE(header_bytes == 0)) {
+ // Malfomrmed LEB128 data
+ return {};
+ }
+
+ const bool is_bit_packed = run_len_type & 1;
+ const uint32_t count = run_len_type >> 1;
+ if (is_bit_packed) {
+ constexpr auto kMaxCount = bit_util::CeilDiv(internal::max_size_for_v<rle_size_t>, 8);
+ if (ARROW_PREDICT_FALSE(count == 0 || count > kMaxCount)) {
+ // Illegal number of encoded values
+ return {0, ControlFlow::Break};
+ }
+
+ const auto values_count = static_cast<rle_size_t>(count * 8);
+ ARROW_DCHECK_LT(count, internal::max_size_for_v<rle_size_t>);
+ // Count Already divided by 8
+ const auto bytes_read =
+ header_bytes + static_cast<rle_size_t>(count) * value_bit_width_;
+
+ auto control = handler.OnBitPackedRun(
+ BitPackedRun(data_ + header_bytes, values_count, value_bit_width_));
+
+ return {bytes_read, control};
+ }
+
+ if (ARROW_PREDICT_FALSE(
+ count == 0 ||
+ count > static_cast<uint32_t>(std::numeric_limits<rle_size_t>::max()))) {
Review Comment:
This is not possible, right? `count` is initially in [0, 2^32 - 1], but then it's shifted to the right by one bit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "wgtmac (via GitHub)" <gi...@apache.org>.
wgtmac commented on PR #47294:
URL: https://github.com/apache/arrow/pull/47294#issuecomment-4047727372
> We tested using a Parquet file with data generated from SSB Flat. We found that the less the number of distinct values, the more the performance regression.
>
> * `parquet-scan-main` is compiled from the latest commit of the main branch.
> * `parquet-scan-old` is compiled from `64f2055ffb68e5077420f4253e76d78952438cab` which is the previous commit of this PR on the main branch.
>
> Both are compiled in release mode with `-DARROW_RUNTIME_SIMD_LEVEL=AVX2` cmake flag.
>
> ```shell
> ❯ env LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$LLVM_ROOT/lib/x86_64-unknown-linux-gnu ARROW_RUNTIME_SIMD_LEVEL=AVX2 hyperfine -w 5 -r 20 --sort mean-time "cpp/out/build/ninja-release/release/parquet-scan-main --columns=6 /dev/shm/TableSink0" "cpp/out/build/ninja-release/release/parquet-scan-old --columns=6 /dev/shm/TableSink0"
> Benchmark 1: cpp/out/build/ninja-release/release/parquet-scan-main --columns=6 /dev/shm/TableSink0
> Time (mean ± σ): 31.1 ms ± 0.3 ms [User: 27.2 ms, System: 3.7 ms]
> Range (min … max): 30.7 ms … 31.8 ms 20 runs
>
> Benchmark 2: cpp/out/build/ninja-release/release/parquet-scan-old --columns=6 /dev/shm/TableSink0
> Time (mean ± σ): 22.8 ms ± 0.4 ms [User: 19.3 ms, System: 3.3 ms]
> Range (min … max): 22.2 ms … 23.3 ms 20 runs
>
> Summary
> cpp/out/build/ninja-release/release/parquet-scan-old --columns=6 /dev/shm/TableSink0 ran
> 1.37 ± 0.02 times faster than cpp/out/build/ninja-release/release/parquet-scan-main --columns=6 /dev/shm/TableSink0
>
> ❯ env LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$LLVM_ROOT/lib/x86_64-unknown-linux-gnu cpp/out/build/ninja-release/release/parquet-reader --only-metadata /dev/shm/TableSink0
> File Name: /dev/shm/TableSink0
> Version: 2.6
> Created By: cz-cpp version BuildInfo:GitBranch:release/20240820_rc8,GitVersion:73a4383,BuildTime:1725298762,CloudEnv:ALIYUN
> Total rows: 6250733
> Number of RowGroups: 1
> Number of Real Columns: 40
> Number of Columns: 40
> Number of Selected Columns: 40
> ......
> Column 6: lo_orderpriority (BYTE_ARRAY / String / UTF8)
> ......
> --- Row Group: 0 ---
> --- Total Bytes: 1016215218 ---
> --- Total Compressed Bytes: 552911018 ---
> --- Sort Columns:
> column_idx: 5, descending: 0, nulls_first: 1
> column_idx: 0, descending: 0, nulls_first: 1
> --- Rows: 6250733 ---
> ......
> Column 6
> Values: 6250733, Null Values: 0, Distinct Values: 5
> Max (exact: unknown): 5-LOW, Min (exact: unknown): 1-URGENT
> Compression: LZ4_RAW, Encodings: PLAIN(DICT_PAGE) RLE_DICTIONARY
> Uncompressed Size: 2267694, Compressed Size: 2092132
> ......
> ```
Update from @HuaHuaY's comment: we just located the performance regression comes from the API change of the internal `unpack` function introduced by https://github.com/apache/arrow/pull/47994. The quick fix is to explicitly set `UnpackOptions::max_read_bytes` to not use its default value -1. @AntoinePrv @pitrou
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "wgtmac (via GitHub)" <gi...@apache.org>.
wgtmac commented on PR #47294:
URL: https://github.com/apache/arrow/pull/47294#issuecomment-4047933899
That's in our internal codebase where the `unpack` function is called somewhere else. I made a mistake by not setting it explicitly :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2322131764
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -207,12 +209,310 @@ TEST(BitUtil, RoundTripIntValues) {
}
}
+/// A Rle run is a simple class owning some data and a repetition count.
+/// It does not know how to read such data.
+TEST(Rle, RleRun) {
+ const std::array<RleRun::byte, 4> value = {21, 2, 0, 0};
+
+ RleRun::values_count_type value_count = 12;
+
+ // 12 times the value 21 fitting over 5 bits
+ auto const run_5 = RleRun(value.data(), value_count, /* value_bit_width= */ 5);
+ EXPECT_EQ(run_5.ValuesCount(), value_count);
+ EXPECT_EQ(run_5.ValuesBitWidth(), 5);
+ EXPECT_EQ(run_5.RawDataSize(), 1); // 5 bits fit in one byte
+ EXPECT_EQ(*run_5.RawDataPtr(), 21);
+
+ // 12 times the value 21 fitting over 16 bits
+ auto const run_8 = RleRun(value.data(), value_count, /* value_bit_width= */ 8);
+ EXPECT_EQ(run_8.ValuesCount(), value_count);
+ EXPECT_EQ(run_8.ValuesBitWidth(), 8);
+ EXPECT_EQ(run_8.RawDataSize(), 1); // 8 bits fit in 1 byte
+ EXPECT_EQ(*run_8.RawDataPtr(), 21);
+
+ // 12 times the value {21, 2} fitting over 10 bits
+ auto const run_10 = RleRun(value.data(), value_count, /* value_bit_width= */ 10);
+
+ EXPECT_EQ(run_10.ValuesCount(), value_count);
+ EXPECT_EQ(run_10.ValuesBitWidth(), 10);
+ EXPECT_EQ(run_10.RawDataSize(), 2); // 10 bits fit in 2 bytes
+ EXPECT_EQ(*(run_10.RawDataPtr() + 0), 21);
+ EXPECT_EQ(*(run_10.RawDataPtr() + 1), 2);
+
+ // 12 times the value {21, 2} fitting over 32 bits
+ auto const run_32 = RleRun(value.data(), value_count, /* value_bit_width= */ 32);
+ EXPECT_EQ(run_32.ValuesCount(), value_count);
+ EXPECT_EQ(run_32.ValuesBitWidth(), 32);
+ EXPECT_EQ(run_32.RawDataSize(), 4); // 32 bits fit in 4 bytes
+ EXPECT_EQ(*(run_32.RawDataPtr() + 0), 21);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 1), 2);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 2), 0);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 3), 0);
+}
+
+/// A BitPacked run is a simple class owning some data and its size.
+/// It does not know how to read such data.
+TEST(BitPacked, BitPackedRun) {
+ const std::array<BitPackedRun::byte, 4> value = {0b10101010, 0, 0, 0b1111111};
+
+ /// 16 values of 1 bit for a total of 16 bits
+ BitPackedRun::values_count_type value_count_1 = 16;
+ auto const run_1 = BitPackedRun(value.data(), value_count_1, /* value_bit_width= */ 1);
+ EXPECT_EQ(run_1.ValuesCount(), value_count_1);
+ EXPECT_EQ(run_1.ValuesBitWidth(), 1);
+ EXPECT_EQ(run_1.RawDataSize(), 2); // 16 bits fit in 2 bytes
+ for (BitPackedRun::raw_data_size_type i = 0; i < run_1.RawDataSize(); ++i) {
+ EXPECT_EQ(*(run_1.RawDataPtr() + i), value[i]);
+ }
+
+ /// 8 values of 3 bits for a total of 24 bits
+ BitPackedRun::values_count_type value_count_3 = 8;
+ auto const run_3 = BitPackedRun(value.data(), value_count_3, /* value_bit_width= */ 3);
+ EXPECT_EQ(run_3.ValuesCount(), value_count_3);
+ EXPECT_EQ(run_3.ValuesBitWidth(), 3);
+ EXPECT_EQ(run_3.RawDataSize(), 3); // 24 bits fit in 3 bytes
+ for (BitPackedRun::raw_data_size_type i = 0; i < run_3.RawDataSize(); ++i) {
+ EXPECT_EQ(*(run_3.RawDataPtr() + i), value[i]);
+ }
+}
+
+template <typename T>
+void TestRleDecoder(std::vector<RleRun::byte> bytes,
+ RleRun::values_count_type value_count,
+ RleRun::bit_size_type bit_width) {
+ // Pre-requisite for this test
+ EXPECT_GT(value_count, 6);
+
+ // Compute value associated with bytes encoded as little endian
+ T value = 0;
+ for (std::size_t i = 0; i < bytes.size(); ++i) {
+ value += static_cast<T>(bytes.at(i)) << (8 * i);
+ }
+
+ auto const run = RleRun(bytes.data(), value_count, bit_width);
+
+ auto decoder = RleDecoder<T>(run);
+ std::vector<T> vals = {0, 0};
+
+ EXPECT_EQ(decoder.Remaining(), value_count);
+
+ typename decltype(decoder)::values_count_type read = 0;
+ EXPECT_EQ(decoder.Get(vals.data()), 1);
+ read += 1;
+ EXPECT_EQ(vals.at(0), value);
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ EXPECT_EQ(decoder.Advance(3), 3);
+ read += 3;
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ vals = {0, 0};
+ EXPECT_EQ(decoder.GetBatch(vals.data(), 2), vals.size());
+ EXPECT_EQ(vals.at(0), value);
+ EXPECT_EQ(vals.at(1), value);
+ read += static_cast<decltype(read)>(vals.size());
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ // Exhaust iteration
+ EXPECT_EQ(decoder.Advance(value_count - read), value_count - read);
+ EXPECT_EQ(decoder.Remaining(), 0);
+ EXPECT_EQ(decoder.Advance(1), 0);
+ vals = {0, 0};
+ EXPECT_EQ(decoder.Get(vals.data()), 0);
+ EXPECT_EQ(vals.at(0), 0);
+
+ // Reset the decoder
+ decoder.Reset(run);
+ EXPECT_EQ(decoder.Remaining(), value_count);
+ vals = {0, 0};
+ EXPECT_EQ(decoder.GetBatch(vals.data(), 2), vals.size());
+ EXPECT_EQ(vals.at(0), value);
+ EXPECT_EQ(vals.at(1), value);
+}
+
+TEST(Rle, RleDecoder) {
+ TestRleDecoder<uint32_t>({21, 0, 0}, /* value_count= */ 21, /* bit_width= */ 5);
+ TestRleDecoder<uint16_t>({1, 0}, /* value_count= */ 13, /* bit_width= */ 1);
+ TestRleDecoder<uint64_t>({21, 2, 0, 1}, /* value_count= */ 20, /* bit_width= */ 30);
+}
+
+template <typename T>
+void TestBitPackedDecoder(std::vector<RleRun::byte> bytes,
+ BitPackedRun::values_count_type value_count,
+ BitPackedRun::bit_size_type bit_width,
+ std::vector<T> expected) {
+ // Pre-requisite for this test
+ EXPECT_GT(value_count, 6);
+
+ auto const run = BitPackedRun(bytes.data(), value_count, bit_width);
+
+ auto decoder = BitPackedDecoder<T>(run);
+ std::vector<T> vals = {0, 0};
+
+ EXPECT_EQ(decoder.Remaining(), value_count);
+
+ typename decltype(decoder)::values_count_type read = 0;
+ EXPECT_EQ(decoder.Get(vals.data()), 1);
+ EXPECT_EQ(vals.at(0), expected.at(0 + read));
+ read += 1;
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ EXPECT_EQ(decoder.Advance(3), 3);
+ read += 3;
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ vals = {0, 0};
+ EXPECT_EQ(decoder.GetBatch(vals.data(), 2), vals.size());
+ EXPECT_EQ(vals.at(0), expected.at(0 + read));
+ EXPECT_EQ(vals.at(1), expected.at(1 + read));
+ read += static_cast<decltype(read)>(vals.size());
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ // Exhaust iteration
+ EXPECT_EQ(decoder.Advance(value_count - read), value_count - read);
+ EXPECT_EQ(decoder.Remaining(), 0);
+ EXPECT_EQ(decoder.Advance(1), 0);
+ vals = {0, 0};
+ EXPECT_EQ(decoder.Get(vals.data()), 0);
+ EXPECT_EQ(vals.at(0), 0);
+
+ // Reset the decoder
+ decoder.Reset(run);
+ read = 0;
+ EXPECT_EQ(decoder.Remaining(), value_count);
+ vals = {0, 0};
+ EXPECT_EQ(decoder.GetBatch(vals.data(), 2), vals.size());
+ EXPECT_EQ(vals.at(0), expected.at(0 + read));
+ EXPECT_EQ(vals.at(1), expected.at(1 + read));
+}
+
+TEST(BitPacked, BitPackedDecoder) {
+ /// See parquet encoding for bytes layout
+ TestBitPackedDecoder<uint16_t>(
+ /* bytes= */ {0x88, 0xc6, 0xfa},
+ /* values_count= */ 8,
+ /* bit_width= */ 3,
+ /* expected= */ {0, 1, 2, 3, 4, 5, 6, 7});
+ TestBitPackedDecoder<uint8_t>(
+ /* bytes= */ {0x0, 0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7},
+ /* values_count= */ 8,
+ /* bit_width= */ 8,
+ /* expected= */ {0, 1, 2, 3, 4, 5, 6, 7});
+ TestBitPackedDecoder<uint32_t>(
+ /* bytes= */ {0x47, 0xc, 0x10, 0x35},
+ /* values_count= */ 8,
+ /* bit_width= */ 4,
+ /* expected= */ {7, 4, 12, 0, 0, 1, 5, 3});
+ TestBitPackedDecoder<uint64_t>(
+ /* bytes= */ {0xe8, 0x7, 0x20, 0xc0, 0x0, 0x4, 0x14, 0x60, 0xc0, 0x1},
+ /* values_count= */ 8,
+ /* bit_width= */ 10,
+ /* expected= */ {1000, 1, 2, 3, 4, 5, 6, 7});
+}
+
+template <typename T>
+void TestRleBitPackedParser(std::vector<RleRun::byte> bytes,
+ RleBitPackedParser::bit_size_type bit_width,
+ std::vector<T> expected) {
+ auto parser = RleBitPackedParser(
+ bytes.data(), static_cast<BitPackedRun::raw_data_size_type>(bytes.size()),
+ bit_width);
+ EXPECT_FALSE(parser.Exhausted());
+
+ // Peek return the same data
+ auto run1 = parser.Peek();
+ EXPECT_TRUE(run1.has_value());
+ auto run2 = parser.Peek();
+ EXPECT_TRUE(run2.has_value());
+ auto ptr1 = std::visit([](auto const& r) { return r.RawDataPtr(); }, run1.value());
+ auto size1 = std::visit([](auto const& r) { return r.RawDataSize(); }, run1.value());
+ auto ptr2 = std::visit([](auto const& r) { return r.RawDataPtr(); }, run2.value());
+ auto size2 = std::visit([](auto const& r) { return r.RawDataSize(); }, run2.value());
+ EXPECT_TRUE(std::equal(ptr1, ptr1 + size1, ptr2, ptr2 + size2));
+ EXPECT_FALSE(parser.Exhausted());
+
+ // Try to decode all data of all runs in the decoded vector
+ decltype(expected) decoded = {};
+ auto rle_decoder = RleDecoder<T>();
+ auto bit_packed_decoder = BitPackedDecoder<T>();
+ // Iterate over all runs
+ while (auto run = parser.Next()) {
+ EXPECT_TRUE(run.has_value());
+
+ if (std::holds_alternative<RleRun>(run.value())) {
+ rle_decoder.Reset(std::get<RleRun>(run.value()));
+
+ auto const n_decoded = decoded.size();
+ auto const n_to_decode = rle_decoder.Remaining();
+ decoded.resize(n_decoded + n_to_decode);
+ EXPECT_EQ(rle_decoder.GetBatch(decoded.data() + n_decoded, n_to_decode),
+ n_to_decode);
+ EXPECT_EQ(rle_decoder.Remaining(), 0);
+ } else {
+ bit_packed_decoder.Reset(std::get<BitPackedRun>(run.value()));
Review Comment:
Same here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2322130602
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -207,12 +209,310 @@ TEST(BitUtil, RoundTripIntValues) {
}
}
+/// A Rle run is a simple class owning some data and a repetition count.
+/// It does not know how to read such data.
+TEST(Rle, RleRun) {
+ const std::array<RleRun::byte, 4> value = {21, 2, 0, 0};
+
+ RleRun::values_count_type value_count = 12;
+
+ // 12 times the value 21 fitting over 5 bits
+ auto const run_5 = RleRun(value.data(), value_count, /* value_bit_width= */ 5);
+ EXPECT_EQ(run_5.ValuesCount(), value_count);
+ EXPECT_EQ(run_5.ValuesBitWidth(), 5);
+ EXPECT_EQ(run_5.RawDataSize(), 1); // 5 bits fit in one byte
+ EXPECT_EQ(*run_5.RawDataPtr(), 21);
+
+ // 12 times the value 21 fitting over 16 bits
+ auto const run_8 = RleRun(value.data(), value_count, /* value_bit_width= */ 8);
+ EXPECT_EQ(run_8.ValuesCount(), value_count);
+ EXPECT_EQ(run_8.ValuesBitWidth(), 8);
+ EXPECT_EQ(run_8.RawDataSize(), 1); // 8 bits fit in 1 byte
+ EXPECT_EQ(*run_8.RawDataPtr(), 21);
+
+ // 12 times the value {21, 2} fitting over 10 bits
+ auto const run_10 = RleRun(value.data(), value_count, /* value_bit_width= */ 10);
+
+ EXPECT_EQ(run_10.ValuesCount(), value_count);
+ EXPECT_EQ(run_10.ValuesBitWidth(), 10);
+ EXPECT_EQ(run_10.RawDataSize(), 2); // 10 bits fit in 2 bytes
+ EXPECT_EQ(*(run_10.RawDataPtr() + 0), 21);
+ EXPECT_EQ(*(run_10.RawDataPtr() + 1), 2);
+
+ // 12 times the value {21, 2} fitting over 32 bits
+ auto const run_32 = RleRun(value.data(), value_count, /* value_bit_width= */ 32);
+ EXPECT_EQ(run_32.ValuesCount(), value_count);
+ EXPECT_EQ(run_32.ValuesBitWidth(), 32);
+ EXPECT_EQ(run_32.RawDataSize(), 4); // 32 bits fit in 4 bytes
+ EXPECT_EQ(*(run_32.RawDataPtr() + 0), 21);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 1), 2);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 2), 0);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 3), 0);
+}
+
+/// A BitPacked run is a simple class owning some data and its size.
+/// It does not know how to read such data.
+TEST(BitPacked, BitPackedRun) {
+ const std::array<BitPackedRun::byte, 4> value = {0b10101010, 0, 0, 0b1111111};
+
+ /// 16 values of 1 bit for a total of 16 bits
+ BitPackedRun::values_count_type value_count_1 = 16;
+ auto const run_1 = BitPackedRun(value.data(), value_count_1, /* value_bit_width= */ 1);
+ EXPECT_EQ(run_1.ValuesCount(), value_count_1);
+ EXPECT_EQ(run_1.ValuesBitWidth(), 1);
+ EXPECT_EQ(run_1.RawDataSize(), 2); // 16 bits fit in 2 bytes
+ for (BitPackedRun::raw_data_size_type i = 0; i < run_1.RawDataSize(); ++i) {
+ EXPECT_EQ(*(run_1.RawDataPtr() + i), value[i]);
+ }
+
+ /// 8 values of 3 bits for a total of 24 bits
+ BitPackedRun::values_count_type value_count_3 = 8;
+ auto const run_3 = BitPackedRun(value.data(), value_count_3, /* value_bit_width= */ 3);
+ EXPECT_EQ(run_3.ValuesCount(), value_count_3);
+ EXPECT_EQ(run_3.ValuesBitWidth(), 3);
+ EXPECT_EQ(run_3.RawDataSize(), 3); // 24 bits fit in 3 bytes
+ for (BitPackedRun::raw_data_size_type i = 0; i < run_3.RawDataSize(); ++i) {
+ EXPECT_EQ(*(run_3.RawDataPtr() + i), value[i]);
+ }
+}
+
+template <typename T>
+void TestRleDecoder(std::vector<RleRun::byte> bytes,
+ RleRun::values_count_type value_count,
+ RleRun::bit_size_type bit_width) {
+ // Pre-requisite for this test
+ EXPECT_GT(value_count, 6);
+
+ // Compute value associated with bytes encoded as little endian
+ T value = 0;
+ for (std::size_t i = 0; i < bytes.size(); ++i) {
+ value += static_cast<T>(bytes.at(i)) << (8 * i);
+ }
+
+ auto const run = RleRun(bytes.data(), value_count, bit_width);
+
+ auto decoder = RleDecoder<T>(run);
+ std::vector<T> vals = {0, 0};
+
+ EXPECT_EQ(decoder.Remaining(), value_count);
+
+ typename decltype(decoder)::values_count_type read = 0;
+ EXPECT_EQ(decoder.Get(vals.data()), 1);
+ read += 1;
+ EXPECT_EQ(vals.at(0), value);
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ EXPECT_EQ(decoder.Advance(3), 3);
+ read += 3;
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ vals = {0, 0};
+ EXPECT_EQ(decoder.GetBatch(vals.data(), 2), vals.size());
+ EXPECT_EQ(vals.at(0), value);
+ EXPECT_EQ(vals.at(1), value);
+ read += static_cast<decltype(read)>(vals.size());
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ // Exhaust iteration
+ EXPECT_EQ(decoder.Advance(value_count - read), value_count - read);
+ EXPECT_EQ(decoder.Remaining(), 0);
+ EXPECT_EQ(decoder.Advance(1), 0);
+ vals = {0, 0};
+ EXPECT_EQ(decoder.Get(vals.data()), 0);
+ EXPECT_EQ(vals.at(0), 0);
+
+ // Reset the decoder
+ decoder.Reset(run);
+ EXPECT_EQ(decoder.Remaining(), value_count);
+ vals = {0, 0};
+ EXPECT_EQ(decoder.GetBatch(vals.data(), 2), vals.size());
+ EXPECT_EQ(vals.at(0), value);
+ EXPECT_EQ(vals.at(1), value);
+}
+
+TEST(Rle, RleDecoder) {
+ TestRleDecoder<uint32_t>({21, 0, 0}, /* value_count= */ 21, /* bit_width= */ 5);
+ TestRleDecoder<uint16_t>({1, 0}, /* value_count= */ 13, /* bit_width= */ 1);
+ TestRleDecoder<uint64_t>({21, 2, 0, 1}, /* value_count= */ 20, /* bit_width= */ 30);
+}
+
+template <typename T>
+void TestBitPackedDecoder(std::vector<RleRun::byte> bytes,
+ BitPackedRun::values_count_type value_count,
+ BitPackedRun::bit_size_type bit_width,
+ std::vector<T> expected) {
+ // Pre-requisite for this test
+ EXPECT_GT(value_count, 6);
+
+ auto const run = BitPackedRun(bytes.data(), value_count, bit_width);
+
+ auto decoder = BitPackedDecoder<T>(run);
+ std::vector<T> vals = {0, 0};
+
+ EXPECT_EQ(decoder.Remaining(), value_count);
+
+ typename decltype(decoder)::values_count_type read = 0;
+ EXPECT_EQ(decoder.Get(vals.data()), 1);
+ EXPECT_EQ(vals.at(0), expected.at(0 + read));
+ read += 1;
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ EXPECT_EQ(decoder.Advance(3), 3);
+ read += 3;
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ vals = {0, 0};
+ EXPECT_EQ(decoder.GetBatch(vals.data(), 2), vals.size());
+ EXPECT_EQ(vals.at(0), expected.at(0 + read));
+ EXPECT_EQ(vals.at(1), expected.at(1 + read));
+ read += static_cast<decltype(read)>(vals.size());
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ // Exhaust iteration
+ EXPECT_EQ(decoder.Advance(value_count - read), value_count - read);
+ EXPECT_EQ(decoder.Remaining(), 0);
+ EXPECT_EQ(decoder.Advance(1), 0);
+ vals = {0, 0};
+ EXPECT_EQ(decoder.Get(vals.data()), 0);
+ EXPECT_EQ(vals.at(0), 0);
+
+ // Reset the decoder
+ decoder.Reset(run);
+ read = 0;
+ EXPECT_EQ(decoder.Remaining(), value_count);
+ vals = {0, 0};
+ EXPECT_EQ(decoder.GetBatch(vals.data(), 2), vals.size());
+ EXPECT_EQ(vals.at(0), expected.at(0 + read));
+ EXPECT_EQ(vals.at(1), expected.at(1 + read));
+}
+
+TEST(BitPacked, BitPackedDecoder) {
+ /// See parquet encoding for bytes layout
+ TestBitPackedDecoder<uint16_t>(
+ /* bytes= */ {0x88, 0xc6, 0xfa},
+ /* values_count= */ 8,
+ /* bit_width= */ 3,
+ /* expected= */ {0, 1, 2, 3, 4, 5, 6, 7});
+ TestBitPackedDecoder<uint8_t>(
+ /* bytes= */ {0x0, 0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7},
+ /* values_count= */ 8,
+ /* bit_width= */ 8,
+ /* expected= */ {0, 1, 2, 3, 4, 5, 6, 7});
+ TestBitPackedDecoder<uint32_t>(
+ /* bytes= */ {0x47, 0xc, 0x10, 0x35},
+ /* values_count= */ 8,
+ /* bit_width= */ 4,
+ /* expected= */ {7, 4, 12, 0, 0, 1, 5, 3});
+ TestBitPackedDecoder<uint64_t>(
+ /* bytes= */ {0xe8, 0x7, 0x20, 0xc0, 0x0, 0x4, 0x14, 0x60, 0xc0, 0x1},
+ /* values_count= */ 8,
+ /* bit_width= */ 10,
+ /* expected= */ {1000, 1, 2, 3, 4, 5, 6, 7});
+}
+
+template <typename T>
+void TestRleBitPackedParser(std::vector<RleRun::byte> bytes,
+ RleBitPackedParser::bit_size_type bit_width,
+ std::vector<T> expected) {
+ auto parser = RleBitPackedParser(
+ bytes.data(), static_cast<BitPackedRun::raw_data_size_type>(bytes.size()),
+ bit_width);
+ EXPECT_FALSE(parser.Exhausted());
+
+ // Peek return the same data
+ auto run1 = parser.Peek();
+ EXPECT_TRUE(run1.has_value());
+ auto run2 = parser.Peek();
+ EXPECT_TRUE(run2.has_value());
+ auto ptr1 = std::visit([](auto const& r) { return r.RawDataPtr(); }, run1.value());
+ auto size1 = std::visit([](auto const& r) { return r.RawDataSize(); }, run1.value());
+ auto ptr2 = std::visit([](auto const& r) { return r.RawDataPtr(); }, run2.value());
+ auto size2 = std::visit([](auto const& r) { return r.RawDataSize(); }, run2.value());
+ EXPECT_TRUE(std::equal(ptr1, ptr1 + size1, ptr2, ptr2 + size2));
+ EXPECT_FALSE(parser.Exhausted());
+
+ // Try to decode all data of all runs in the decoded vector
+ decltype(expected) decoded = {};
+ auto rle_decoder = RleDecoder<T>();
+ auto bit_packed_decoder = BitPackedDecoder<T>();
+ // Iterate over all runs
+ while (auto run = parser.Next()) {
+ EXPECT_TRUE(run.has_value());
+
+ if (std::holds_alternative<RleRun>(run.value())) {
+ rle_decoder.Reset(std::get<RleRun>(run.value()));
Review Comment:
You could directly declare `rle_decoder` here, there is not point in having being in the outer scope.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2319514772
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ ARROW_DCHECK_LT(values_read, batch_size);
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+ auto const read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
return values_read;
}
-template <typename T, typename RunType, typename Converter>
-inline int RleDecoder::GetSpaced(Converter converter, int batch_size, int null_count,
- const uint8_t* valid_bits, int64_t valid_bits_offset,
- T* out) {
- if (ARROW_PREDICT_FALSE(null_count == batch_size)) {
- converter.FillZero(out, out + batch_size);
- return batch_size;
+namespace internal {
+
+/// Utility class to safely handle values and null count without too error-prone
+/// verbosity.
+class BatchCounter {
+ public:
+ using size_type = int32_t;
+
+ [[nodiscard]] static constexpr BatchCounter FromBatchSizeAndNulls(
+ size_type batch_size, size_type null_count) {
+ ARROW_DCHECK_LE(null_count, batch_size);
+ return {batch_size - null_count, null_count};
}
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
- int values_remaining = batch_size - null_count;
+ constexpr BatchCounter(size_type values_count, size_type null_count) noexcept
+ : values_count_(values_count), null_count_(null_count) {}
- // Assume no bits to start.
- arrow::internal::BitRunReader bit_reader(valid_bits, valid_bits_offset,
- /*length=*/batch_size);
- arrow::internal::BitRun valid_run = bit_reader.NextRun();
- while (values_read < batch_size) {
- if (ARROW_PREDICT_FALSE(valid_run.length == 0)) {
- valid_run = bit_reader.NextRun();
+ [[nodiscard]] constexpr size_type ValuesCount() const noexcept { return values_count_; }
+
+ [[nodiscard]] constexpr size_type ValuesRead() const noexcept { return values_read_; }
+
+ [[nodiscard]] constexpr size_type ValuesRemaining() const noexcept {
+ ARROW_DCHECK_LE(values_read_, values_count_);
+ return values_count_ - values_read_;
+ }
+
+ constexpr void AccrueReadValues(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, ValuesRemaining());
+ values_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type NullCount() const noexcept { return null_count_; }
+
+ [[nodiscard]] constexpr size_type NullRead() const noexcept { return null_read_; }
+
+ [[nodiscard]] constexpr size_type NullRemaining() const noexcept {
+ ARROW_DCHECK_LE(null_read_, null_count_);
+ return null_count_ - null_read_;
+ }
+
+ constexpr void AccrueReadNulls(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, NullRemaining());
+ null_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type TotalRemaining() const noexcept {
+ return ValuesRemaining() + NullRemaining();
+ }
+
+ [[nodiscard]] constexpr size_type TotalRead() const noexcept {
+ return values_read_ + null_read_;
+ }
+
+ [[nodiscard]] constexpr bool IsFullyNull() const noexcept {
+ return ValuesRemaining() == 0;
+ }
+
+ [[nodiscard]] constexpr bool IsDone() const noexcept { return TotalRemaining() == 0; }
+
+ private:
+ size_type values_count_ = 0;
+ size_type values_read_ = 0;
+ size_type null_count_ = 0;
+ size_type null_read_ = 0;
+};
+
+// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+/// Overload for GetSpaced for a single run in a RleDecoder
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ RleDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto values_remaining_run = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // Consume as much as possible from the repeated run.
+ // We only need to count the number of nulls and non-nulls because we can fill in the
+ // same value for nulls and non-nulls.
+ // This proves to be a big efficiency win.
+ while (values_remaining_run() > 0 && !batch.IsDone()) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const& validity_run_size = static_cast<values_count_type>(validity_run.length);
+
+ if (validity_run.set) {
+ // We may end the current RLE run in the middle of the validity run
+ auto update_size = std::min(validity_run_size, values_remaining_run());
+ batch.AccrueReadValues(update_size);
+ validity_run.length -= update_size;
+ } else {
+ // We can consume all nulls here because it does not matter if we consume on this
+ // RLE run, or an a next encoded run. The value filled does not matter.
+ auto update_size = std::min(validity_run_size, batch.NullRemaining());
+ batch.AccrueReadNulls(update_size);
+ validity_run.length -= update_size;
+ }
+
+ if (ARROW_PREDICT_TRUE(validity_run.length == 0)) {
+ validity_run = validity_reader.NextRun();
}
+ }
+
+ value_type const value = decoder.Value();
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(value))) {
+ return {0, 0};
+ }
+ converter.WriteRepeated(out, out + batch.TotalRead(), value);
+ auto const actual_values_read = decoder.Advance(batch.ValuesRead());
+ // We always cropped the number of values_read by the remaining values in the run.
+ // What's more the RLE decoder should not encounter any errors.
+ ARROW_DCHECK_EQ(actual_values_read, batch.ValuesRead());
- ARROW_DCHECK_GT(batch_size, 0);
- ARROW_DCHECK_GT(valid_run.length, 0);
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+template <typename T, typename... Ts>
+[[nodiscard]] constexpr T min(T x, Ts... ys) {
+ ((x = std::min(x, ys)), ...);
+ return x;
+}
+
+static_assert(min(5) == 5);
+static_assert(min(5, 4, -1) == -1);
+static_assert(min(5, 41) == 5);
+
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ BitPackedDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto run_values_remaining = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ while (run_values_remaining() > 0 && batch.ValuesRemaining() > 0) {
+ // TODO should this size be tune depending on sizeof(value_size)? cpu cache size?
Review Comment:
Agreed with this TODO, and 1024 seems a bit high anyway...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2319512345
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ ARROW_DCHECK_LT(values_read, batch_size);
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+ auto const read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
return values_read;
}
-template <typename T, typename RunType, typename Converter>
-inline int RleDecoder::GetSpaced(Converter converter, int batch_size, int null_count,
- const uint8_t* valid_bits, int64_t valid_bits_offset,
- T* out) {
- if (ARROW_PREDICT_FALSE(null_count == batch_size)) {
- converter.FillZero(out, out + batch_size);
- return batch_size;
+namespace internal {
+
+/// Utility class to safely handle values and null count without too error-prone
+/// verbosity.
+class BatchCounter {
+ public:
+ using size_type = int32_t;
+
+ [[nodiscard]] static constexpr BatchCounter FromBatchSizeAndNulls(
+ size_type batch_size, size_type null_count) {
+ ARROW_DCHECK_LE(null_count, batch_size);
+ return {batch_size - null_count, null_count};
}
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
- int values_remaining = batch_size - null_count;
+ constexpr BatchCounter(size_type values_count, size_type null_count) noexcept
+ : values_count_(values_count), null_count_(null_count) {}
- // Assume no bits to start.
- arrow::internal::BitRunReader bit_reader(valid_bits, valid_bits_offset,
- /*length=*/batch_size);
- arrow::internal::BitRun valid_run = bit_reader.NextRun();
- while (values_read < batch_size) {
- if (ARROW_PREDICT_FALSE(valid_run.length == 0)) {
- valid_run = bit_reader.NextRun();
+ [[nodiscard]] constexpr size_type ValuesCount() const noexcept { return values_count_; }
+
+ [[nodiscard]] constexpr size_type ValuesRead() const noexcept { return values_read_; }
+
+ [[nodiscard]] constexpr size_type ValuesRemaining() const noexcept {
+ ARROW_DCHECK_LE(values_read_, values_count_);
+ return values_count_ - values_read_;
+ }
+
+ constexpr void AccrueReadValues(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, ValuesRemaining());
+ values_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type NullCount() const noexcept { return null_count_; }
+
+ [[nodiscard]] constexpr size_type NullRead() const noexcept { return null_read_; }
+
+ [[nodiscard]] constexpr size_type NullRemaining() const noexcept {
+ ARROW_DCHECK_LE(null_read_, null_count_);
+ return null_count_ - null_read_;
+ }
+
+ constexpr void AccrueReadNulls(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, NullRemaining());
+ null_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type TotalRemaining() const noexcept {
+ return ValuesRemaining() + NullRemaining();
+ }
+
+ [[nodiscard]] constexpr size_type TotalRead() const noexcept {
+ return values_read_ + null_read_;
+ }
+
+ [[nodiscard]] constexpr bool IsFullyNull() const noexcept {
+ return ValuesRemaining() == 0;
+ }
+
+ [[nodiscard]] constexpr bool IsDone() const noexcept { return TotalRemaining() == 0; }
+
+ private:
+ size_type values_count_ = 0;
+ size_type values_read_ = 0;
+ size_type null_count_ = 0;
+ size_type null_read_ = 0;
+};
+
+// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+/// Overload for GetSpaced for a single run in a RleDecoder
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ RleDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto values_remaining_run = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // Consume as much as possible from the repeated run.
+ // We only need to count the number of nulls and non-nulls because we can fill in the
+ // same value for nulls and non-nulls.
+ // This proves to be a big efficiency win.
+ while (values_remaining_run() > 0 && !batch.IsDone()) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const& validity_run_size = static_cast<values_count_type>(validity_run.length);
+
+ if (validity_run.set) {
+ // We may end the current RLE run in the middle of the validity run
+ auto update_size = std::min(validity_run_size, values_remaining_run());
+ batch.AccrueReadValues(update_size);
+ validity_run.length -= update_size;
+ } else {
+ // We can consume all nulls here because it does not matter if we consume on this
+ // RLE run, or an a next encoded run. The value filled does not matter.
+ auto update_size = std::min(validity_run_size, batch.NullRemaining());
+ batch.AccrueReadNulls(update_size);
+ validity_run.length -= update_size;
+ }
+
+ if (ARROW_PREDICT_TRUE(validity_run.length == 0)) {
+ validity_run = validity_reader.NextRun();
}
+ }
+
+ value_type const value = decoder.Value();
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(value))) {
+ return {0, 0};
+ }
+ converter.WriteRepeated(out, out + batch.TotalRead(), value);
+ auto const actual_values_read = decoder.Advance(batch.ValuesRead());
+ // We always cropped the number of values_read by the remaining values in the run.
+ // What's more the RLE decoder should not encounter any errors.
+ ARROW_DCHECK_EQ(actual_values_read, batch.ValuesRead());
- ARROW_DCHECK_GT(batch_size, 0);
- ARROW_DCHECK_GT(valid_run.length, 0);
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+template <typename T, typename... Ts>
+[[nodiscard]] constexpr T min(T x, Ts... ys) {
+ ((x = std::min(x, ys)), ...);
+ return x;
+}
+
+static_assert(min(5) == 5);
+static_assert(min(5, 4, -1) == -1);
+static_assert(min(5, 41) == 5);
Review Comment:
If we really want this utility, it should probably be factored out in one of the `arrow/util` headers?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2363120215
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +612,663 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/************************
+ * RleBitPackedParser *
+ ************************/
+
+template <typename Handler>
+void RleBitPackedParser::Parse(Handler&& handler) {
+ while (!exhausted()) {
+ auto [read, control] = PeekImpl(handler);
+ data_ += read;
+ data_size_ -= read;
+ if (ARROW_PREDICT_FALSE(control == ControlFlow::Break)) {
+ break;
+ }
+ }
+}
+
+namespace internal {
+/// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+} // namespace internal
+
+template <typename Handler>
+auto RleBitPackedParser::PeekImpl(Handler&& handler) const
+ -> std::pair<rle_size_t, ControlFlow> {
+ ARROW_DCHECK(!exhausted());
+
+ constexpr auto kMaxSize = bit_util::kMaxLEB128ByteLenFor<uint32_t>;
+ uint32_t run_len_type = 0;
+ const auto header_bytes = bit_util::ParseLeadingLEB128(data_, kMaxSize, &run_len_type);
+
+ if (ARROW_PREDICT_FALSE(header_bytes == 0)) {
+ // Malfomrmed LEB128 data
+ return {};
+ }
+
+ const bool is_bit_packed = run_len_type & 1;
+ const uint32_t count = run_len_type >> 1;
+ if (is_bit_packed) {
+ constexpr auto kMaxCount = bit_util::CeilDiv(internal::max_size_for_v<rle_size_t>, 8);
+ if (ARROW_PREDICT_FALSE(count == 0 || count > kMaxCount)) {
+ // Illegal number of encoded values
+ return {0, ControlFlow::Break};
+ }
+
+ const auto values_count = static_cast<rle_size_t>(count * 8);
+ ARROW_DCHECK_LT(count, internal::max_size_for_v<rle_size_t>);
Review Comment:
I changed for a check useful before the `static_cast`.
```
ARROW_DCHECK_LT(static_cast<uint64_t>(count) * 8, internal::max_size_for_v<rle_size_t>);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2349571760
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
Review Comment:
Benchmarks seem to prefer these predictions (and the other ones with similar comment).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2349595596
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ ARROW_DCHECK_LT(values_read, batch_size);
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+ auto const read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
return values_read;
}
-template <typename T, typename RunType, typename Converter>
-inline int RleDecoder::GetSpaced(Converter converter, int batch_size, int null_count,
- const uint8_t* valid_bits, int64_t valid_bits_offset,
- T* out) {
- if (ARROW_PREDICT_FALSE(null_count == batch_size)) {
- converter.FillZero(out, out + batch_size);
- return batch_size;
+namespace internal {
+
+/// Utility class to safely handle values and null count without too error-prone
+/// verbosity.
+class BatchCounter {
+ public:
+ using size_type = int32_t;
+
+ [[nodiscard]] static constexpr BatchCounter FromBatchSizeAndNulls(
+ size_type batch_size, size_type null_count) {
+ ARROW_DCHECK_LE(null_count, batch_size);
+ return {batch_size - null_count, null_count};
}
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
- int values_remaining = batch_size - null_count;
+ constexpr BatchCounter(size_type values_count, size_type null_count) noexcept
+ : values_count_(values_count), null_count_(null_count) {}
- // Assume no bits to start.
- arrow::internal::BitRunReader bit_reader(valid_bits, valid_bits_offset,
- /*length=*/batch_size);
- arrow::internal::BitRun valid_run = bit_reader.NextRun();
- while (values_read < batch_size) {
- if (ARROW_PREDICT_FALSE(valid_run.length == 0)) {
- valid_run = bit_reader.NextRun();
+ [[nodiscard]] constexpr size_type ValuesCount() const noexcept { return values_count_; }
+
+ [[nodiscard]] constexpr size_type ValuesRead() const noexcept { return values_read_; }
+
+ [[nodiscard]] constexpr size_type ValuesRemaining() const noexcept {
+ ARROW_DCHECK_LE(values_read_, values_count_);
+ return values_count_ - values_read_;
+ }
+
+ constexpr void AccrueReadValues(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, ValuesRemaining());
+ values_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type NullCount() const noexcept { return null_count_; }
+
+ [[nodiscard]] constexpr size_type NullRead() const noexcept { return null_read_; }
+
+ [[nodiscard]] constexpr size_type NullRemaining() const noexcept {
+ ARROW_DCHECK_LE(null_read_, null_count_);
+ return null_count_ - null_read_;
+ }
+
+ constexpr void AccrueReadNulls(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, NullRemaining());
+ null_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type TotalRemaining() const noexcept {
+ return ValuesRemaining() + NullRemaining();
+ }
+
+ [[nodiscard]] constexpr size_type TotalRead() const noexcept {
+ return values_read_ + null_read_;
+ }
+
+ [[nodiscard]] constexpr bool IsFullyNull() const noexcept {
+ return ValuesRemaining() == 0;
+ }
+
+ [[nodiscard]] constexpr bool IsDone() const noexcept { return TotalRemaining() == 0; }
+
+ private:
+ size_type values_count_ = 0;
+ size_type values_read_ = 0;
+ size_type null_count_ = 0;
+ size_type null_read_ = 0;
+};
+
+// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+/// Overload for GetSpaced for a single run in a RleDecoder
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ RleDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto values_remaining_run = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // Consume as much as possible from the repeated run.
+ // We only need to count the number of nulls and non-nulls because we can fill in the
+ // same value for nulls and non-nulls.
+ // This proves to be a big efficiency win.
+ while (values_remaining_run() > 0 && !batch.IsDone()) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const& validity_run_size = static_cast<values_count_type>(validity_run.length);
+
+ if (validity_run.set) {
+ // We may end the current RLE run in the middle of the validity run
+ auto update_size = std::min(validity_run_size, values_remaining_run());
+ batch.AccrueReadValues(update_size);
+ validity_run.length -= update_size;
+ } else {
+ // We can consume all nulls here because it does not matter if we consume on this
+ // RLE run, or an a next encoded run. The value filled does not matter.
+ auto update_size = std::min(validity_run_size, batch.NullRemaining());
+ batch.AccrueReadNulls(update_size);
+ validity_run.length -= update_size;
+ }
+
+ if (ARROW_PREDICT_TRUE(validity_run.length == 0)) {
+ validity_run = validity_reader.NextRun();
}
+ }
+
+ value_type const value = decoder.Value();
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(value))) {
+ return {0, 0};
+ }
+ converter.WriteRepeated(out, out + batch.TotalRead(), value);
+ auto const actual_values_read = decoder.Advance(batch.ValuesRead());
+ // We always cropped the number of values_read by the remaining values in the run.
+ // What's more the RLE decoder should not encounter any errors.
+ ARROW_DCHECK_EQ(actual_values_read, batch.ValuesRead());
- ARROW_DCHECK_GT(batch_size, 0);
- ARROW_DCHECK_GT(valid_run.length, 0);
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+template <typename T, typename... Ts>
+[[nodiscard]] constexpr T min(T x, Ts... ys) {
+ ((x = std::min(x, ys)), ...);
+ return x;
+}
+
+static_assert(min(5) == 5);
+static_assert(min(5, 4, -1) == -1);
+static_assert(min(5, 41) == 5);
+
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ BitPackedDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto run_values_remaining = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ while (run_values_remaining() > 0 && batch.ValuesRemaining() > 0) {
+ // TODO should this size be tune depending on sizeof(value_size)? cpu cache size?
+ // Pull a batch of values from the bit packed encoded data and store it in a local
+ // buffer to benefit from unpacking intrinsics and data locality.
+ static constexpr values_count_type kBufferCapacity = 1024;
+ std::array<value_type, kBufferCapacity> buffer = {};
+
+ values_count_type buffer_start = 0;
+ values_count_type buffer_end = 0;
+ auto buffer_size = [&]() {
+ auto out = buffer_end - buffer_start;
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // buffer_start is 0 at this point so size is end
+ buffer_end = min(run_values_remaining(), batch.ValuesRemaining(), kBufferCapacity);
+ buffer_end = decoder.GetBatch(buffer.data(), buffer_size());
+ ARROW_DCHECK_LE(buffer_size(), kBufferCapacity);
+
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(buffer.data(), buffer_size()))) {
+ return {batch.ValuesRead(), batch.NullRead()};
Review Comment:
`RleRunDecoder` is simple enough that all errors fall in that category.
Previous implementation returned bytes read if they encountered an error (e.g. [here](https://github.com/apache/arrow/blob/eb6b637cc8857419ced5379dd5693c57dafa2458/cpp/src/arrow/util/rle_encoding_internal.h#L399) and [here](https://github.com/apache/arrow/blob/eb6b637cc8857419ced5379dd5693c57dafa2458/cpp/src/arrow/util/rle_encoding_internal.h#L414)). I assume because these bytes were indeed written to the output buffer (as it is the case here).
I agree that zero or even negative return type would be even more descriptive. Should we make the change in this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2355077953
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -602,20 +1038,126 @@ struct GetBatchSpacedTestCase {
int bit_width;
};
-TEST(RleDecoder, GetBatchSpaced) {
- uint32_t kSeed = 1337;
- ::arrow::random::RandomArrayGenerator rand(kSeed);
-
- std::vector<GetBatchSpacedTestCase<int32_t>> int32_cases{
- {1, 100000, 0.01, 1}, {1, 100000, 0.1, 1}, {1, 100000, 0.5, 1},
- {4, 100000, 0.05, 3}, {100, 100000, 0.05, 7},
+template <typename T>
+void DoTestGetBatchSpacedRoundtrip() {
+ using Data = DataTestRleBitPacked<T>;
+ using ArrowType = typename Data::ArrowType;
+ using RandomPart = typename Data::RandomPart;
+ using NullPart = typename Data::NullPart;
+ using RepeatPart = typename Data::RepeatPart;
+
+ std::vector<Data> test_cases = {
+ {
+ {RandomPart{/* max=*/1, /* size=*/400, /* null_proba= */ 0.1}},
+ /* bit_width= */ 1,
+ },
+ {
+ {
+ RandomPart{/* max=*/7, /* size=*/10037, /* null_proba= */ 0.0},
+ NullPart{/* size= */ 1153},
+ RandomPart{/* max=*/7, /* size=*/800, /* null_proba= */ 0.5},
+ },
+ /* bit_width= */ 3,
+ },
+ {
+ {
+ NullPart{/* size= */ 80},
+ RandomPart{/* max=*/static_cast<T>(1023), /* size=*/800,
+ /* null_proba= */ 0.01},
+ NullPart{/* size= */ 1023},
+ },
+ /* bit_width= */ 11,
+ },
+ {
+ {RepeatPart{/* value=*/13, /* size=*/100000, /* null_proba= */ 0.01}},
+ /* bit_width= */ 10,
+ },
+ {
+ {
+ NullPart{/* size= */ 1024},
+ RepeatPart{/* value=*/static_cast<T>(10000), /* size=*/100000,
+ /* null_proba= */ 0.1},
+ NullPart{/* size= */ 77},
+ },
+ /* bit_width= */ 23,
+ },
+ {
+ {
+ RepeatPart{/* value=*/13, /* size=*/100000, /* null_proba= */ 0.0},
+ NullPart{/* size= */ 1153},
+ RepeatPart{/* value=*/72, /* size=*/100799, /* null_proba= */ 0.5},
+ },
+ /* bit_width= */ 10,
+ },
+ {
+ {
+ RandomPart{/* max=*/1, /* size=*/1013, /* null_proba= */ 0.01},
+ NullPart{/* size=*/8},
+ RepeatPart{1, /* size= */ 256, /* null_proba= */ 0.1},
+ NullPart{/* size=*/128},
+ RepeatPart{0, /* size= */ 256, /* null_proba= */ 0.0},
+ NullPart{/* size=*/15},
+ RandomPart{/* max=*/1, /* size=*/8 * 1024, /* null_proba= */ 0.01},
+ },
+ /* bit_width= */ 1,
+ },
};
- for (auto case_ : int32_cases) {
- auto arr = rand.Int32(case_.size, /*min=*/0, case_.max_value, case_.null_probability);
- CheckRoundTripSpaced<Int32Type>(*arr, case_.bit_width);
- CheckRoundTripSpaced<Int32Type>(*arr->Slice(1), case_.bit_width);
+
+ ::arrow::random::RandomArrayGenerator rand(/* seed= */ 12);
+ // FRAGILE: we create a dictionary large enough so that any encoded value from the
+ // previous test cases can be used as an index in the dictionary.
+ // Its size must be increased accordingly if larger values are encoded in the test
+ // cases.
+ auto dict = std::static_pointer_cast<arrow::FloatArray>(rand.Float32(20000, -1.0, 1.0));
+
+ // Number of bits available in T to write a positive integer.
+ constexpr int kBitsAvailable = 8 * sizeof(T) - (std::is_signed_v<T> ? 1 : 0);
Review Comment:
Not sure what you mean here. Here we test multiple integer type, and this is used to filter test cases that will not fit in the given integer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #47294:
URL: https://github.com/apache/arrow/pull/47294#issuecomment-3323472227
@ursabot please benchmark
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2322162945
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -553,47 +853,178 @@ TEST(BitRle, Overflow) {
}
}
-template <typename Type>
-void CheckRoundTripSpaced(const Array& data, int bit_width) {
+/// Check RleBitPacked encoding/decoding round trip.
+///
+/// \tparam kSpaced If set to false, treat Nulls in the input array as regular data.
+/// \tparam kParts The number of parts in which the data will be decoded.
+/// For number greater than one, this ensure that the decoder intermediary state
+/// is valid.
+template <typename Type, bool kSpaced, int32_t kParts>
+void CheckRoundTrip(const Array& data, int bit_width,
+ std::shared_ptr<FloatArray> dict = {}) {
using ArrayType = typename TypeTraits<Type>::ArrayType;
- using T = typename Type::c_type;
+ using value_type = typename Type::c_type;
- int num_values = static_cast<int>(data.length());
- int buffer_size = RleEncoder::MaxBufferSize(bit_width, num_values);
+ int const data_size = static_cast<int>(data.length());
+ int const data_values_count =
+ static_cast<int>(data.length() - kSpaced * data.null_count());
+ int const buffer_size = RleBitPackedEncoder::MaxBufferSize(bit_width, data_size);
+ ASSERT_GE(kParts, 1);
+ ASSERT_LE(kParts, data_size);
- const T* values = static_cast<const ArrayType&>(data).raw_values();
+ const value_type* data_values = static_cast<const ArrayType&>(data).raw_values();
+ // Encode the data into ``buffer`` using the encoder.
std::vector<uint8_t> buffer(buffer_size);
- RleEncoder encoder(buffer.data(), buffer_size, bit_width);
- for (int i = 0; i < num_values; ++i) {
- if (data.IsValid(i)) {
- if (!encoder.Put(static_cast<uint64_t>(values[i]))) {
- FAIL() << "Encoding failed";
- }
+ RleBitPackedEncoder encoder(buffer.data(), buffer_size, bit_width);
+ int32_t encoded_values_size = 0;
+ for (int i = 0; i < data_size; ++i) {
+ // Depending on kSpaced we treat nulls as regular values.
+ if (data.IsValid(i) || !kSpaced) {
+ bool success = encoder.Put(static_cast<uint64_t>(data_values[i]));
+ ASSERT_TRUE(success) << "Encoding failed in pos " << i;
+ ++encoded_values_size;
}
}
- int encoded_size = encoder.Flush();
+ int encoded_byte_size = encoder.Flush();
+ ASSERT_EQ(encoded_values_size, data_values_count)
+ << "All values input were not encoded successfully by the encoder";
+
+ // On to verify batch read
+ RleBitPackedDecoder<value_type> decoder(buffer.data(), encoded_byte_size, bit_width);
+ // We will only use one of them depending on whether this is a dictonnary tests
+ std::vector<float> dict_read;
+ std::vector<value_type> values_read;
+ if (dict) {
+ dict_read.resize(data_size);
+ } else {
+ values_read.resize(data_size);
+ }
- // Verify batch read
- RleDecoder decoder(buffer.data(), encoded_size, bit_width);
- std::vector<T> values_read(num_values);
+ // We will read the data in kParts calls to make sure intermediate states are valid
+ int32_t actual_read_count = 0;
+ int32_t requested_read_count = 0;
+ while (requested_read_count < data_size) {
+ auto const remaining = data_size - requested_read_count;
+ auto to_read = data_size / kParts;
+ if (remaining / to_read == 1) {
+ to_read = remaining;
+ }
- if (num_values != decoder.GetBatchSpaced(
- num_values, static_cast<int>(data.null_count()),
- data.null_bitmap_data(), data.offset(), values_read.data())) {
- FAIL();
- }
+ auto read = 0;
+ if constexpr (kSpaced) {
+ // We need to slice the input array get the proper null count and bitmap
+ auto data_remaining = data.Slice(requested_read_count, to_read);
+
+ if (dict) {
+ auto* out = dict_read.data() + requested_read_count;
+ read = decoder.GetBatchWithDictSpaced(
+ dict->raw_values(), static_cast<int32_t>(dict->length()), out, to_read,
+ static_cast<int32_t>(data_remaining->null_count()),
+ data_remaining->null_bitmap_data(), data_remaining->offset());
+ } else {
+ auto* out = values_read.data() + requested_read_count;
+ read = decoder.GetBatchSpaced(
+ to_read, static_cast<int32_t>(data_remaining->null_count()),
+ data_remaining->null_bitmap_data(), data_remaining->offset(), out);
+ }
+ } else {
+ if (dict) {
+ auto* out = dict_read.data() + requested_read_count;
+ read = decoder.GetBatchWithDict(
+ dict->raw_values(), static_cast<int32_t>(dict->length()), out, to_read);
+ } else {
+ auto* out = values_read.data() + requested_read_count;
+ read = decoder.GetBatch(out, to_read);
+ }
+ }
+ ASSERT_EQ(read, to_read) << "Decoder did not read as many values as requested";
- for (int64_t i = 0; i < num_values; ++i) {
- if (data.IsValid(i)) {
- if (values_read[i] != values[i]) {
- FAIL() << "Index " << i << " read " << values_read[i] << " but should be "
- << values[i];
+ actual_read_count += read;
+ requested_read_count += to_read;
+ }
+ EXPECT_EQ(requested_read_count, data_size) << "This test logic is wrong";
+ EXPECT_EQ(actual_read_count, data_size) << "Total number of values read is off";
+
+ // Verify the round trip: encoded-decoded values must equal the original one
+ for (int64_t i = 0; i < data_size; ++i) {
+ if (data.IsValid(i) || !kSpaced) {
+ if (dict) {
+ EXPECT_EQ(dict_read.at(i), dict->Value(data_values[i]))
+ << "Encoded then decoded and mapped value at position " << i << " ("
+ << values_read[i] << ") differs from original value (" << data_values[i]
+ << " mapped to " << dict->Value(data_values[i]) << ")";
+ } else {
+ EXPECT_EQ(values_read.at(i), data_values[i])
+ << "Encoded then decoded value at position " << i << " (" << values_read.at(i)
+ << ") differs from original value (" << data_values[i] << ")";
}
}
}
}
+template <typename T>
+struct DataTestRleBitPackedRandomPart {
+ using value_type = T;
+
+ value_type max;
+ int32_t size;
+ double null_probability;
+};
+
+template <typename T>
+struct DataTestRleBitPackedRepeatPart {
+ using value_type = T;
+
+ value_type value;
+ int32_t size;
+};
+
+template <typename T>
+struct DataTestRleBitPackedNullPart {
+ using value_type = T;
+
+ int32_t size;
+};
+
+template <typename T>
+struct DataTestRleBitPacked {
+ using value_type = T;
+ using ArrowType = typename arrow::CTypeTraits<value_type>::ArrowType;
+ using RandomPart = DataTestRleBitPackedRandomPart<value_type>;
+ using RepeatPart = DataTestRleBitPackedRepeatPart<value_type>;
+ using NullPart = DataTestRleBitPackedNullPart<value_type>;
+
+ std::vector<std::variant<RandomPart, RepeatPart, NullPart>> parts;
+ int32_t bit_width;
+
+ std::shared_ptr<::arrow::Array> MakeArray(
+ ::arrow::random::RandomArrayGenerator& rand) const {
+ std::vector<std::shared_ptr<::arrow::Array>> arrays = {};
+
+ for (auto const& dyn_part : parts) {
+ if (auto* part = std::get_if<RandomPart>(&dyn_part)) {
+ auto arr = rand.Numeric<ArrowType>(part->size, /* min= */ value_type(0),
+ part->max, part->null_probability);
+ arrays.push_back(std::move(arr));
+
+ } else if (auto* part = std::get_if<RepeatPart>(&dyn_part)) {
+ auto scalar = ::arrow::MakeScalar(part->value);
+ arrays.push_back(::arrow::MakeArrayFromScalar(*scalar, part->size).ValueOrDie());
Review Comment:
Note you may use `EXPECT_OK_AND_ASSIGN` instead of `ValueOrDie` (but it's probably not necessary here).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #47294:
URL: https://github.com/apache/arrow/pull/47294#issuecomment-3245526211
I ran the benchmarks locally with an AMD Zen 2 CPU on Ubuntu 24.04:
https://gist.github.com/pitrou/9539520b5f7850b9cb6ffd25341fdeb0
There are a couple significant speedups and a couple significant regressions, but the overall picture is rather reassuring: this refactoring is globally neutral performance-wise.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2319536697
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ ARROW_DCHECK_LT(values_read, batch_size);
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+ auto const read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
return values_read;
}
-template <typename T, typename RunType, typename Converter>
-inline int RleDecoder::GetSpaced(Converter converter, int batch_size, int null_count,
- const uint8_t* valid_bits, int64_t valid_bits_offset,
- T* out) {
- if (ARROW_PREDICT_FALSE(null_count == batch_size)) {
- converter.FillZero(out, out + batch_size);
- return batch_size;
+namespace internal {
+
+/// Utility class to safely handle values and null count without too error-prone
+/// verbosity.
+class BatchCounter {
+ public:
+ using size_type = int32_t;
+
+ [[nodiscard]] static constexpr BatchCounter FromBatchSizeAndNulls(
+ size_type batch_size, size_type null_count) {
+ ARROW_DCHECK_LE(null_count, batch_size);
+ return {batch_size - null_count, null_count};
}
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
- int values_remaining = batch_size - null_count;
+ constexpr BatchCounter(size_type values_count, size_type null_count) noexcept
+ : values_count_(values_count), null_count_(null_count) {}
- // Assume no bits to start.
- arrow::internal::BitRunReader bit_reader(valid_bits, valid_bits_offset,
- /*length=*/batch_size);
- arrow::internal::BitRun valid_run = bit_reader.NextRun();
- while (values_read < batch_size) {
- if (ARROW_PREDICT_FALSE(valid_run.length == 0)) {
- valid_run = bit_reader.NextRun();
+ [[nodiscard]] constexpr size_type ValuesCount() const noexcept { return values_count_; }
+
+ [[nodiscard]] constexpr size_type ValuesRead() const noexcept { return values_read_; }
+
+ [[nodiscard]] constexpr size_type ValuesRemaining() const noexcept {
+ ARROW_DCHECK_LE(values_read_, values_count_);
+ return values_count_ - values_read_;
+ }
+
+ constexpr void AccrueReadValues(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, ValuesRemaining());
+ values_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type NullCount() const noexcept { return null_count_; }
+
+ [[nodiscard]] constexpr size_type NullRead() const noexcept { return null_read_; }
+
+ [[nodiscard]] constexpr size_type NullRemaining() const noexcept {
+ ARROW_DCHECK_LE(null_read_, null_count_);
+ return null_count_ - null_read_;
+ }
+
+ constexpr void AccrueReadNulls(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, NullRemaining());
+ null_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type TotalRemaining() const noexcept {
+ return ValuesRemaining() + NullRemaining();
+ }
+
+ [[nodiscard]] constexpr size_type TotalRead() const noexcept {
+ return values_read_ + null_read_;
+ }
+
+ [[nodiscard]] constexpr bool IsFullyNull() const noexcept {
+ return ValuesRemaining() == 0;
+ }
+
+ [[nodiscard]] constexpr bool IsDone() const noexcept { return TotalRemaining() == 0; }
+
+ private:
+ size_type values_count_ = 0;
+ size_type values_read_ = 0;
+ size_type null_count_ = 0;
+ size_type null_read_ = 0;
+};
+
+// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+/// Overload for GetSpaced for a single run in a RleDecoder
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ RleDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto values_remaining_run = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // Consume as much as possible from the repeated run.
+ // We only need to count the number of nulls and non-nulls because we can fill in the
+ // same value for nulls and non-nulls.
+ // This proves to be a big efficiency win.
+ while (values_remaining_run() > 0 && !batch.IsDone()) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const& validity_run_size = static_cast<values_count_type>(validity_run.length);
+
+ if (validity_run.set) {
+ // We may end the current RLE run in the middle of the validity run
+ auto update_size = std::min(validity_run_size, values_remaining_run());
+ batch.AccrueReadValues(update_size);
+ validity_run.length -= update_size;
+ } else {
+ // We can consume all nulls here because it does not matter if we consume on this
+ // RLE run, or an a next encoded run. The value filled does not matter.
+ auto update_size = std::min(validity_run_size, batch.NullRemaining());
+ batch.AccrueReadNulls(update_size);
+ validity_run.length -= update_size;
+ }
+
+ if (ARROW_PREDICT_TRUE(validity_run.length == 0)) {
+ validity_run = validity_reader.NextRun();
}
+ }
+
+ value_type const value = decoder.Value();
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(value))) {
+ return {0, 0};
+ }
+ converter.WriteRepeated(out, out + batch.TotalRead(), value);
+ auto const actual_values_read = decoder.Advance(batch.ValuesRead());
+ // We always cropped the number of values_read by the remaining values in the run.
+ // What's more the RLE decoder should not encounter any errors.
+ ARROW_DCHECK_EQ(actual_values_read, batch.ValuesRead());
- ARROW_DCHECK_GT(batch_size, 0);
- ARROW_DCHECK_GT(valid_run.length, 0);
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+template <typename T, typename... Ts>
+[[nodiscard]] constexpr T min(T x, Ts... ys) {
+ ((x = std::min(x, ys)), ...);
+ return x;
+}
+
+static_assert(min(5) == 5);
+static_assert(min(5, 4, -1) == -1);
+static_assert(min(5, 41) == 5);
+
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ BitPackedDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto run_values_remaining = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ while (run_values_remaining() > 0 && batch.ValuesRemaining() > 0) {
+ // TODO should this size be tune depending on sizeof(value_size)? cpu cache size?
+ // Pull a batch of values from the bit packed encoded data and store it in a local
+ // buffer to benefit from unpacking intrinsics and data locality.
+ static constexpr values_count_type kBufferCapacity = 1024;
+ std::array<value_type, kBufferCapacity> buffer = {};
+
+ values_count_type buffer_start = 0;
+ values_count_type buffer_end = 0;
+ auto buffer_size = [&]() {
+ auto out = buffer_end - buffer_start;
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // buffer_start is 0 at this point so size is end
+ buffer_end = min(run_values_remaining(), batch.ValuesRemaining(), kBufferCapacity);
+ buffer_end = decoder.GetBatch(buffer.data(), buffer_size());
+ ARROW_DCHECK_LE(buffer_size(), kBufferCapacity);
+
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(buffer.data(), buffer_size()))) {
+ return {batch.ValuesRead(), batch.NullRead()};
+ }
- if (valid_run.set) {
- if ((repeat_count_ == 0) && (literal_count_ == 0)) {
- if (!NextCounts<RunType>()) return values_read;
- ARROW_DCHECK((repeat_count_ > 0) ^ (literal_count_ > 0));
+ // Copy chunks of valid values into the output, while adjusting spacing for null
+ // values.
+ while (buffer_size() > 0) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const validity_run_length =
+ static_cast<values_count_type>(validity_run.length);
+
+ // Copy as much as possible from the buffer into the output while not exceeding
+ // validity run
+ if (validity_run.set) {
+ auto const update_size = std::min(validity_run_length, buffer_size());
+ converter.WriteRange(out, buffer.data() + buffer_start, update_size);
+ buffer_start += update_size;
+ batch.AccrueReadValues(update_size);
+ out += update_size;
+ validity_run.length -= update_size;
+ // Simply write zeros in the output
+ } else {
+ auto const update_size = std::min(validity_run_length, batch.NullRemaining());
+ converter.WriteZero(out, out + update_size);
+ batch.AccrueReadNulls(update_size);
+ out += update_size;
+ validity_run.length -= update_size;
}
- if (repeat_count_ > 0) {
- int repeat_batch = 0;
- // Consume the entire repeat counts incrementing repeat_batch to
- // be the total of nulls + values consumed, we only need to
- // get the total count because we can fill in the same value for
- // nulls and non-nulls. This proves to be a big efficiency win.
- while (repeat_count_ > 0 && (values_read + repeat_batch) < batch_size) {
- ARROW_DCHECK_GT(valid_run.length, 0);
- if (valid_run.set) {
- int update_size = std::min(static_cast<int>(valid_run.length), repeat_count_);
- repeat_count_ -= update_size;
- repeat_batch += update_size;
- valid_run.length -= update_size;
- values_remaining -= update_size;
- } else {
- // We can consume all nulls here because we would do so on
- // the next loop anyways.
- repeat_batch += static_cast<int>(valid_run.length);
- valid_run.length = 0;
- }
- if (valid_run.length == 0) {
- valid_run = bit_reader.NextRun();
- }
- }
- RunType current_value = static_cast<RunType>(current_value_);
- if (ARROW_PREDICT_FALSE(!converter.IsValid(current_value))) {
- return values_read;
- }
- converter.Fill(out, out + repeat_batch, current_value);
- out += repeat_batch;
- values_read += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(values_remaining, literal_count_);
- ARROW_DCHECK_GT(literal_batch, 0);
-
- // Decode the literals
- constexpr int kBufferSize = 1024;
- RunType indices[kBufferSize];
- literal_batch = std::min(literal_batch, kBufferSize);
- int actual_read = bit_reader_.GetBatch(bit_width_, indices, literal_batch);
- if (ARROW_PREDICT_FALSE(actual_read != literal_batch)) {
- return values_read;
- }
- if (!converter.IsValid(indices, /*length=*/actual_read)) {
- return values_read;
- }
- int skipped = 0;
- int literals_read = 0;
- while (literals_read < literal_batch) {
- if (valid_run.set) {
- int update_size = std::min(literal_batch - literals_read,
- static_cast<int>(valid_run.length));
- converter.Copy(out, indices + literals_read, update_size);
- literals_read += update_size;
- out += update_size;
- valid_run.length -= update_size;
- } else {
- converter.FillZero(out, out + valid_run.length);
- out += valid_run.length;
- skipped += static_cast<int>(valid_run.length);
- valid_run.length = 0;
- }
- if (valid_run.length == 0) {
- valid_run = bit_reader.NextRun();
- }
- }
- literal_count_ -= literal_batch;
- values_remaining -= literal_batch;
- values_read += literal_batch + skipped;
+ if (validity_run.length == 0) {
+ validity_run = validity_reader.NextRun();
}
- } else {
- converter.FillZero(out, out + valid_run.length);
- out += valid_run.length;
- values_read += static_cast<int>(valid_run.length);
- valid_run.length = 0;
}
+
+ ARROW_DCHECK_EQ(buffer_size(), 0);
}
- ARROW_DCHECK_EQ(valid_run.length, 0);
- ARROW_DCHECK_EQ(values_remaining, 0);
- return values_read;
+
+ ARROW_DCHECK_EQ(values_available - decoder.Remaining(), batch.ValuesRead());
+ ARROW_DCHECK_LE(batch.TotalRead(), batch_size);
Review Comment:
I'm not sure why that's always the case, what if there are too few values in the `decoder`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2319518770
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ ARROW_DCHECK_LT(values_read, batch_size);
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+ auto const read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
return values_read;
}
-template <typename T, typename RunType, typename Converter>
-inline int RleDecoder::GetSpaced(Converter converter, int batch_size, int null_count,
- const uint8_t* valid_bits, int64_t valid_bits_offset,
- T* out) {
- if (ARROW_PREDICT_FALSE(null_count == batch_size)) {
- converter.FillZero(out, out + batch_size);
- return batch_size;
+namespace internal {
+
+/// Utility class to safely handle values and null count without too error-prone
+/// verbosity.
+class BatchCounter {
+ public:
+ using size_type = int32_t;
+
+ [[nodiscard]] static constexpr BatchCounter FromBatchSizeAndNulls(
+ size_type batch_size, size_type null_count) {
+ ARROW_DCHECK_LE(null_count, batch_size);
+ return {batch_size - null_count, null_count};
}
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
- int values_remaining = batch_size - null_count;
+ constexpr BatchCounter(size_type values_count, size_type null_count) noexcept
+ : values_count_(values_count), null_count_(null_count) {}
- // Assume no bits to start.
- arrow::internal::BitRunReader bit_reader(valid_bits, valid_bits_offset,
- /*length=*/batch_size);
- arrow::internal::BitRun valid_run = bit_reader.NextRun();
- while (values_read < batch_size) {
- if (ARROW_PREDICT_FALSE(valid_run.length == 0)) {
- valid_run = bit_reader.NextRun();
+ [[nodiscard]] constexpr size_type ValuesCount() const noexcept { return values_count_; }
+
+ [[nodiscard]] constexpr size_type ValuesRead() const noexcept { return values_read_; }
+
+ [[nodiscard]] constexpr size_type ValuesRemaining() const noexcept {
+ ARROW_DCHECK_LE(values_read_, values_count_);
+ return values_count_ - values_read_;
+ }
+
+ constexpr void AccrueReadValues(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, ValuesRemaining());
+ values_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type NullCount() const noexcept { return null_count_; }
+
+ [[nodiscard]] constexpr size_type NullRead() const noexcept { return null_read_; }
+
+ [[nodiscard]] constexpr size_type NullRemaining() const noexcept {
+ ARROW_DCHECK_LE(null_read_, null_count_);
+ return null_count_ - null_read_;
+ }
+
+ constexpr void AccrueReadNulls(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, NullRemaining());
+ null_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type TotalRemaining() const noexcept {
+ return ValuesRemaining() + NullRemaining();
+ }
+
+ [[nodiscard]] constexpr size_type TotalRead() const noexcept {
+ return values_read_ + null_read_;
+ }
+
+ [[nodiscard]] constexpr bool IsFullyNull() const noexcept {
+ return ValuesRemaining() == 0;
+ }
+
+ [[nodiscard]] constexpr bool IsDone() const noexcept { return TotalRemaining() == 0; }
+
+ private:
+ size_type values_count_ = 0;
+ size_type values_read_ = 0;
+ size_type null_count_ = 0;
+ size_type null_read_ = 0;
+};
+
+// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+/// Overload for GetSpaced for a single run in a RleDecoder
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ RleDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto values_remaining_run = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // Consume as much as possible from the repeated run.
+ // We only need to count the number of nulls and non-nulls because we can fill in the
+ // same value for nulls and non-nulls.
+ // This proves to be a big efficiency win.
+ while (values_remaining_run() > 0 && !batch.IsDone()) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const& validity_run_size = static_cast<values_count_type>(validity_run.length);
+
+ if (validity_run.set) {
+ // We may end the current RLE run in the middle of the validity run
+ auto update_size = std::min(validity_run_size, values_remaining_run());
+ batch.AccrueReadValues(update_size);
+ validity_run.length -= update_size;
+ } else {
+ // We can consume all nulls here because it does not matter if we consume on this
+ // RLE run, or an a next encoded run. The value filled does not matter.
+ auto update_size = std::min(validity_run_size, batch.NullRemaining());
+ batch.AccrueReadNulls(update_size);
+ validity_run.length -= update_size;
+ }
+
+ if (ARROW_PREDICT_TRUE(validity_run.length == 0)) {
+ validity_run = validity_reader.NextRun();
}
+ }
+
+ value_type const value = decoder.Value();
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(value))) {
+ return {0, 0};
+ }
+ converter.WriteRepeated(out, out + batch.TotalRead(), value);
+ auto const actual_values_read = decoder.Advance(batch.ValuesRead());
+ // We always cropped the number of values_read by the remaining values in the run.
+ // What's more the RLE decoder should not encounter any errors.
+ ARROW_DCHECK_EQ(actual_values_read, batch.ValuesRead());
- ARROW_DCHECK_GT(batch_size, 0);
- ARROW_DCHECK_GT(valid_run.length, 0);
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+template <typename T, typename... Ts>
+[[nodiscard]] constexpr T min(T x, Ts... ys) {
+ ((x = std::min(x, ys)), ...);
+ return x;
+}
+
+static_assert(min(5) == 5);
+static_assert(min(5, 4, -1) == -1);
+static_assert(min(5, 41) == 5);
+
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ BitPackedDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto run_values_remaining = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ while (run_values_remaining() > 0 && batch.ValuesRemaining() > 0) {
+ // TODO should this size be tune depending on sizeof(value_size)? cpu cache size?
+ // Pull a batch of values from the bit packed encoded data and store it in a local
+ // buffer to benefit from unpacking intrinsics and data locality.
+ static constexpr values_count_type kBufferCapacity = 1024;
+ std::array<value_type, kBufferCapacity> buffer = {};
+
+ values_count_type buffer_start = 0;
+ values_count_type buffer_end = 0;
+ auto buffer_size = [&]() {
+ auto out = buffer_end - buffer_start;
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // buffer_start is 0 at this point so size is end
+ buffer_end = min(run_values_remaining(), batch.ValuesRemaining(), kBufferCapacity);
+ buffer_end = decoder.GetBatch(buffer.data(), buffer_size());
+ ARROW_DCHECK_LE(buffer_size(), kBufferCapacity);
+
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(buffer.data(), buffer_size()))) {
+ return {batch.ValuesRead(), batch.NullRead()};
Review Comment:
The same error case for `RleDecoder` returns `{0, 0}`, why do we return a partial result instead here.
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ ARROW_DCHECK_LT(values_read, batch_size);
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+ auto const read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
return values_read;
}
-template <typename T, typename RunType, typename Converter>
-inline int RleDecoder::GetSpaced(Converter converter, int batch_size, int null_count,
- const uint8_t* valid_bits, int64_t valid_bits_offset,
- T* out) {
- if (ARROW_PREDICT_FALSE(null_count == batch_size)) {
- converter.FillZero(out, out + batch_size);
- return batch_size;
+namespace internal {
+
+/// Utility class to safely handle values and null count without too error-prone
+/// verbosity.
+class BatchCounter {
+ public:
+ using size_type = int32_t;
+
+ [[nodiscard]] static constexpr BatchCounter FromBatchSizeAndNulls(
+ size_type batch_size, size_type null_count) {
+ ARROW_DCHECK_LE(null_count, batch_size);
+ return {batch_size - null_count, null_count};
}
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
- int values_remaining = batch_size - null_count;
+ constexpr BatchCounter(size_type values_count, size_type null_count) noexcept
+ : values_count_(values_count), null_count_(null_count) {}
- // Assume no bits to start.
- arrow::internal::BitRunReader bit_reader(valid_bits, valid_bits_offset,
- /*length=*/batch_size);
- arrow::internal::BitRun valid_run = bit_reader.NextRun();
- while (values_read < batch_size) {
- if (ARROW_PREDICT_FALSE(valid_run.length == 0)) {
- valid_run = bit_reader.NextRun();
+ [[nodiscard]] constexpr size_type ValuesCount() const noexcept { return values_count_; }
+
+ [[nodiscard]] constexpr size_type ValuesRead() const noexcept { return values_read_; }
+
+ [[nodiscard]] constexpr size_type ValuesRemaining() const noexcept {
+ ARROW_DCHECK_LE(values_read_, values_count_);
+ return values_count_ - values_read_;
+ }
+
+ constexpr void AccrueReadValues(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, ValuesRemaining());
+ values_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type NullCount() const noexcept { return null_count_; }
+
+ [[nodiscard]] constexpr size_type NullRead() const noexcept { return null_read_; }
+
+ [[nodiscard]] constexpr size_type NullRemaining() const noexcept {
+ ARROW_DCHECK_LE(null_read_, null_count_);
+ return null_count_ - null_read_;
+ }
+
+ constexpr void AccrueReadNulls(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, NullRemaining());
+ null_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type TotalRemaining() const noexcept {
+ return ValuesRemaining() + NullRemaining();
+ }
+
+ [[nodiscard]] constexpr size_type TotalRead() const noexcept {
+ return values_read_ + null_read_;
+ }
+
+ [[nodiscard]] constexpr bool IsFullyNull() const noexcept {
+ return ValuesRemaining() == 0;
+ }
+
+ [[nodiscard]] constexpr bool IsDone() const noexcept { return TotalRemaining() == 0; }
+
+ private:
+ size_type values_count_ = 0;
+ size_type values_read_ = 0;
+ size_type null_count_ = 0;
+ size_type null_read_ = 0;
+};
+
+// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+/// Overload for GetSpaced for a single run in a RleDecoder
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ RleDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto values_remaining_run = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // Consume as much as possible from the repeated run.
+ // We only need to count the number of nulls and non-nulls because we can fill in the
+ // same value for nulls and non-nulls.
+ // This proves to be a big efficiency win.
+ while (values_remaining_run() > 0 && !batch.IsDone()) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const& validity_run_size = static_cast<values_count_type>(validity_run.length);
+
+ if (validity_run.set) {
+ // We may end the current RLE run in the middle of the validity run
+ auto update_size = std::min(validity_run_size, values_remaining_run());
+ batch.AccrueReadValues(update_size);
+ validity_run.length -= update_size;
+ } else {
+ // We can consume all nulls here because it does not matter if we consume on this
+ // RLE run, or an a next encoded run. The value filled does not matter.
+ auto update_size = std::min(validity_run_size, batch.NullRemaining());
+ batch.AccrueReadNulls(update_size);
+ validity_run.length -= update_size;
+ }
+
+ if (ARROW_PREDICT_TRUE(validity_run.length == 0)) {
+ validity_run = validity_reader.NextRun();
}
+ }
+
+ value_type const value = decoder.Value();
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(value))) {
+ return {0, 0};
+ }
+ converter.WriteRepeated(out, out + batch.TotalRead(), value);
+ auto const actual_values_read = decoder.Advance(batch.ValuesRead());
+ // We always cropped the number of values_read by the remaining values in the run.
+ // What's more the RLE decoder should not encounter any errors.
+ ARROW_DCHECK_EQ(actual_values_read, batch.ValuesRead());
- ARROW_DCHECK_GT(batch_size, 0);
- ARROW_DCHECK_GT(valid_run.length, 0);
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+template <typename T, typename... Ts>
+[[nodiscard]] constexpr T min(T x, Ts... ys) {
+ ((x = std::min(x, ys)), ...);
+ return x;
+}
+
+static_assert(min(5) == 5);
+static_assert(min(5, 4, -1) == -1);
+static_assert(min(5, 41) == 5);
+
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ BitPackedDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto run_values_remaining = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ while (run_values_remaining() > 0 && batch.ValuesRemaining() > 0) {
+ // TODO should this size be tune depending on sizeof(value_size)? cpu cache size?
+ // Pull a batch of values from the bit packed encoded data and store it in a local
+ // buffer to benefit from unpacking intrinsics and data locality.
+ static constexpr values_count_type kBufferCapacity = 1024;
+ std::array<value_type, kBufferCapacity> buffer = {};
+
+ values_count_type buffer_start = 0;
+ values_count_type buffer_end = 0;
+ auto buffer_size = [&]() {
+ auto out = buffer_end - buffer_start;
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // buffer_start is 0 at this point so size is end
+ buffer_end = min(run_values_remaining(), batch.ValuesRemaining(), kBufferCapacity);
+ buffer_end = decoder.GetBatch(buffer.data(), buffer_size());
+ ARROW_DCHECK_LE(buffer_size(), kBufferCapacity);
+
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(buffer.data(), buffer_size()))) {
+ return {batch.ValuesRead(), batch.NullRead()};
Review Comment:
The same error case for `RleDecoder` returns `{0, 0}`, why do we return a partial result instead here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2356262537
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ ARROW_DCHECK_LT(values_read, batch_size);
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+ auto const read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
return values_read;
}
-template <typename T, typename RunType, typename Converter>
-inline int RleDecoder::GetSpaced(Converter converter, int batch_size, int null_count,
- const uint8_t* valid_bits, int64_t valid_bits_offset,
- T* out) {
- if (ARROW_PREDICT_FALSE(null_count == batch_size)) {
- converter.FillZero(out, out + batch_size);
- return batch_size;
+namespace internal {
+
+/// Utility class to safely handle values and null count without too error-prone
+/// verbosity.
+class BatchCounter {
+ public:
+ using size_type = int32_t;
+
+ [[nodiscard]] static constexpr BatchCounter FromBatchSizeAndNulls(
+ size_type batch_size, size_type null_count) {
+ ARROW_DCHECK_LE(null_count, batch_size);
+ return {batch_size - null_count, null_count};
}
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
- int values_remaining = batch_size - null_count;
+ constexpr BatchCounter(size_type values_count, size_type null_count) noexcept
+ : values_count_(values_count), null_count_(null_count) {}
- // Assume no bits to start.
- arrow::internal::BitRunReader bit_reader(valid_bits, valid_bits_offset,
- /*length=*/batch_size);
- arrow::internal::BitRun valid_run = bit_reader.NextRun();
- while (values_read < batch_size) {
- if (ARROW_PREDICT_FALSE(valid_run.length == 0)) {
- valid_run = bit_reader.NextRun();
+ [[nodiscard]] constexpr size_type ValuesCount() const noexcept { return values_count_; }
+
+ [[nodiscard]] constexpr size_type ValuesRead() const noexcept { return values_read_; }
+
+ [[nodiscard]] constexpr size_type ValuesRemaining() const noexcept {
+ ARROW_DCHECK_LE(values_read_, values_count_);
+ return values_count_ - values_read_;
+ }
+
+ constexpr void AccrueReadValues(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, ValuesRemaining());
+ values_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type NullCount() const noexcept { return null_count_; }
+
+ [[nodiscard]] constexpr size_type NullRead() const noexcept { return null_read_; }
+
+ [[nodiscard]] constexpr size_type NullRemaining() const noexcept {
+ ARROW_DCHECK_LE(null_read_, null_count_);
+ return null_count_ - null_read_;
+ }
+
+ constexpr void AccrueReadNulls(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, NullRemaining());
+ null_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type TotalRemaining() const noexcept {
+ return ValuesRemaining() + NullRemaining();
+ }
+
+ [[nodiscard]] constexpr size_type TotalRead() const noexcept {
+ return values_read_ + null_read_;
+ }
+
+ [[nodiscard]] constexpr bool IsFullyNull() const noexcept {
+ return ValuesRemaining() == 0;
+ }
+
+ [[nodiscard]] constexpr bool IsDone() const noexcept { return TotalRemaining() == 0; }
+
+ private:
+ size_type values_count_ = 0;
+ size_type values_read_ = 0;
+ size_type null_count_ = 0;
+ size_type null_read_ = 0;
+};
+
+// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+/// Overload for GetSpaced for a single run in a RleDecoder
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ RleDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto values_remaining_run = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // Consume as much as possible from the repeated run.
+ // We only need to count the number of nulls and non-nulls because we can fill in the
+ // same value for nulls and non-nulls.
+ // This proves to be a big efficiency win.
+ while (values_remaining_run() > 0 && !batch.IsDone()) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const& validity_run_size = static_cast<values_count_type>(validity_run.length);
+
+ if (validity_run.set) {
+ // We may end the current RLE run in the middle of the validity run
+ auto update_size = std::min(validity_run_size, values_remaining_run());
+ batch.AccrueReadValues(update_size);
+ validity_run.length -= update_size;
+ } else {
+ // We can consume all nulls here because it does not matter if we consume on this
+ // RLE run, or an a next encoded run. The value filled does not matter.
+ auto update_size = std::min(validity_run_size, batch.NullRemaining());
+ batch.AccrueReadNulls(update_size);
+ validity_run.length -= update_size;
+ }
+
+ if (ARROW_PREDICT_TRUE(validity_run.length == 0)) {
+ validity_run = validity_reader.NextRun();
}
+ }
+
+ value_type const value = decoder.Value();
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(value))) {
+ return {0, 0};
+ }
+ converter.WriteRepeated(out, out + batch.TotalRead(), value);
+ auto const actual_values_read = decoder.Advance(batch.ValuesRead());
+ // We always cropped the number of values_read by the remaining values in the run.
+ // What's more the RLE decoder should not encounter any errors.
+ ARROW_DCHECK_EQ(actual_values_read, batch.ValuesRead());
- ARROW_DCHECK_GT(batch_size, 0);
- ARROW_DCHECK_GT(valid_run.length, 0);
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+template <typename T, typename... Ts>
+[[nodiscard]] constexpr T min(T x, Ts... ys) {
+ ((x = std::min(x, ys)), ...);
+ return x;
+}
+
+static_assert(min(5) == 5);
+static_assert(min(5, 4, -1) == -1);
+static_assert(min(5, 41) == 5);
+
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ BitPackedDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto run_values_remaining = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ while (run_values_remaining() > 0 && batch.ValuesRemaining() > 0) {
+ // TODO should this size be tune depending on sizeof(value_size)? cpu cache size?
+ // Pull a batch of values from the bit packed encoded data and store it in a local
+ // buffer to benefit from unpacking intrinsics and data locality.
+ static constexpr values_count_type kBufferCapacity = 1024;
+ std::array<value_type, kBufferCapacity> buffer = {};
+
+ values_count_type buffer_start = 0;
+ values_count_type buffer_end = 0;
+ auto buffer_size = [&]() {
+ auto out = buffer_end - buffer_start;
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // buffer_start is 0 at this point so size is end
+ buffer_end = min(run_values_remaining(), batch.ValuesRemaining(), kBufferCapacity);
+ buffer_end = decoder.GetBatch(buffer.data(), buffer_size());
+ ARROW_DCHECK_LE(buffer_size(), kBufferCapacity);
+
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(buffer.data(), buffer_size()))) {
+ return {batch.ValuesRead(), batch.NullRead()};
Review Comment:
I think this could be done in another PR, as it also involves changing return values on the `BitReader` that is used underneath.
For now, the previous behaviour is maintained.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2337142516
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ ARROW_DCHECK_LT(values_read, batch_size);
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+ auto const read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
return values_read;
}
-template <typename T, typename RunType, typename Converter>
-inline int RleDecoder::GetSpaced(Converter converter, int batch_size, int null_count,
- const uint8_t* valid_bits, int64_t valid_bits_offset,
- T* out) {
- if (ARROW_PREDICT_FALSE(null_count == batch_size)) {
- converter.FillZero(out, out + batch_size);
- return batch_size;
+namespace internal {
+
+/// Utility class to safely handle values and null count without too error-prone
+/// verbosity.
+class BatchCounter {
+ public:
+ using size_type = int32_t;
+
+ [[nodiscard]] static constexpr BatchCounter FromBatchSizeAndNulls(
+ size_type batch_size, size_type null_count) {
+ ARROW_DCHECK_LE(null_count, batch_size);
+ return {batch_size - null_count, null_count};
}
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
- int values_remaining = batch_size - null_count;
+ constexpr BatchCounter(size_type values_count, size_type null_count) noexcept
+ : values_count_(values_count), null_count_(null_count) {}
- // Assume no bits to start.
- arrow::internal::BitRunReader bit_reader(valid_bits, valid_bits_offset,
- /*length=*/batch_size);
- arrow::internal::BitRun valid_run = bit_reader.NextRun();
- while (values_read < batch_size) {
- if (ARROW_PREDICT_FALSE(valid_run.length == 0)) {
- valid_run = bit_reader.NextRun();
+ [[nodiscard]] constexpr size_type ValuesCount() const noexcept { return values_count_; }
+
+ [[nodiscard]] constexpr size_type ValuesRead() const noexcept { return values_read_; }
+
+ [[nodiscard]] constexpr size_type ValuesRemaining() const noexcept {
+ ARROW_DCHECK_LE(values_read_, values_count_);
+ return values_count_ - values_read_;
+ }
+
+ constexpr void AccrueReadValues(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, ValuesRemaining());
+ values_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type NullCount() const noexcept { return null_count_; }
+
+ [[nodiscard]] constexpr size_type NullRead() const noexcept { return null_read_; }
+
+ [[nodiscard]] constexpr size_type NullRemaining() const noexcept {
+ ARROW_DCHECK_LE(null_read_, null_count_);
+ return null_count_ - null_read_;
+ }
+
+ constexpr void AccrueReadNulls(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, NullRemaining());
+ null_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type TotalRemaining() const noexcept {
+ return ValuesRemaining() + NullRemaining();
+ }
+
+ [[nodiscard]] constexpr size_type TotalRead() const noexcept {
+ return values_read_ + null_read_;
+ }
+
+ [[nodiscard]] constexpr bool IsFullyNull() const noexcept {
+ return ValuesRemaining() == 0;
+ }
+
+ [[nodiscard]] constexpr bool IsDone() const noexcept { return TotalRemaining() == 0; }
+
+ private:
+ size_type values_count_ = 0;
+ size_type values_read_ = 0;
+ size_type null_count_ = 0;
+ size_type null_read_ = 0;
+};
+
+// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+/// Overload for GetSpaced for a single run in a RleDecoder
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
Review Comment:
Replaced with pointers
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2337028434
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
Review Comment:
I don't have the data at hand, but yes, iteration style was slower.
My intuition is that with `parser_.Next()` and a Run variant, the run type is hidden between parsing and reading and the compiler is not always able to see through it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2367098870
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -82,86 +81,400 @@ namespace util {
/// 200 ints = 25 groups of 8
/// <varint((25 << 1) | 1)> <25 bytes of values, bitpacked>
/// (total 26 bytes, 1 byte overhead)
-//
-/// Decoder class for RLE encoded data.
-class RleDecoder {
+/// The type for an encoded Rle of BitPacked run size, between 1 and 2^31-1 as per Parquet
+/// spec.
+/// This is also pragmatically used for other integer used in the Rle and BitPacked runs
+/// and decoder to avoid conversions.
+/// It can therefore be referred to as a "typical" size for Rle and BitPacked logic.
+using rle_size_t = int32_t;
+
+template <typename T>
+class RleRunDecoder;
+
+class RleRun {
public:
- /// Create a decoder object. buffer/buffer_len is the decoded data.
- /// bit_width is the width of each value (before encoding).
- RleDecoder(const uint8_t* buffer, int buffer_len, int bit_width)
- : bit_reader_(buffer, buffer_len),
- bit_width_(bit_width),
- current_value_(0),
- repeat_count_(0),
- literal_count_(0) {
- ARROW_DCHECK_GE(bit_width_, 0);
- ARROW_DCHECK_LE(bit_width_, 64);
+ /// The decoder class used to decode a single run in the given type.
+ template <typename T>
+ using DecoderType = RleRunDecoder<T>;
+
+ constexpr RleRun() noexcept = default;
+
+ explicit RleRun(const uint8_t* data, rle_size_t values_count,
+ rle_size_t value_bit_width) noexcept
+ : values_count_(values_count), value_bit_width_(value_bit_width) {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_GE(values_count, 0);
+ std::copy(data, data + raw_data_size(), data_.begin());
}
- RleDecoder() : bit_width_(-1) {}
+ /// The number of repeated values in this run.
+ constexpr rle_size_t values_count() const noexcept { return values_count_; }
- void Reset(const uint8_t* buffer, int buffer_len, int bit_width) {
- ARROW_DCHECK_GE(bit_width, 0);
- ARROW_DCHECK_LE(bit_width, 64);
- bit_reader_.Reset(buffer, buffer_len);
- bit_width_ = bit_width;
- current_value_ = 0;
- repeat_count_ = 0;
- literal_count_ = 0;
+ /// The size in bits of each encoded value.
+ constexpr rle_size_t values_bit_width() const noexcept { return value_bit_width_; }
+
+ /// A pointer to the repeated value raw bytes.
+ constexpr const uint8_t* raw_data_ptr() const noexcept { return data_.data(); }
+
+ /// The number of bytes used for the raw repeated value.
+ constexpr rle_size_t raw_data_size() const noexcept {
+ auto out = bit_util::BytesForBits(value_bit_width_);
+ ARROW_DCHECK_LE(out, std::numeric_limits<rle_size_t>::max());
+ return static_cast<rle_size_t>(out);
}
- /// Gets the next value. Returns false if there are no more.
+ private:
+ /// The repeated value raw bytes stored inside the class with enough space to store
+ /// up to a 64 bit value.
+ std::array<uint8_t, 8> data_ = {};
+ /// The number of time the value is repeated.
+ rle_size_t values_count_ = 0;
+ /// The size in bit of a packed value in the run.
+ rle_size_t value_bit_width_ = 0;
+};
+
+template <typename T>
+class BitPackedRunDecoder;
+
+class BitPackedRun {
+ public:
+ /// The decoder class used to decode a single run in the given type.
+ template <typename T>
+ using DecoderType = BitPackedRunDecoder<T>;
+
+ constexpr BitPackedRun() noexcept = default;
+
+ constexpr BitPackedRun(const uint8_t* data, rle_size_t values_count,
+ rle_size_t value_bit_width) noexcept
+ : data_(data), values_count_(values_count), value_bit_width_(value_bit_width) {
+ ARROW_CHECK_GE(value_bit_width_, 0);
+ ARROW_CHECK_GE(values_count_, 0);
+ }
+
+ constexpr rle_size_t values_count() const noexcept { return values_count_; }
+
+ /// The size in bits of each encoded value.
+ constexpr rle_size_t values_bit_width() const noexcept { return value_bit_width_; }
+
+ constexpr const uint8_t* raw_data_ptr() const noexcept { return data_; }
+
+ constexpr rle_size_t raw_data_size() const noexcept {
+ auto out = bit_util::BytesForBits(static_cast<int64_t>(value_bit_width_) *
+ static_cast<int64_t>(values_count_));
+ ARROW_CHECK_LE(out, std::numeric_limits<rle_size_t>::max());
+ return static_cast<rle_size_t>(out);
+ }
+
+ private:
+ /// The pointer to the beginning of the run
+ const uint8_t* data_ = nullptr;
+ /// Number of values in this run.
+ rle_size_t values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ rle_size_t value_bit_width_ = 0;
+};
+
+/// A parser that emits either a ``BitPackedRun`` or a ``RleRun``.
+class RleBitPackedParser {
+ public:
+ /// The different types of runs emitted by the parser
+ using dynamic_run_type = std::variant<RleRun, BitPackedRun>;
+
+ constexpr RleBitPackedParser() noexcept = default;
+
+ constexpr RleBitPackedParser(const uint8_t* data, rle_size_t data_size,
+ rle_size_t value_bit_width) noexcept
+ : data_(data), data_size_(data_size), value_bit_width_(value_bit_width) {}
+
+ constexpr void Reset(const uint8_t* data, rle_size_t data_size,
+ rle_size_t value_bit_width) noexcept {
+ *this = {data, data_size, value_bit_width};
+ }
+
+ /// Whether there is still runs to iterate over.
+ ///
+ /// WARN: Due to simplistic error handling, iteration with Next and Peek could
+ /// fail to return data while the parser is not exhausted.
+ /// This is how one can check for errors.
+ bool exhausted() const { return data_size_ == 0; }
+
+ /// Enum to return from an ``Parse`` handler.
+ ///
+ /// Since a callback has no way to know when to stop, the handler must return
+ /// a value indicating to the ``Parse`` function whether to stop or continue.
+ enum class ControlFlow {
+ Continue,
+ Break,
+ };
+
+ /// A callback approach to parsing.
+ ///
+ /// This approach is used to reduce the number of dynamic lookups involved with using a
+ /// variant.
+ ///
+ /// The handler must be of the form
+ /// ```cpp
+ /// struct Handler {
+ /// ControlFlow OnBitPackedRun(BitPackedRun run);
+ ///
+ /// ControlFlow OnRleRun(RleRun run);
+ /// };
+ /// ```
+ template <typename Handler>
+ void Parse(Handler&& handler);
+
+ private:
+ /// The pointer to the beginning of the run
+ const uint8_t* data_ = nullptr;
+ /// Size in bytes of the run.
+ rle_size_t data_size_ = 0;
+ /// The size in bit of a packed value in the run
+ rle_size_t value_bit_width_ = 0;
Review Comment:
Again, it depends if it helps improve performance. If not, then let's not bother.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on PR #47294:
URL: https://github.com/apache/arrow/pull/47294#issuecomment-3224791934
@pitrou this is ready for review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2307046385
##########
cpp/src/arrow/util/bit_util_test.cc:
##########
@@ -1997,11 +1997,119 @@ TEST(BitUtil, RoundUpToPowerOf2) {
#undef U64
#undef S64
+/// Test the maximum number of bytes needed to write a LEB128 of a give size.
+TEST(BitStreamUtil, MaxLEB128ByteLenFor) {
Review Comment:
Either way works for me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2349449723
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ ARROW_DCHECK_LT(values_read, batch_size);
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+ auto const read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
return values_read;
}
-template <typename T, typename RunType, typename Converter>
-inline int RleDecoder::GetSpaced(Converter converter, int batch_size, int null_count,
- const uint8_t* valid_bits, int64_t valid_bits_offset,
- T* out) {
- if (ARROW_PREDICT_FALSE(null_count == batch_size)) {
- converter.FillZero(out, out + batch_size);
- return batch_size;
+namespace internal {
+
+/// Utility class to safely handle values and null count without too error-prone
+/// verbosity.
+class BatchCounter {
+ public:
+ using size_type = int32_t;
+
+ [[nodiscard]] static constexpr BatchCounter FromBatchSizeAndNulls(
+ size_type batch_size, size_type null_count) {
+ ARROW_DCHECK_LE(null_count, batch_size);
+ return {batch_size - null_count, null_count};
}
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
- int values_remaining = batch_size - null_count;
+ constexpr BatchCounter(size_type values_count, size_type null_count) noexcept
+ : values_count_(values_count), null_count_(null_count) {}
- // Assume no bits to start.
- arrow::internal::BitRunReader bit_reader(valid_bits, valid_bits_offset,
- /*length=*/batch_size);
- arrow::internal::BitRun valid_run = bit_reader.NextRun();
- while (values_read < batch_size) {
- if (ARROW_PREDICT_FALSE(valid_run.length == 0)) {
- valid_run = bit_reader.NextRun();
+ [[nodiscard]] constexpr size_type ValuesCount() const noexcept { return values_count_; }
+
+ [[nodiscard]] constexpr size_type ValuesRead() const noexcept { return values_read_; }
+
+ [[nodiscard]] constexpr size_type ValuesRemaining() const noexcept {
+ ARROW_DCHECK_LE(values_read_, values_count_);
+ return values_count_ - values_read_;
+ }
+
+ constexpr void AccrueReadValues(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, ValuesRemaining());
+ values_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type NullCount() const noexcept { return null_count_; }
+
+ [[nodiscard]] constexpr size_type NullRead() const noexcept { return null_read_; }
+
+ [[nodiscard]] constexpr size_type NullRemaining() const noexcept {
+ ARROW_DCHECK_LE(null_read_, null_count_);
+ return null_count_ - null_read_;
+ }
+
+ constexpr void AccrueReadNulls(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, NullRemaining());
+ null_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type TotalRemaining() const noexcept {
+ return ValuesRemaining() + NullRemaining();
+ }
+
+ [[nodiscard]] constexpr size_type TotalRead() const noexcept {
+ return values_read_ + null_read_;
+ }
+
+ [[nodiscard]] constexpr bool IsFullyNull() const noexcept {
+ return ValuesRemaining() == 0;
+ }
+
+ [[nodiscard]] constexpr bool IsDone() const noexcept { return TotalRemaining() == 0; }
+
+ private:
+ size_type values_count_ = 0;
+ size_type values_read_ = 0;
+ size_type null_count_ = 0;
+ size_type null_read_ = 0;
+};
+
+// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+/// Overload for GetSpaced for a single run in a RleDecoder
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ RleDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto values_remaining_run = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // Consume as much as possible from the repeated run.
+ // We only need to count the number of nulls and non-nulls because we can fill in the
+ // same value for nulls and non-nulls.
+ // This proves to be a big efficiency win.
+ while (values_remaining_run() > 0 && !batch.IsDone()) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const& validity_run_size = static_cast<values_count_type>(validity_run.length);
+
+ if (validity_run.set) {
+ // We may end the current RLE run in the middle of the validity run
+ auto update_size = std::min(validity_run_size, values_remaining_run());
+ batch.AccrueReadValues(update_size);
+ validity_run.length -= update_size;
+ } else {
+ // We can consume all nulls here because it does not matter if we consume on this
+ // RLE run, or an a next encoded run. The value filled does not matter.
+ auto update_size = std::min(validity_run_size, batch.NullRemaining());
+ batch.AccrueReadNulls(update_size);
+ validity_run.length -= update_size;
+ }
+
+ if (ARROW_PREDICT_TRUE(validity_run.length == 0)) {
+ validity_run = validity_reader.NextRun();
}
+ }
+
+ value_type const value = decoder.Value();
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(value))) {
+ return {0, 0};
+ }
+ converter.WriteRepeated(out, out + batch.TotalRead(), value);
+ auto const actual_values_read = decoder.Advance(batch.ValuesRead());
+ // We always cropped the number of values_read by the remaining values in the run.
+ // What's more the RLE decoder should not encounter any errors.
+ ARROW_DCHECK_EQ(actual_values_read, batch.ValuesRead());
- ARROW_DCHECK_GT(batch_size, 0);
- ARROW_DCHECK_GT(valid_run.length, 0);
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+template <typename T, typename... Ts>
+[[nodiscard]] constexpr T min(T x, Ts... ys) {
+ ((x = std::min(x, ys)), ...);
+ return x;
+}
+
+static_assert(min(5) == 5);
+static_assert(min(5, 4, -1) == -1);
+static_assert(min(5, 41) == 5);
+
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ BitPackedDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto run_values_remaining = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ while (run_values_remaining() > 0 && batch.ValuesRemaining() > 0) {
+ // TODO should this size be tune depending on sizeof(value_size)? cpu cache size?
+ // Pull a batch of values from the bit packed encoded data and store it in a local
+ // buffer to benefit from unpacking intrinsics and data locality.
+ static constexpr values_count_type kBufferCapacity = 1024;
+ std::array<value_type, kBufferCapacity> buffer = {};
+
+ values_count_type buffer_start = 0;
+ values_count_type buffer_end = 0;
+ auto buffer_size = [&]() {
+ auto out = buffer_end - buffer_start;
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // buffer_start is 0 at this point so size is end
+ buffer_end = min(run_values_remaining(), batch.ValuesRemaining(), kBufferCapacity);
+ buffer_end = decoder.GetBatch(buffer.data(), buffer_size());
+ ARROW_DCHECK_LE(buffer_size(), kBufferCapacity);
+
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(buffer.data(), buffer_size()))) {
+ return {batch.ValuesRead(), batch.NullRead()};
+ }
- if (valid_run.set) {
- if ((repeat_count_ == 0) && (literal_count_ == 0)) {
- if (!NextCounts<RunType>()) return values_read;
- ARROW_DCHECK((repeat_count_ > 0) ^ (literal_count_ > 0));
+ // Copy chunks of valid values into the output, while adjusting spacing for null
+ // values.
+ while (buffer_size() > 0) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const validity_run_length =
+ static_cast<values_count_type>(validity_run.length);
+
+ // Copy as much as possible from the buffer into the output while not exceeding
+ // validity run
+ if (validity_run.set) {
+ auto const update_size = std::min(validity_run_length, buffer_size());
+ converter.WriteRange(out, buffer.data() + buffer_start, update_size);
+ buffer_start += update_size;
+ batch.AccrueReadValues(update_size);
+ out += update_size;
+ validity_run.length -= update_size;
+ // Simply write zeros in the output
+ } else {
+ auto const update_size = std::min(validity_run_length, batch.NullRemaining());
+ converter.WriteZero(out, out + update_size);
+ batch.AccrueReadNulls(update_size);
+ out += update_size;
+ validity_run.length -= update_size;
}
- if (repeat_count_ > 0) {
- int repeat_batch = 0;
- // Consume the entire repeat counts incrementing repeat_batch to
- // be the total of nulls + values consumed, we only need to
- // get the total count because we can fill in the same value for
- // nulls and non-nulls. This proves to be a big efficiency win.
- while (repeat_count_ > 0 && (values_read + repeat_batch) < batch_size) {
- ARROW_DCHECK_GT(valid_run.length, 0);
- if (valid_run.set) {
- int update_size = std::min(static_cast<int>(valid_run.length), repeat_count_);
- repeat_count_ -= update_size;
- repeat_batch += update_size;
- valid_run.length -= update_size;
- values_remaining -= update_size;
- } else {
- // We can consume all nulls here because we would do so on
- // the next loop anyways.
- repeat_batch += static_cast<int>(valid_run.length);
- valid_run.length = 0;
- }
- if (valid_run.length == 0) {
- valid_run = bit_reader.NextRun();
- }
- }
- RunType current_value = static_cast<RunType>(current_value_);
- if (ARROW_PREDICT_FALSE(!converter.IsValid(current_value))) {
- return values_read;
- }
- converter.Fill(out, out + repeat_batch, current_value);
- out += repeat_batch;
- values_read += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(values_remaining, literal_count_);
- ARROW_DCHECK_GT(literal_batch, 0);
-
- // Decode the literals
- constexpr int kBufferSize = 1024;
- RunType indices[kBufferSize];
- literal_batch = std::min(literal_batch, kBufferSize);
- int actual_read = bit_reader_.GetBatch(bit_width_, indices, literal_batch);
- if (ARROW_PREDICT_FALSE(actual_read != literal_batch)) {
- return values_read;
- }
- if (!converter.IsValid(indices, /*length=*/actual_read)) {
- return values_read;
- }
- int skipped = 0;
- int literals_read = 0;
- while (literals_read < literal_batch) {
- if (valid_run.set) {
- int update_size = std::min(literal_batch - literals_read,
- static_cast<int>(valid_run.length));
- converter.Copy(out, indices + literals_read, update_size);
- literals_read += update_size;
- out += update_size;
- valid_run.length -= update_size;
- } else {
- converter.FillZero(out, out + valid_run.length);
- out += valid_run.length;
- skipped += static_cast<int>(valid_run.length);
- valid_run.length = 0;
- }
- if (valid_run.length == 0) {
- valid_run = bit_reader.NextRun();
- }
- }
- literal_count_ -= literal_batch;
- values_remaining -= literal_batch;
- values_read += literal_batch + skipped;
+ if (validity_run.length == 0) {
+ validity_run = validity_reader.NextRun();
}
- } else {
- converter.FillZero(out, out + valid_run.length);
- out += valid_run.length;
- values_read += static_cast<int>(valid_run.length);
- valid_run.length = 0;
}
+
+ ARROW_DCHECK_EQ(buffer_size(), 0);
}
- ARROW_DCHECK_EQ(valid_run.length, 0);
- ARROW_DCHECK_EQ(values_remaining, 0);
- return values_read;
+
+ ARROW_DCHECK_EQ(values_available - decoder.Remaining(), batch.ValuesRead());
+ ARROW_DCHECK_LE(batch.TotalRead(), batch_size);
+ ARROW_DCHECK_LE(batch.NullRead(), batch.NullCount());
+
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+/// Overload for GetSpaced for a single run in a decoder variant
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(
+ Converter& converter, typename Converter::out_type* out, values_count_type batch_size,
+ values_count_type null_count, BitRunReader&& validity_reader, BitRun&& validity_run,
+ std::variant<RleDecoder<value_type>, BitPackedDecoder<value_type>>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ return std::visit(
+ [&](auto& dec) {
+ ARROW_DCHECK_GT(dec.Remaining(), 0);
+ return RunGetSpaced(converter, out, batch_size, null_count, validity_reader,
+ validity_run, dec);
+ },
+ decoder);
}
+} // namespace internal
+
+template <typename T>
+template <typename Converter>
+auto RleBitPackedDecoder<T>::GetSpaced(
+ Converter converter, typename Converter::out_type* out, values_count_type batch_size,
+ const byte* validity_bits, int64_t validity_bits_offset, values_count_type null_count)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
+
+ ARROW_DCHECK_GT(batch_size, 0);
+
+ auto batch = internal::BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ if (ARROW_PREDICT_FALSE(batch.IsFullyNull())) {
+ converter.WriteZero(out, out + batch.NullRemaining());
+ return batch.NullRemaining();
+ }
+
+ arrow::internal::BitRunReader validity_reader(validity_bits, validity_bits_offset,
+ /*length=*/batch.TotalRemaining());
+ arrow::internal::BitRun validity_run = validity_reader.NextRun();
+
+ auto const check_and_handle_fully_null_remaining = [&]() {
+ if (batch.IsFullyNull()) {
+ ARROW_DCHECK(validity_run.length == 0 || !validity_run.set);
+ ARROW_DCHECK_GE(validity_run.length, batch.NullRemaining());
+
+ converter.WriteZero(out, out + batch.NullRemaining());
+ out += batch.NullRemaining();
+ batch.AccrueReadNulls(batch.NullRemaining());
+ }
+ };
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
Review Comment:
Benchmarks seems to prefer these predictions (and the other ones with similar comment).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2349567807
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -84,32 +85,278 @@ namespace util {
/// (total 26 bytes, 1 byte overhead)
//
+class RleRun {
+ public:
+ using byte = uint8_t;
+ /// Enough space to store a 64bit value
+ using raw_data_storage = std::array<byte, 8>;
+ using raw_data_const_pointer = const byte*;
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr RleRun() noexcept = default;
+ constexpr RleRun(RleRun const&) noexcept = default;
+ constexpr RleRun(RleRun&&) noexcept = default;
+
+ explicit RleRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr RleRun& operator=(RleRun const&) noexcept = default;
+ constexpr RleRun& operator=(RleRun&&) noexcept = default;
+
+ /// The number of repeated values in this run.
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ /// A pointer to the repeated value raw bytes.
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ /// The number of bytes used for the raw repeated value.
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The repeated value raw bytes stored inside the class
+ raw_data_storage data_ = {};
+ /// The number of time the value is repeated
+ values_count_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+class BitPackedRun {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// According to the Parquet thrift definition the page size can be written into an
+ /// int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr BitPackedRun() noexcept = default;
+ constexpr BitPackedRun(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun(BitPackedRun&&) noexcept = default;
+
+ constexpr BitPackedRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr BitPackedRun& operator=(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun& operator=(BitPackedRun&&) noexcept = default;
+
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Number of values in this run.
+ raw_data_size_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+/// A parser that emits either a ``BitPackedRun`` or a ``RleRun``.
+class RleBitPackedParser {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// By Parquet thrift definition the page size can be written into an int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+ /// The different types of runs emitted by the parser
+ using dynamic_run_type = std::variant<RleRun, BitPackedRun>;
+
+ constexpr RleBitPackedParser() noexcept = default;
+
+ constexpr RleBitPackedParser(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr void Reset(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width_) noexcept;
+
+ /// Get the current run with a small parsing cost without advancing the iteration.
+ [[nodiscard]] std::optional<dynamic_run_type> Peek() const;
+
+ /// Move to the next run.
+ [[nodiscard]] bool Advance();
+
+ /// Advance and return the current run.
+ [[nodiscard]] std::optional<dynamic_run_type> Next();
+
+ /// Whether there is still runs to iterate over.
+ ///
+ /// WARN: Due to lack of proper error handling, iteration with Next and Peek could
+ /// return not data while the parser is not exhausted.
+ /// This is how one can check for errors.
+ [[nodiscard]] bool Exhausted() const;
+
+ /// Enum to return from an ``Parse`` handler.
+ ///
+ /// Since a callback has no way to know when to stop, the handler must return
+ /// a value indicating to the ``Parse`` function whether to stop or continue.
+ enum class ControlFlow {
+ Continue,
+ Break,
+ };
+
+ /// A callback approach to parsing.
+ ///
+ /// This approach is used to reduce the number of dynamic lookups involved with using a
+ /// variant.
+ ///
+ /// The handler must be of the form
+ /// ```cpp`
+ /// struct Handler {
+ /// ControlFlow OnBitPackedRun(BitPackedRun run);
+ ///
+ /// ControlFlow OnRleRun(RleRun run);
+ /// };
+ /// ```
+ template <typename Handler>
+ void Parse(Handler&& handler);
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Size in bytes of the run.
+ raw_data_size_type data_size_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+
+ /// Run the handler on the run read and return the number of values read.
+ /// Does not advance the parser.
+ template <typename Handler>
+ std::pair<raw_data_size_type, ControlFlow> PeekImpl(Handler&&) const;
+};
+
/// Decoder class for RLE encoded data.
+template <typename T>
class RleDecoder {
public:
- /// Create a decoder object. buffer/buffer_len is the decoded data.
- /// bit_width is the width of each value (before encoding).
- RleDecoder(const uint8_t* buffer, int buffer_len, int bit_width)
- : bit_reader_(buffer, buffer_len),
- bit_width_(bit_width),
- current_value_(0),
- repeat_count_(0),
- literal_count_(0) {
- ARROW_DCHECK_GE(bit_width_, 0);
- ARROW_DCHECK_LE(bit_width_, 64);
- }
+ /// The type in which the data should be decoded.
+ using value_type = T;
+ /// The type of run that can be decoded.
+ using run_type = RleRun;
+ using values_count_type = run_type::values_count_type;
- RleDecoder() : bit_width_(-1) {}
+ constexpr RleDecoder() noexcept = default;
- void Reset(const uint8_t* buffer, int buffer_len, int bit_width) {
- ARROW_DCHECK_GE(bit_width, 0);
- ARROW_DCHECK_LE(bit_width, 64);
- bit_reader_.Reset(buffer, buffer_len);
- bit_width_ = bit_width;
- current_value_ = 0;
- repeat_count_ = 0;
- literal_count_ = 0;
- }
+ explicit RleDecoder(run_type const& run) noexcept;
+
+ void Reset(run_type const& run) noexcept;
+
+ /// Return the number of values that can be advanced.
+ [[nodiscard]] values_count_type Remaining() const;
Review Comment:
Yes, I have applied them liberally in all cases where "*it does nothing if you call this function and not use the output*".
> For now, we only use `[[nodiscard]]` in cases where it is dangerous to ignore a return value (such as an error).
For all functions reading data, I'd consider it dangerous not to read the output (e.g. number of bytes read) since it carries error information (e.g. when it is zero). What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2349553232
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -84,32 +85,278 @@ namespace util {
/// (total 26 bytes, 1 byte overhead)
//
+class RleRun {
+ public:
+ using byte = uint8_t;
+ /// Enough space to store a 64bit value
+ using raw_data_storage = std::array<byte, 8>;
+ using raw_data_const_pointer = const byte*;
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr RleRun() noexcept = default;
+ constexpr RleRun(RleRun const&) noexcept = default;
+ constexpr RleRun(RleRun&&) noexcept = default;
+
+ explicit RleRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr RleRun& operator=(RleRun const&) noexcept = default;
+ constexpr RleRun& operator=(RleRun&&) noexcept = default;
+
+ /// The number of repeated values in this run.
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ /// A pointer to the repeated value raw bytes.
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ /// The number of bytes used for the raw repeated value.
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The repeated value raw bytes stored inside the class
+ raw_data_storage data_ = {};
+ /// The number of time the value is repeated
+ values_count_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+class BitPackedRun {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// According to the Parquet thrift definition the page size can be written into an
+ /// int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr BitPackedRun() noexcept = default;
+ constexpr BitPackedRun(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun(BitPackedRun&&) noexcept = default;
+
+ constexpr BitPackedRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr BitPackedRun& operator=(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun& operator=(BitPackedRun&&) noexcept = default;
+
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Number of values in this run.
+ raw_data_size_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+/// A parser that emits either a ``BitPackedRun`` or a ``RleRun``.
+class RleBitPackedParser {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// By Parquet thrift definition the page size can be written into an int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+ /// The different types of runs emitted by the parser
+ using dynamic_run_type = std::variant<RleRun, BitPackedRun>;
+
+ constexpr RleBitPackedParser() noexcept = default;
+
+ constexpr RleBitPackedParser(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr void Reset(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width_) noexcept;
+
+ /// Get the current run with a small parsing cost without advancing the iteration.
+ [[nodiscard]] std::optional<dynamic_run_type> Peek() const;
+
+ /// Move to the next run.
+ [[nodiscard]] bool Advance();
+
+ /// Advance and return the current run.
+ [[nodiscard]] std::optional<dynamic_run_type> Next();
+
+ /// Whether there is still runs to iterate over.
+ ///
+ /// WARN: Due to lack of proper error handling, iteration with Next and Peek could
+ /// return not data while the parser is not exhausted.
+ /// This is how one can check for errors.
+ [[nodiscard]] bool Exhausted() const;
+
+ /// Enum to return from an ``Parse`` handler.
+ ///
+ /// Since a callback has no way to know when to stop, the handler must return
+ /// a value indicating to the ``Parse`` function whether to stop or continue.
+ enum class ControlFlow {
+ Continue,
+ Break,
+ };
+
+ /// A callback approach to parsing.
+ ///
+ /// This approach is used to reduce the number of dynamic lookups involved with using a
+ /// variant.
+ ///
+ /// The handler must be of the form
+ /// ```cpp`
+ /// struct Handler {
+ /// ControlFlow OnBitPackedRun(BitPackedRun run);
+ ///
+ /// ControlFlow OnRleRun(RleRun run);
+ /// };
+ /// ```
+ template <typename Handler>
+ void Parse(Handler&& handler);
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Size in bytes of the run.
+ raw_data_size_type data_size_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+
+ /// Run the handler on the run read and return the number of values read.
+ /// Does not advance the parser.
+ template <typename Handler>
+ std::pair<raw_data_size_type, ControlFlow> PeekImpl(Handler&&) const;
+};
+
/// Decoder class for RLE encoded data.
+template <typename T>
class RleDecoder {
public:
- /// Create a decoder object. buffer/buffer_len is the decoded data.
- /// bit_width is the width of each value (before encoding).
- RleDecoder(const uint8_t* buffer, int buffer_len, int bit_width)
- : bit_reader_(buffer, buffer_len),
- bit_width_(bit_width),
- current_value_(0),
- repeat_count_(0),
- literal_count_(0) {
- ARROW_DCHECK_GE(bit_width_, 0);
- ARROW_DCHECK_LE(bit_width_, 64);
- }
+ /// The type in which the data should be decoded.
+ using value_type = T;
+ /// The type of run that can be decoded.
+ using run_type = RleRun;
+ using values_count_type = run_type::values_count_type;
- RleDecoder() : bit_width_(-1) {}
+ constexpr RleDecoder() noexcept = default;
- void Reset(const uint8_t* buffer, int buffer_len, int bit_width) {
- ARROW_DCHECK_GE(bit_width, 0);
- ARROW_DCHECK_LE(bit_width, 64);
- bit_reader_.Reset(buffer, buffer_len);
- bit_width_ = bit_width;
- current_value_ = 0;
- repeat_count_ = 0;
- literal_count_ = 0;
- }
+ explicit RleDecoder(run_type const& run) noexcept;
Review Comment:
> Ok, does a `RleDecoder` decode a single RLE run? This should be explicit in the docstrings somewhere.
Done.
> Also, I would expect all methods to be marked `inline` if that is really the case.
`inline` is already implied by templates.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2355593962
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ ARROW_DCHECK_LT(values_read, batch_size);
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+ auto const read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
return values_read;
}
-template <typename T, typename RunType, typename Converter>
-inline int RleDecoder::GetSpaced(Converter converter, int batch_size, int null_count,
- const uint8_t* valid_bits, int64_t valid_bits_offset,
- T* out) {
- if (ARROW_PREDICT_FALSE(null_count == batch_size)) {
- converter.FillZero(out, out + batch_size);
- return batch_size;
+namespace internal {
+
+/// Utility class to safely handle values and null count without too error-prone
+/// verbosity.
+class BatchCounter {
+ public:
+ using size_type = int32_t;
+
+ [[nodiscard]] static constexpr BatchCounter FromBatchSizeAndNulls(
+ size_type batch_size, size_type null_count) {
+ ARROW_DCHECK_LE(null_count, batch_size);
+ return {batch_size - null_count, null_count};
}
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
- int values_remaining = batch_size - null_count;
+ constexpr BatchCounter(size_type values_count, size_type null_count) noexcept
+ : values_count_(values_count), null_count_(null_count) {}
- // Assume no bits to start.
- arrow::internal::BitRunReader bit_reader(valid_bits, valid_bits_offset,
- /*length=*/batch_size);
- arrow::internal::BitRun valid_run = bit_reader.NextRun();
- while (values_read < batch_size) {
- if (ARROW_PREDICT_FALSE(valid_run.length == 0)) {
- valid_run = bit_reader.NextRun();
+ [[nodiscard]] constexpr size_type ValuesCount() const noexcept { return values_count_; }
+
+ [[nodiscard]] constexpr size_type ValuesRead() const noexcept { return values_read_; }
+
+ [[nodiscard]] constexpr size_type ValuesRemaining() const noexcept {
+ ARROW_DCHECK_LE(values_read_, values_count_);
+ return values_count_ - values_read_;
+ }
+
+ constexpr void AccrueReadValues(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, ValuesRemaining());
+ values_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type NullCount() const noexcept { return null_count_; }
+
+ [[nodiscard]] constexpr size_type NullRead() const noexcept { return null_read_; }
+
+ [[nodiscard]] constexpr size_type NullRemaining() const noexcept {
+ ARROW_DCHECK_LE(null_read_, null_count_);
+ return null_count_ - null_read_;
+ }
+
+ constexpr void AccrueReadNulls(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, NullRemaining());
+ null_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type TotalRemaining() const noexcept {
+ return ValuesRemaining() + NullRemaining();
+ }
+
+ [[nodiscard]] constexpr size_type TotalRead() const noexcept {
+ return values_read_ + null_read_;
+ }
+
+ [[nodiscard]] constexpr bool IsFullyNull() const noexcept {
+ return ValuesRemaining() == 0;
+ }
+
+ [[nodiscard]] constexpr bool IsDone() const noexcept { return TotalRemaining() == 0; }
+
+ private:
+ size_type values_count_ = 0;
+ size_type values_read_ = 0;
+ size_type null_count_ = 0;
+ size_type null_read_ = 0;
+};
+
+// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+/// Overload for GetSpaced for a single run in a RleDecoder
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ RleDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto values_remaining_run = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // Consume as much as possible from the repeated run.
+ // We only need to count the number of nulls and non-nulls because we can fill in the
+ // same value for nulls and non-nulls.
+ // This proves to be a big efficiency win.
+ while (values_remaining_run() > 0 && !batch.IsDone()) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const& validity_run_size = static_cast<values_count_type>(validity_run.length);
+
+ if (validity_run.set) {
+ // We may end the current RLE run in the middle of the validity run
+ auto update_size = std::min(validity_run_size, values_remaining_run());
+ batch.AccrueReadValues(update_size);
+ validity_run.length -= update_size;
+ } else {
+ // We can consume all nulls here because it does not matter if we consume on this
+ // RLE run, or an a next encoded run. The value filled does not matter.
+ auto update_size = std::min(validity_run_size, batch.NullRemaining());
+ batch.AccrueReadNulls(update_size);
+ validity_run.length -= update_size;
+ }
+
+ if (ARROW_PREDICT_TRUE(validity_run.length == 0)) {
+ validity_run = validity_reader.NextRun();
}
+ }
+
+ value_type const value = decoder.Value();
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(value))) {
+ return {0, 0};
+ }
+ converter.WriteRepeated(out, out + batch.TotalRead(), value);
+ auto const actual_values_read = decoder.Advance(batch.ValuesRead());
+ // We always cropped the number of values_read by the remaining values in the run.
+ // What's more the RLE decoder should not encounter any errors.
+ ARROW_DCHECK_EQ(actual_values_read, batch.ValuesRead());
- ARROW_DCHECK_GT(batch_size, 0);
- ARROW_DCHECK_GT(valid_run.length, 0);
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+template <typename T, typename... Ts>
+[[nodiscard]] constexpr T min(T x, Ts... ys) {
+ ((x = std::min(x, ys)), ...);
+ return x;
+}
+
+static_assert(min(5) == 5);
+static_assert(min(5, 4, -1) == -1);
+static_assert(min(5, 41) == 5);
+
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ BitPackedDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto run_values_remaining = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ while (run_values_remaining() > 0 && batch.ValuesRemaining() > 0) {
+ // TODO should this size be tune depending on sizeof(value_size)? cpu cache size?
Review Comment:
Under (quick) investigation for sizes 256, 512, 1024, 2048.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2307043163
##########
cpp/src/arrow/util/bit_util.h:
##########
@@ -365,5 +365,118 @@ void PackBits(const uint32_t* values, uint8_t* out) {
}
}
+constexpr int64_t MaxLEB128ByteLen(int64_t n_bits) { return CeilDiv(n_bits, 7); }
+
+template <typename Int>
+constexpr int64_t MaxLEB128ByteLenFor = MaxLEB128ByteLen(sizeof(Int) * 8);
+
+/// Write a integer as LEB128
+///
+/// Write the input value as LEB128 into the outptu buffer and return the number of bytes
+/// written.
+/// If the output buffer size is insufficient, return 0 but the output may have been
+/// written to.
+///
+/// \see https://en.wikipedia.org/wiki/LEB128
+/// \see MaxLEB128ByteLenFor
+template <typename Int>
+constexpr int32_t WriteLEB128(Int value, uint8_t* out, int32_t max_out_size) {
Review Comment:
The intent *is* to move the functions and refactor them.
The reason is previous function was a method of `BitReader` while this PR need to be able to parse ints without a bit reader.
`BitReader::GetVlqInt` uses this new function, though because the previous function parses from a contiguous array while the `BitReader` used `GetAligned` there is are checks to find such an array. Another way could be that the new function takes a callback to get the next int but that felt less readable (would have been quite generic with C++ iterators but there are no such APIs in this part of the code).
> why is BitWriter::PutVlqInt(uint32_t v) not replaced?
An oversight, I should modify it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2349123162
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
Review Comment:
Method is removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2349510714
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -84,32 +85,278 @@ namespace util {
/// (total 26 bytes, 1 byte overhead)
//
+class RleRun {
+ public:
+ using byte = uint8_t;
+ /// Enough space to store a 64bit value
+ using raw_data_storage = std::array<byte, 8>;
+ using raw_data_const_pointer = const byte*;
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr RleRun() noexcept = default;
Review Comment:
Looking only at the Run classes, they do feel like they could be simple classes.
However, in the much other functions in this file, they bring a much welcomed (small) reduction of complexity:
- The two run classes have a similar interface;
- They manage the conversion from bit size and number of values to a ready to use ptr+len;
- `RleRun` hides the unused bytes.
Perhaps we could instead put the member functions definition within the class (as suggested by another comment).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2334113286
##########
cpp/src/arrow/util/bit_util_test.cc:
##########
@@ -1997,11 +1997,119 @@ TEST(BitUtil, RoundUpToPowerOf2) {
#undef U64
#undef S64
+/// Test the maximum number of bytes needed to write a LEB128 of a give size.
+TEST(BitStreamUtil, MaxLEB128ByteLenFor) {
+ EXPECT_EQ(bit_util::MaxLEB128ByteLenFor<int16_t>, 3);
+ EXPECT_EQ(bit_util::MaxLEB128ByteLenFor<int32_t>, 5);
+ EXPECT_EQ(bit_util::MaxLEB128ByteLenFor<int64_t>, 10);
+}
+
+/// Utility function to test LEB128 encoding with known input value and expected byte
+/// array
+template <typename Int, std::size_t N>
+void TestLEB128Encode(Int input_value, std::array<uint8_t, N> const& expected_data,
+ int32_t expected_bytes_written) {
+ std::array<uint8_t, N> buffer{};
+ auto bytes_written = bit_util::WriteLEB128(input_value, buffer.data(),
+ static_cast<int32_t>(buffer.size()));
+ EXPECT_EQ(bytes_written, expected_bytes_written);
+ if (bytes_written > 0) {
+ EXPECT_EQ(buffer, expected_data);
+ }
+}
+
+/// Test encoding to known LEB128 byte sequences
+TEST(WriteLEB128Test, KnownArrayValues) {
+ // Single byte value 0
+ TestLEB128Encode(0U, std::array<uint8_t, 1>{0x00}, 1);
+ // Single byte value 127
+ TestLEB128Encode(127U, std::array<uint8_t, 1>{0x7F}, 1);
+ // Two byte value 128
+ TestLEB128Encode(128U, std::array<uint8_t, 2>{0x80, 0x01}, 2);
+ // Two byte value 300
+ TestLEB128Encode(300U, std::array<uint8_t, 2>{0xAC, 0x02}, 2);
+ // Three byte value 16384
+ TestLEB128Encode(16384U, std::array<uint8_t, 3>{0x80, 0x80, 0x01}, 3);
+ // Four byte value 268435455
+ TestLEB128Encode(268435455U, std::array<uint8_t, 4>{0xFF, 0xFF, 0xFF, 0x7F}, 4);
+ // Five byte uint32_t max value
+ TestLEB128Encode(4294967295U, std::array<uint8_t, 5>{0xFF, 0xFF, 0xFF, 0xFF, 0x0F}, 5);
+ // uint64_t value requiring 10 bytes
+ TestLEB128Encode(
+ 18446744073709551615ULL,
+ std::array<uint8_t, 10>{0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x01},
+ 10);
+ // Edge case: Exact buffer size match
+ TestLEB128Encode(16384U, std::array<uint8_t, 3>{0x80, 0x80, 0x01}, 3);
+ // Various single byte values
+ TestLEB128Encode(1U, std::array<uint8_t, 1>{0x01}, 1);
+ TestLEB128Encode(63U, std::array<uint8_t, 1>{0x3F}, 1);
+ TestLEB128Encode(64U, std::array<uint8_t, 1>{0x40}, 1);
+ // Two byte boundary values
+ TestLEB128Encode(129U, std::array<uint8_t, 2>{0x81, 0x01}, 2);
+ TestLEB128Encode(16383U, std::array<uint8_t, 2>{0xFF, 0x7F}, 2);
+ // Error case: Buffer too small for value 128 (needs 2 bytes but only 1 provided)
+ TestLEB128Encode(128U, std::array<uint8_t, 1>{}, 0);
+ // Error case: Buffer too small for uint32_t max (needs 5 bytes but only 4 provided)
+ TestLEB128Encode(4294967295U, std::array<uint8_t, 4>{}, 0);
+ // Error case: Zero buffer size
+ TestLEB128Encode(52U, std::array<uint8_t, 0>{}, 0);
+}
+
+/// Utility function to test LEB128 decoding with known byte array and expected result
+template <typename Int, std::size_t N>
+void TestLEB128Decode(std::array<uint8_t, N> const& data, Int expected_value,
+ int32_t expected_bytes_read) {
+ Int result = 0;
+ auto bytes_read = bit_util::ParseLeadingLEB128(
+ data.data(), static_cast<int32_t>(data.size()), &result);
+ EXPECT_EQ(bytes_read, expected_bytes_read);
+ if (expected_bytes_read > 0) {
+ EXPECT_EQ(result, expected_value);
+ }
+}
+
+/// Test decoding from known LEB128 byte sequences
+TEST(BitStreamUtil, LEB128) {
+ // Single byte value 0
+ TestLEB128Decode(std::array<uint8_t, 1>{0x00}, 0U, 1);
+ // Single byte value 127
+ TestLEB128Decode(std::array<uint8_t, 1>{0x7F}, 127U, 1);
+ // Two byte value 128
+ TestLEB128Decode(std::array<uint8_t, 2>{0x80, 0x01}, 128U, 2);
+ // Two byte value 300
+ TestLEB128Decode(std::array<uint8_t, 2>{0xAC, 0x02}, 300U, 2);
+ // Three byte value 16384
+ TestLEB128Decode(std::array<uint8_t, 3>{0x80, 0x80, 0x01}, 16384U, 3);
+ // Three byte value 16384, with remaining data
+ TestLEB128Decode(std::array<uint8_t, 5>{0x80, 0x80, 0x01, 0x80, 0x00}, 16384U, 3);
+ // Four byte value 268435455
+ TestLEB128Decode(std::array<uint8_t, 4>{0xFF, 0xFF, 0xFF, 0x7F}, 268435455U, 4);
+ // Five byte uint32_t max value
+ TestLEB128Decode(std::array<uint8_t, 5>{0xFF, 0xFF, 0xFF, 0xFF, 0x0F}, 4294967295U, 5);
+ // uint64_t value requiring 10 bytes
+ TestLEB128Decode(
+ std::array<uint8_t, 10>{0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x01},
+ 18446744073709551615ULL, 10);
+ // int32_t with maximum size (31 bits of 1)
+ TestLEB128Decode(std::array<uint8_t, 5>{0xFF, 0xFF, 0xFF, 0xFF, 0x7},
+ std::numeric_limits<int32_t>::max(), 5);
+
+ // Error case: Truncated sequence (continuation bit set but no more data)
+ TestLEB128Decode(std::array<uint8_t, 1>{0x80}, 0U, 0);
+ // Error case: Input over the maximum number of bytes for a int32_t (5), but the
+ // overflow none the less (7 * 5 = 35 bits of data).
+ TestLEB128Decode(std::array<uint8_t, 5>{0xFF, 0xFF, 0xFF, 0xFF, 0x7F}, int32_t{}, 0);
Review Comment:
These tests are already here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2333879793
##########
cpp/src/arrow/util/bit_stream_utils_internal.h:
##########
@@ -483,13 +496,17 @@ inline bool BitReader::GetZigZagVlqInt(int32_t* v) {
}
inline bool BitWriter::PutVlqInt(uint64_t v) {
- bool result = true;
- while ((v & 0xFFFFFFFFFFFFFF80ULL) != 0ULL) {
- result &= PutAligned<uint8_t>(static_cast<uint8_t>((v & 0x7F) | 0x80), 1);
- v >>= 7;
+ constexpr auto kMaxBytes = bit_util::MaxLEB128ByteLenFor<decltype(v)>;
Review Comment:
That's member of the reader that was kept for compatibility.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2337172234
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ ARROW_DCHECK_LT(values_read, batch_size);
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+ auto const read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
return values_read;
}
-template <typename T, typename RunType, typename Converter>
-inline int RleDecoder::GetSpaced(Converter converter, int batch_size, int null_count,
- const uint8_t* valid_bits, int64_t valid_bits_offset,
- T* out) {
- if (ARROW_PREDICT_FALSE(null_count == batch_size)) {
- converter.FillZero(out, out + batch_size);
- return batch_size;
+namespace internal {
+
+/// Utility class to safely handle values and null count without too error-prone
+/// verbosity.
+class BatchCounter {
+ public:
+ using size_type = int32_t;
+
+ [[nodiscard]] static constexpr BatchCounter FromBatchSizeAndNulls(
+ size_type batch_size, size_type null_count) {
+ ARROW_DCHECK_LE(null_count, batch_size);
+ return {batch_size - null_count, null_count};
}
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
- int values_remaining = batch_size - null_count;
+ constexpr BatchCounter(size_type values_count, size_type null_count) noexcept
+ : values_count_(values_count), null_count_(null_count) {}
- // Assume no bits to start.
- arrow::internal::BitRunReader bit_reader(valid_bits, valid_bits_offset,
- /*length=*/batch_size);
- arrow::internal::BitRun valid_run = bit_reader.NextRun();
- while (values_read < batch_size) {
- if (ARROW_PREDICT_FALSE(valid_run.length == 0)) {
- valid_run = bit_reader.NextRun();
+ [[nodiscard]] constexpr size_type ValuesCount() const noexcept { return values_count_; }
+
+ [[nodiscard]] constexpr size_type ValuesRead() const noexcept { return values_read_; }
+
+ [[nodiscard]] constexpr size_type ValuesRemaining() const noexcept {
+ ARROW_DCHECK_LE(values_read_, values_count_);
+ return values_count_ - values_read_;
+ }
+
+ constexpr void AccrueReadValues(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, ValuesRemaining());
+ values_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type NullCount() const noexcept { return null_count_; }
+
+ [[nodiscard]] constexpr size_type NullRead() const noexcept { return null_read_; }
+
+ [[nodiscard]] constexpr size_type NullRemaining() const noexcept {
+ ARROW_DCHECK_LE(null_read_, null_count_);
+ return null_count_ - null_read_;
+ }
+
+ constexpr void AccrueReadNulls(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, NullRemaining());
+ null_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type TotalRemaining() const noexcept {
+ return ValuesRemaining() + NullRemaining();
+ }
+
+ [[nodiscard]] constexpr size_type TotalRead() const noexcept {
+ return values_read_ + null_read_;
+ }
+
+ [[nodiscard]] constexpr bool IsFullyNull() const noexcept {
+ return ValuesRemaining() == 0;
+ }
+
+ [[nodiscard]] constexpr bool IsDone() const noexcept { return TotalRemaining() == 0; }
+
+ private:
+ size_type values_count_ = 0;
+ size_type values_read_ = 0;
+ size_type null_count_ = 0;
+ size_type null_read_ = 0;
+};
+
+// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+/// Overload for GetSpaced for a single run in a RleDecoder
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ RleDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto values_remaining_run = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // Consume as much as possible from the repeated run.
+ // We only need to count the number of nulls and non-nulls because we can fill in the
+ // same value for nulls and non-nulls.
+ // This proves to be a big efficiency win.
+ while (values_remaining_run() > 0 && !batch.IsDone()) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const& validity_run_size = static_cast<values_count_type>(validity_run.length);
+
+ if (validity_run.set) {
+ // We may end the current RLE run in the middle of the validity run
+ auto update_size = std::min(validity_run_size, values_remaining_run());
+ batch.AccrueReadValues(update_size);
+ validity_run.length -= update_size;
+ } else {
+ // We can consume all nulls here because it does not matter if we consume on this
+ // RLE run, or an a next encoded run. The value filled does not matter.
+ auto update_size = std::min(validity_run_size, batch.NullRemaining());
+ batch.AccrueReadNulls(update_size);
+ validity_run.length -= update_size;
+ }
+
+ if (ARROW_PREDICT_TRUE(validity_run.length == 0)) {
+ validity_run = validity_reader.NextRun();
}
+ }
+
+ value_type const value = decoder.Value();
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(value))) {
+ return {0, 0};
+ }
+ converter.WriteRepeated(out, out + batch.TotalRead(), value);
+ auto const actual_values_read = decoder.Advance(batch.ValuesRead());
+ // We always cropped the number of values_read by the remaining values in the run.
+ // What's more the RLE decoder should not encounter any errors.
+ ARROW_DCHECK_EQ(actual_values_read, batch.ValuesRead());
- ARROW_DCHECK_GT(batch_size, 0);
- ARROW_DCHECK_GT(valid_run.length, 0);
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+template <typename T, typename... Ts>
+[[nodiscard]] constexpr T min(T x, Ts... ys) {
+ ((x = std::min(x, ys)), ...);
+ return x;
+}
+
+static_assert(min(5) == 5);
+static_assert(min(5, 4, -1) == -1);
+static_assert(min(5, 41) == 5);
+
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ BitPackedDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto run_values_remaining = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ while (run_values_remaining() > 0 && batch.ValuesRemaining() > 0) {
+ // TODO should this size be tune depending on sizeof(value_size)? cpu cache size?
+ // Pull a batch of values from the bit packed encoded data and store it in a local
+ // buffer to benefit from unpacking intrinsics and data locality.
+ static constexpr values_count_type kBufferCapacity = 1024;
+ std::array<value_type, kBufferCapacity> buffer = {};
+
+ values_count_type buffer_start = 0;
+ values_count_type buffer_end = 0;
+ auto buffer_size = [&]() {
+ auto out = buffer_end - buffer_start;
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // buffer_start is 0 at this point so size is end
+ buffer_end = min(run_values_remaining(), batch.ValuesRemaining(), kBufferCapacity);
+ buffer_end = decoder.GetBatch(buffer.data(), buffer_size());
+ ARROW_DCHECK_LE(buffer_size(), kBufferCapacity);
+
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(buffer.data(), buffer_size()))) {
+ return {batch.ValuesRead(), batch.NullRead()};
+ }
- if (valid_run.set) {
- if ((repeat_count_ == 0) && (literal_count_ == 0)) {
- if (!NextCounts<RunType>()) return values_read;
- ARROW_DCHECK((repeat_count_ > 0) ^ (literal_count_ > 0));
+ // Copy chunks of valid values into the output, while adjusting spacing for null
+ // values.
+ while (buffer_size() > 0) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const validity_run_length =
+ static_cast<values_count_type>(validity_run.length);
+
+ // Copy as much as possible from the buffer into the output while not exceeding
+ // validity run
+ if (validity_run.set) {
+ auto const update_size = std::min(validity_run_length, buffer_size());
+ converter.WriteRange(out, buffer.data() + buffer_start, update_size);
+ buffer_start += update_size;
+ batch.AccrueReadValues(update_size);
+ out += update_size;
+ validity_run.length -= update_size;
+ // Simply write zeros in the output
+ } else {
+ auto const update_size = std::min(validity_run_length, batch.NullRemaining());
+ converter.WriteZero(out, out + update_size);
+ batch.AccrueReadNulls(update_size);
+ out += update_size;
+ validity_run.length -= update_size;
}
- if (repeat_count_ > 0) {
- int repeat_batch = 0;
- // Consume the entire repeat counts incrementing repeat_batch to
- // be the total of nulls + values consumed, we only need to
- // get the total count because we can fill in the same value for
- // nulls and non-nulls. This proves to be a big efficiency win.
- while (repeat_count_ > 0 && (values_read + repeat_batch) < batch_size) {
- ARROW_DCHECK_GT(valid_run.length, 0);
- if (valid_run.set) {
- int update_size = std::min(static_cast<int>(valid_run.length), repeat_count_);
- repeat_count_ -= update_size;
- repeat_batch += update_size;
- valid_run.length -= update_size;
- values_remaining -= update_size;
- } else {
- // We can consume all nulls here because we would do so on
- // the next loop anyways.
- repeat_batch += static_cast<int>(valid_run.length);
- valid_run.length = 0;
- }
- if (valid_run.length == 0) {
- valid_run = bit_reader.NextRun();
- }
- }
- RunType current_value = static_cast<RunType>(current_value_);
- if (ARROW_PREDICT_FALSE(!converter.IsValid(current_value))) {
- return values_read;
- }
- converter.Fill(out, out + repeat_batch, current_value);
- out += repeat_batch;
- values_read += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(values_remaining, literal_count_);
- ARROW_DCHECK_GT(literal_batch, 0);
-
- // Decode the literals
- constexpr int kBufferSize = 1024;
- RunType indices[kBufferSize];
- literal_batch = std::min(literal_batch, kBufferSize);
- int actual_read = bit_reader_.GetBatch(bit_width_, indices, literal_batch);
- if (ARROW_PREDICT_FALSE(actual_read != literal_batch)) {
- return values_read;
- }
- if (!converter.IsValid(indices, /*length=*/actual_read)) {
- return values_read;
- }
- int skipped = 0;
- int literals_read = 0;
- while (literals_read < literal_batch) {
- if (valid_run.set) {
- int update_size = std::min(literal_batch - literals_read,
- static_cast<int>(valid_run.length));
- converter.Copy(out, indices + literals_read, update_size);
- literals_read += update_size;
- out += update_size;
- valid_run.length -= update_size;
- } else {
- converter.FillZero(out, out + valid_run.length);
- out += valid_run.length;
- skipped += static_cast<int>(valid_run.length);
- valid_run.length = 0;
- }
- if (valid_run.length == 0) {
- valid_run = bit_reader.NextRun();
- }
- }
- literal_count_ -= literal_batch;
- values_remaining -= literal_batch;
- values_read += literal_batch + skipped;
+ if (validity_run.length == 0) {
+ validity_run = validity_reader.NextRun();
}
- } else {
- converter.FillZero(out, out + valid_run.length);
- out += valid_run.length;
- values_read += static_cast<int>(valid_run.length);
- valid_run.length = 0;
}
+
+ ARROW_DCHECK_EQ(buffer_size(), 0);
}
- ARROW_DCHECK_EQ(valid_run.length, 0);
- ARROW_DCHECK_EQ(values_remaining, 0);
- return values_read;
+
+ ARROW_DCHECK_EQ(values_available - decoder.Remaining(), batch.ValuesRead());
+ ARROW_DCHECK_LE(batch.TotalRead(), batch_size);
Review Comment:
The first assertion states "as many values pulled from decoder as actually read", that seems ok to to me.
The second assertion states "fewer values read than asked for" which is part of the contract.
Perhaps `batch` is not the best name, it's not the input but a counter for this run.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2355146603
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -602,20 +1038,126 @@ struct GetBatchSpacedTestCase {
int bit_width;
};
-TEST(RleDecoder, GetBatchSpaced) {
- uint32_t kSeed = 1337;
- ::arrow::random::RandomArrayGenerator rand(kSeed);
-
- std::vector<GetBatchSpacedTestCase<int32_t>> int32_cases{
- {1, 100000, 0.01, 1}, {1, 100000, 0.1, 1}, {1, 100000, 0.5, 1},
- {4, 100000, 0.05, 3}, {100, 100000, 0.05, 7},
+template <typename T>
+void DoTestGetBatchSpacedRoundtrip() {
+ using Data = DataTestRleBitPacked<T>;
+ using ArrowType = typename Data::ArrowType;
+ using RandomPart = typename Data::RandomPart;
+ using NullPart = typename Data::NullPart;
+ using RepeatPart = typename Data::RepeatPart;
+
+ std::vector<Data> test_cases = {
+ {
+ {RandomPart{/* max=*/1, /* size=*/400, /* null_proba= */ 0.1}},
+ /* bit_width= */ 1,
+ },
+ {
+ {
+ RandomPart{/* max=*/7, /* size=*/10037, /* null_proba= */ 0.0},
+ NullPart{/* size= */ 1153},
+ RandomPart{/* max=*/7, /* size=*/800, /* null_proba= */ 0.5},
+ },
+ /* bit_width= */ 3,
+ },
+ {
+ {
+ NullPart{/* size= */ 80},
+ RandomPart{/* max=*/static_cast<T>(1023), /* size=*/800,
+ /* null_proba= */ 0.01},
+ NullPart{/* size= */ 1023},
+ },
+ /* bit_width= */ 11,
+ },
+ {
+ {RepeatPart{/* value=*/13, /* size=*/100000, /* null_proba= */ 0.01}},
+ /* bit_width= */ 10,
+ },
+ {
+ {
+ NullPart{/* size= */ 1024},
+ RepeatPart{/* value=*/static_cast<T>(10000), /* size=*/100000,
+ /* null_proba= */ 0.1},
+ NullPart{/* size= */ 77},
+ },
+ /* bit_width= */ 23,
+ },
+ {
+ {
+ RepeatPart{/* value=*/13, /* size=*/100000, /* null_proba= */ 0.0},
+ NullPart{/* size= */ 1153},
+ RepeatPart{/* value=*/72, /* size=*/100799, /* null_proba= */ 0.5},
+ },
+ /* bit_width= */ 10,
+ },
+ {
+ {
+ RandomPart{/* max=*/1, /* size=*/1013, /* null_proba= */ 0.01},
+ NullPart{/* size=*/8},
+ RepeatPart{1, /* size= */ 256, /* null_proba= */ 0.1},
+ NullPart{/* size=*/128},
+ RepeatPart{0, /* size= */ 256, /* null_proba= */ 0.0},
+ NullPart{/* size=*/15},
+ RandomPart{/* max=*/1, /* size=*/8 * 1024, /* null_proba= */ 0.01},
+ },
+ /* bit_width= */ 1,
+ },
};
- for (auto case_ : int32_cases) {
- auto arr = rand.Int32(case_.size, /*min=*/0, case_.max_value, case_.null_probability);
- CheckRoundTripSpaced<Int32Type>(*arr, case_.bit_width);
- CheckRoundTripSpaced<Int32Type>(*arr->Slice(1), case_.bit_width);
+
+ ::arrow::random::RandomArrayGenerator rand(/* seed= */ 12);
+ // FRAGILE: we create a dictionary large enough so that any encoded value from the
+ // previous test cases can be used as an index in the dictionary.
+ // Its size must be increased accordingly if larger values are encoded in the test
+ // cases.
+ auto dict = std::static_pointer_cast<arrow::FloatArray>(rand.Float32(20000, -1.0, 1.0));
+
+ // Number of bits available in T to write a positive integer.
+ constexpr int kBitsAvailable = 8 * sizeof(T) - (std::is_signed_v<T> ? 1 : 0);
Review Comment:
I mean that we always seem to test with the largest possible bit width, am I mistaken?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "conbench-apache-arrow[bot] (via GitHub)" <gi...@apache.org>.
conbench-apache-arrow[bot] commented on PR #47294:
URL: https://github.com/apache/arrow/pull/47294#issuecomment-3307744708
Thanks for your patience. Conbench analyzed the 4 benchmarking runs that have been run so far on PR commit c6a6c2140003c3a56ffde37d561e98a8103f5333.
There were 13 benchmark results indicating a performance regression:
- Pull Request Run on `amd64-c6a-4xlarge-linux` at [2025-09-18 08:48:18Z](https://conbench.ursa.dev/compare/runs/2f4d0583f0dc4872b22bd434e63b811a...f36456aae229413bad0c4e601c195db7/)
- [`BM_RleDecoding` (C++) with params=65536/16, source=cpp-micro, suite=parquet-column-io-benchmark](https://conbench.ursa.dev/compare/benchmarks/068ca96e80557043800083b7a88f2776...068cbc8096af76868000c5cbbda7d0f8)
- [`BM_RleDecoding` (C++) with params=32768/16, source=cpp-micro, suite=parquet-column-io-benchmark](https://conbench.ursa.dev/compare/benchmarks/068ca96e7d9e756f800093f69503363a...068cbc8093f773438000c39cd3e845c3)
- and 11 more (see the report linked below)
The [full Conbench report](https://github.com/apache/arrow/runs/50698442233) has more details.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2319457668
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ ARROW_DCHECK_LT(values_read, batch_size);
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+ auto const read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
return values_read;
}
-template <typename T, typename RunType, typename Converter>
-inline int RleDecoder::GetSpaced(Converter converter, int batch_size, int null_count,
- const uint8_t* valid_bits, int64_t valid_bits_offset,
- T* out) {
- if (ARROW_PREDICT_FALSE(null_count == batch_size)) {
- converter.FillZero(out, out + batch_size);
- return batch_size;
+namespace internal {
+
+/// Utility class to safely handle values and null count without too error-prone
+/// verbosity.
+class BatchCounter {
+ public:
+ using size_type = int32_t;
+
+ [[nodiscard]] static constexpr BatchCounter FromBatchSizeAndNulls(
+ size_type batch_size, size_type null_count) {
+ ARROW_DCHECK_LE(null_count, batch_size);
+ return {batch_size - null_count, null_count};
}
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
- int values_remaining = batch_size - null_count;
+ constexpr BatchCounter(size_type values_count, size_type null_count) noexcept
+ : values_count_(values_count), null_count_(null_count) {}
- // Assume no bits to start.
- arrow::internal::BitRunReader bit_reader(valid_bits, valid_bits_offset,
- /*length=*/batch_size);
- arrow::internal::BitRun valid_run = bit_reader.NextRun();
- while (values_read < batch_size) {
- if (ARROW_PREDICT_FALSE(valid_run.length == 0)) {
- valid_run = bit_reader.NextRun();
+ [[nodiscard]] constexpr size_type ValuesCount() const noexcept { return values_count_; }
+
+ [[nodiscard]] constexpr size_type ValuesRead() const noexcept { return values_read_; }
+
+ [[nodiscard]] constexpr size_type ValuesRemaining() const noexcept {
+ ARROW_DCHECK_LE(values_read_, values_count_);
+ return values_count_ - values_read_;
+ }
+
+ constexpr void AccrueReadValues(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, ValuesRemaining());
+ values_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type NullCount() const noexcept { return null_count_; }
+
+ [[nodiscard]] constexpr size_type NullRead() const noexcept { return null_read_; }
+
+ [[nodiscard]] constexpr size_type NullRemaining() const noexcept {
+ ARROW_DCHECK_LE(null_read_, null_count_);
+ return null_count_ - null_read_;
+ }
+
+ constexpr void AccrueReadNulls(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, NullRemaining());
+ null_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type TotalRemaining() const noexcept {
+ return ValuesRemaining() + NullRemaining();
+ }
+
+ [[nodiscard]] constexpr size_type TotalRead() const noexcept {
+ return values_read_ + null_read_;
+ }
+
+ [[nodiscard]] constexpr bool IsFullyNull() const noexcept {
+ return ValuesRemaining() == 0;
+ }
+
+ [[nodiscard]] constexpr bool IsDone() const noexcept { return TotalRemaining() == 0; }
+
+ private:
+ size_type values_count_ = 0;
+ size_type values_read_ = 0;
+ size_type null_count_ = 0;
+ size_type null_read_ = 0;
+};
+
+// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+/// Overload for GetSpaced for a single run in a RleDecoder
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ RleDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
Review Comment:
An opaque `std::pair` isn't very nice for the reader, especially when both members have the same type, perhaps make it a simple `struct`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2322157001
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -553,47 +853,178 @@ TEST(BitRle, Overflow) {
}
}
-template <typename Type>
-void CheckRoundTripSpaced(const Array& data, int bit_width) {
+/// Check RleBitPacked encoding/decoding round trip.
+///
+/// \tparam kSpaced If set to false, treat Nulls in the input array as regular data.
+/// \tparam kParts The number of parts in which the data will be decoded.
+/// For number greater than one, this ensure that the decoder intermediary state
+/// is valid.
+template <typename Type, bool kSpaced, int32_t kParts>
+void CheckRoundTrip(const Array& data, int bit_width,
+ std::shared_ptr<FloatArray> dict = {}) {
using ArrayType = typename TypeTraits<Type>::ArrayType;
- using T = typename Type::c_type;
+ using value_type = typename Type::c_type;
- int num_values = static_cast<int>(data.length());
- int buffer_size = RleEncoder::MaxBufferSize(bit_width, num_values);
+ int const data_size = static_cast<int>(data.length());
+ int const data_values_count =
+ static_cast<int>(data.length() - kSpaced * data.null_count());
+ int const buffer_size = RleBitPackedEncoder::MaxBufferSize(bit_width, data_size);
+ ASSERT_GE(kParts, 1);
+ ASSERT_LE(kParts, data_size);
- const T* values = static_cast<const ArrayType&>(data).raw_values();
+ const value_type* data_values = static_cast<const ArrayType&>(data).raw_values();
+ // Encode the data into ``buffer`` using the encoder.
std::vector<uint8_t> buffer(buffer_size);
- RleEncoder encoder(buffer.data(), buffer_size, bit_width);
- for (int i = 0; i < num_values; ++i) {
- if (data.IsValid(i)) {
- if (!encoder.Put(static_cast<uint64_t>(values[i]))) {
- FAIL() << "Encoding failed";
- }
+ RleBitPackedEncoder encoder(buffer.data(), buffer_size, bit_width);
+ int32_t encoded_values_size = 0;
+ for (int i = 0; i < data_size; ++i) {
+ // Depending on kSpaced we treat nulls as regular values.
+ if (data.IsValid(i) || !kSpaced) {
+ bool success = encoder.Put(static_cast<uint64_t>(data_values[i]));
+ ASSERT_TRUE(success) << "Encoding failed in pos " << i;
+ ++encoded_values_size;
}
}
- int encoded_size = encoder.Flush();
+ int encoded_byte_size = encoder.Flush();
+ ASSERT_EQ(encoded_values_size, data_values_count)
+ << "All values input were not encoded successfully by the encoder";
+
+ // On to verify batch read
+ RleBitPackedDecoder<value_type> decoder(buffer.data(), encoded_byte_size, bit_width);
+ // We will only use one of them depending on whether this is a dictonnary tests
+ std::vector<float> dict_read;
+ std::vector<value_type> values_read;
+ if (dict) {
+ dict_read.resize(data_size);
+ } else {
+ values_read.resize(data_size);
+ }
- // Verify batch read
- RleDecoder decoder(buffer.data(), encoded_size, bit_width);
- std::vector<T> values_read(num_values);
+ // We will read the data in kParts calls to make sure intermediate states are valid
+ int32_t actual_read_count = 0;
+ int32_t requested_read_count = 0;
+ while (requested_read_count < data_size) {
+ auto const remaining = data_size - requested_read_count;
+ auto to_read = data_size / kParts;
+ if (remaining / to_read == 1) {
+ to_read = remaining;
+ }
- if (num_values != decoder.GetBatchSpaced(
- num_values, static_cast<int>(data.null_count()),
- data.null_bitmap_data(), data.offset(), values_read.data())) {
- FAIL();
- }
+ auto read = 0;
+ if constexpr (kSpaced) {
+ // We need to slice the input array get the proper null count and bitmap
+ auto data_remaining = data.Slice(requested_read_count, to_read);
+
+ if (dict) {
+ auto* out = dict_read.data() + requested_read_count;
+ read = decoder.GetBatchWithDictSpaced(
+ dict->raw_values(), static_cast<int32_t>(dict->length()), out, to_read,
+ static_cast<int32_t>(data_remaining->null_count()),
+ data_remaining->null_bitmap_data(), data_remaining->offset());
+ } else {
+ auto* out = values_read.data() + requested_read_count;
+ read = decoder.GetBatchSpaced(
+ to_read, static_cast<int32_t>(data_remaining->null_count()),
+ data_remaining->null_bitmap_data(), data_remaining->offset(), out);
+ }
+ } else {
+ if (dict) {
+ auto* out = dict_read.data() + requested_read_count;
+ read = decoder.GetBatchWithDict(
+ dict->raw_values(), static_cast<int32_t>(dict->length()), out, to_read);
+ } else {
+ auto* out = values_read.data() + requested_read_count;
+ read = decoder.GetBatch(out, to_read);
+ }
+ }
+ ASSERT_EQ(read, to_read) << "Decoder did not read as many values as requested";
- for (int64_t i = 0; i < num_values; ++i) {
- if (data.IsValid(i)) {
- if (values_read[i] != values[i]) {
- FAIL() << "Index " << i << " read " << values_read[i] << " but should be "
- << values[i];
+ actual_read_count += read;
+ requested_read_count += to_read;
+ }
+ EXPECT_EQ(requested_read_count, data_size) << "This test logic is wrong";
+ EXPECT_EQ(actual_read_count, data_size) << "Total number of values read is off";
+
+ // Verify the round trip: encoded-decoded values must equal the original one
+ for (int64_t i = 0; i < data_size; ++i) {
+ if (data.IsValid(i) || !kSpaced) {
+ if (dict) {
+ EXPECT_EQ(dict_read.at(i), dict->Value(data_values[i]))
+ << "Encoded then decoded and mapped value at position " << i << " ("
+ << values_read[i] << ") differs from original value (" << data_values[i]
+ << " mapped to " << dict->Value(data_values[i]) << ")";
+ } else {
+ EXPECT_EQ(values_read.at(i), data_values[i])
+ << "Encoded then decoded value at position " << i << " (" << values_read.at(i)
+ << ") differs from original value (" << data_values[i] << ")";
}
}
}
}
+template <typename T>
+struct DataTestRleBitPackedRandomPart {
+ using value_type = T;
+
+ value_type max;
+ int32_t size;
+ double null_probability;
+};
+
+template <typename T>
+struct DataTestRleBitPackedRepeatPart {
+ using value_type = T;
+
+ value_type value;
+ int32_t size;
+};
+
+template <typename T>
+struct DataTestRleBitPackedNullPart {
+ using value_type = T;
+
+ int32_t size;
+};
+
+template <typename T>
+struct DataTestRleBitPacked {
+ using value_type = T;
+ using ArrowType = typename arrow::CTypeTraits<value_type>::ArrowType;
+ using RandomPart = DataTestRleBitPackedRandomPart<value_type>;
+ using RepeatPart = DataTestRleBitPackedRepeatPart<value_type>;
+ using NullPart = DataTestRleBitPackedNullPart<value_type>;
+
+ std::vector<std::variant<RandomPart, RepeatPart, NullPart>> parts;
+ int32_t bit_width;
+
+ std::shared_ptr<::arrow::Array> MakeArray(
+ ::arrow::random::RandomArrayGenerator& rand) const {
+ std::vector<std::shared_ptr<::arrow::Array>> arrays = {};
+
+ for (auto const& dyn_part : parts) {
+ if (auto* part = std::get_if<RandomPart>(&dyn_part)) {
+ auto arr = rand.Numeric<ArrowType>(part->size, /* min= */ value_type(0),
+ part->max, part->null_probability);
Review Comment:
It is weird to have a null probability here but not for repeated parts. Is there a reason for this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2319494732
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -118,50 +365,56 @@ class RleDecoder {
/// input with zeros. Since the encoding does not differentiate between
/// input values and padding, Get() returns true even for these padding
/// values.
- template <typename T>
- bool Get(T* val);
+ [[nodiscard]] bool Get(value_type* val);
Review Comment:
I'm curious, is this method actually used somewhere? It sounds unexpected to decode data one value at a time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2319542322
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ ARROW_DCHECK_LT(values_read, batch_size);
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+ auto const read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
return values_read;
}
-template <typename T, typename RunType, typename Converter>
-inline int RleDecoder::GetSpaced(Converter converter, int batch_size, int null_count,
- const uint8_t* valid_bits, int64_t valid_bits_offset,
- T* out) {
- if (ARROW_PREDICT_FALSE(null_count == batch_size)) {
- converter.FillZero(out, out + batch_size);
- return batch_size;
+namespace internal {
+
+/// Utility class to safely handle values and null count without too error-prone
+/// verbosity.
+class BatchCounter {
+ public:
+ using size_type = int32_t;
+
+ [[nodiscard]] static constexpr BatchCounter FromBatchSizeAndNulls(
+ size_type batch_size, size_type null_count) {
+ ARROW_DCHECK_LE(null_count, batch_size);
+ return {batch_size - null_count, null_count};
}
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
- int values_remaining = batch_size - null_count;
+ constexpr BatchCounter(size_type values_count, size_type null_count) noexcept
+ : values_count_(values_count), null_count_(null_count) {}
- // Assume no bits to start.
- arrow::internal::BitRunReader bit_reader(valid_bits, valid_bits_offset,
- /*length=*/batch_size);
- arrow::internal::BitRun valid_run = bit_reader.NextRun();
- while (values_read < batch_size) {
- if (ARROW_PREDICT_FALSE(valid_run.length == 0)) {
- valid_run = bit_reader.NextRun();
+ [[nodiscard]] constexpr size_type ValuesCount() const noexcept { return values_count_; }
+
+ [[nodiscard]] constexpr size_type ValuesRead() const noexcept { return values_read_; }
+
+ [[nodiscard]] constexpr size_type ValuesRemaining() const noexcept {
+ ARROW_DCHECK_LE(values_read_, values_count_);
+ return values_count_ - values_read_;
+ }
+
+ constexpr void AccrueReadValues(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, ValuesRemaining());
+ values_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type NullCount() const noexcept { return null_count_; }
+
+ [[nodiscard]] constexpr size_type NullRead() const noexcept { return null_read_; }
+
+ [[nodiscard]] constexpr size_type NullRemaining() const noexcept {
+ ARROW_DCHECK_LE(null_read_, null_count_);
+ return null_count_ - null_read_;
+ }
+
+ constexpr void AccrueReadNulls(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, NullRemaining());
+ null_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type TotalRemaining() const noexcept {
+ return ValuesRemaining() + NullRemaining();
+ }
+
+ [[nodiscard]] constexpr size_type TotalRead() const noexcept {
+ return values_read_ + null_read_;
+ }
+
+ [[nodiscard]] constexpr bool IsFullyNull() const noexcept {
+ return ValuesRemaining() == 0;
+ }
+
+ [[nodiscard]] constexpr bool IsDone() const noexcept { return TotalRemaining() == 0; }
+
+ private:
+ size_type values_count_ = 0;
+ size_type values_read_ = 0;
+ size_type null_count_ = 0;
+ size_type null_read_ = 0;
+};
+
+// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+/// Overload for GetSpaced for a single run in a RleDecoder
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ RleDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto values_remaining_run = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // Consume as much as possible from the repeated run.
+ // We only need to count the number of nulls and non-nulls because we can fill in the
+ // same value for nulls and non-nulls.
+ // This proves to be a big efficiency win.
+ while (values_remaining_run() > 0 && !batch.IsDone()) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const& validity_run_size = static_cast<values_count_type>(validity_run.length);
+
+ if (validity_run.set) {
+ // We may end the current RLE run in the middle of the validity run
+ auto update_size = std::min(validity_run_size, values_remaining_run());
+ batch.AccrueReadValues(update_size);
+ validity_run.length -= update_size;
+ } else {
+ // We can consume all nulls here because it does not matter if we consume on this
+ // RLE run, or an a next encoded run. The value filled does not matter.
+ auto update_size = std::min(validity_run_size, batch.NullRemaining());
+ batch.AccrueReadNulls(update_size);
+ validity_run.length -= update_size;
+ }
+
+ if (ARROW_PREDICT_TRUE(validity_run.length == 0)) {
+ validity_run = validity_reader.NextRun();
}
+ }
+
+ value_type const value = decoder.Value();
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(value))) {
+ return {0, 0};
+ }
+ converter.WriteRepeated(out, out + batch.TotalRead(), value);
+ auto const actual_values_read = decoder.Advance(batch.ValuesRead());
+ // We always cropped the number of values_read by the remaining values in the run.
+ // What's more the RLE decoder should not encounter any errors.
+ ARROW_DCHECK_EQ(actual_values_read, batch.ValuesRead());
- ARROW_DCHECK_GT(batch_size, 0);
- ARROW_DCHECK_GT(valid_run.length, 0);
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+template <typename T, typename... Ts>
+[[nodiscard]] constexpr T min(T x, Ts... ys) {
+ ((x = std::min(x, ys)), ...);
+ return x;
+}
+
+static_assert(min(5) == 5);
+static_assert(min(5, 4, -1) == -1);
+static_assert(min(5, 41) == 5);
+
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ BitPackedDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto run_values_remaining = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ while (run_values_remaining() > 0 && batch.ValuesRemaining() > 0) {
+ // TODO should this size be tune depending on sizeof(value_size)? cpu cache size?
+ // Pull a batch of values from the bit packed encoded data and store it in a local
+ // buffer to benefit from unpacking intrinsics and data locality.
+ static constexpr values_count_type kBufferCapacity = 1024;
+ std::array<value_type, kBufferCapacity> buffer = {};
+
+ values_count_type buffer_start = 0;
+ values_count_type buffer_end = 0;
+ auto buffer_size = [&]() {
+ auto out = buffer_end - buffer_start;
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // buffer_start is 0 at this point so size is end
+ buffer_end = min(run_values_remaining(), batch.ValuesRemaining(), kBufferCapacity);
+ buffer_end = decoder.GetBatch(buffer.data(), buffer_size());
+ ARROW_DCHECK_LE(buffer_size(), kBufferCapacity);
+
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(buffer.data(), buffer_size()))) {
+ return {batch.ValuesRead(), batch.NullRead()};
+ }
- if (valid_run.set) {
- if ((repeat_count_ == 0) && (literal_count_ == 0)) {
- if (!NextCounts<RunType>()) return values_read;
- ARROW_DCHECK((repeat_count_ > 0) ^ (literal_count_ > 0));
+ // Copy chunks of valid values into the output, while adjusting spacing for null
+ // values.
+ while (buffer_size() > 0) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const validity_run_length =
+ static_cast<values_count_type>(validity_run.length);
+
+ // Copy as much as possible from the buffer into the output while not exceeding
+ // validity run
+ if (validity_run.set) {
+ auto const update_size = std::min(validity_run_length, buffer_size());
+ converter.WriteRange(out, buffer.data() + buffer_start, update_size);
+ buffer_start += update_size;
+ batch.AccrueReadValues(update_size);
+ out += update_size;
+ validity_run.length -= update_size;
+ // Simply write zeros in the output
+ } else {
+ auto const update_size = std::min(validity_run_length, batch.NullRemaining());
+ converter.WriteZero(out, out + update_size);
+ batch.AccrueReadNulls(update_size);
+ out += update_size;
+ validity_run.length -= update_size;
}
- if (repeat_count_ > 0) {
- int repeat_batch = 0;
- // Consume the entire repeat counts incrementing repeat_batch to
- // be the total of nulls + values consumed, we only need to
- // get the total count because we can fill in the same value for
- // nulls and non-nulls. This proves to be a big efficiency win.
- while (repeat_count_ > 0 && (values_read + repeat_batch) < batch_size) {
- ARROW_DCHECK_GT(valid_run.length, 0);
- if (valid_run.set) {
- int update_size = std::min(static_cast<int>(valid_run.length), repeat_count_);
- repeat_count_ -= update_size;
- repeat_batch += update_size;
- valid_run.length -= update_size;
- values_remaining -= update_size;
- } else {
- // We can consume all nulls here because we would do so on
- // the next loop anyways.
- repeat_batch += static_cast<int>(valid_run.length);
- valid_run.length = 0;
- }
- if (valid_run.length == 0) {
- valid_run = bit_reader.NextRun();
- }
- }
- RunType current_value = static_cast<RunType>(current_value_);
- if (ARROW_PREDICT_FALSE(!converter.IsValid(current_value))) {
- return values_read;
- }
- converter.Fill(out, out + repeat_batch, current_value);
- out += repeat_batch;
- values_read += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(values_remaining, literal_count_);
- ARROW_DCHECK_GT(literal_batch, 0);
-
- // Decode the literals
- constexpr int kBufferSize = 1024;
- RunType indices[kBufferSize];
- literal_batch = std::min(literal_batch, kBufferSize);
- int actual_read = bit_reader_.GetBatch(bit_width_, indices, literal_batch);
- if (ARROW_PREDICT_FALSE(actual_read != literal_batch)) {
- return values_read;
- }
- if (!converter.IsValid(indices, /*length=*/actual_read)) {
- return values_read;
- }
- int skipped = 0;
- int literals_read = 0;
- while (literals_read < literal_batch) {
- if (valid_run.set) {
- int update_size = std::min(literal_batch - literals_read,
- static_cast<int>(valid_run.length));
- converter.Copy(out, indices + literals_read, update_size);
- literals_read += update_size;
- out += update_size;
- valid_run.length -= update_size;
- } else {
- converter.FillZero(out, out + valid_run.length);
- out += valid_run.length;
- skipped += static_cast<int>(valid_run.length);
- valid_run.length = 0;
- }
- if (valid_run.length == 0) {
- valid_run = bit_reader.NextRun();
- }
- }
- literal_count_ -= literal_batch;
- values_remaining -= literal_batch;
- values_read += literal_batch + skipped;
+ if (validity_run.length == 0) {
+ validity_run = validity_reader.NextRun();
}
- } else {
- converter.FillZero(out, out + valid_run.length);
- out += valid_run.length;
- values_read += static_cast<int>(valid_run.length);
- valid_run.length = 0;
}
+
+ ARROW_DCHECK_EQ(buffer_size(), 0);
}
- ARROW_DCHECK_EQ(valid_run.length, 0);
- ARROW_DCHECK_EQ(values_remaining, 0);
- return values_read;
+
+ ARROW_DCHECK_EQ(values_available - decoder.Remaining(), batch.ValuesRead());
+ ARROW_DCHECK_LE(batch.TotalRead(), batch_size);
+ ARROW_DCHECK_LE(batch.NullRead(), batch.NullCount());
+
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+/// Overload for GetSpaced for a single run in a decoder variant
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(
+ Converter& converter, typename Converter::out_type* out, values_count_type batch_size,
+ values_count_type null_count, BitRunReader&& validity_reader, BitRun&& validity_run,
+ std::variant<RleDecoder<value_type>, BitPackedDecoder<value_type>>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ return std::visit(
+ [&](auto& dec) {
+ ARROW_DCHECK_GT(dec.Remaining(), 0);
+ return RunGetSpaced(converter, out, batch_size, null_count, validity_reader,
+ validity_run, dec);
+ },
+ decoder);
}
+} // namespace internal
+
+template <typename T>
+template <typename Converter>
+auto RleBitPackedDecoder<T>::GetSpaced(
+ Converter converter, typename Converter::out_type* out, values_count_type batch_size,
+ const byte* validity_bits, int64_t validity_bits_offset, values_count_type null_count)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
+
+ ARROW_DCHECK_GT(batch_size, 0);
+
+ auto batch = internal::BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ if (ARROW_PREDICT_FALSE(batch.IsFullyNull())) {
+ converter.WriteZero(out, out + batch.NullRemaining());
+ return batch.NullRemaining();
+ }
+
+ arrow::internal::BitRunReader validity_reader(validity_bits, validity_bits_offset,
+ /*length=*/batch.TotalRemaining());
+ arrow::internal::BitRun validity_run = validity_reader.NextRun();
+
+ auto const check_and_handle_fully_null_remaining = [&]() {
+ if (batch.IsFullyNull()) {
+ ARROW_DCHECK(validity_run.length == 0 || !validity_run.set);
+ ARROW_DCHECK_GE(validity_run.length, batch.NullRemaining());
+
+ converter.WriteZero(out, out + batch.NullRemaining());
+ out += batch.NullRemaining();
+ batch.AccrueReadNulls(batch.NullRemaining());
+ }
+ };
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const [values_read, null_read] =
+ RunGetSpaced(converter, out, batch.TotalRemaining(), batch.NullRemaining(),
+ validity_reader, validity_run, decoder_);
+
+ batch.AccrueReadNulls(null_read);
+ batch.AccrueReadValues(values_read);
+ out += values_read + null_read;
+
+ // Either we fulfilled all the batch values to be read
+ if (ARROW_PREDICT_FALSE(batch.ValuesRemaining() == 0)) {
+ // There may be remaining null if they are not greedily filled
+ check_and_handle_fully_null_remaining();
+ return batch.TotalRead();
+ }
+
+ /// We finished the remaining run
Review Comment:
```suggestion
// We finished the remaining run
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2322142141
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -553,47 +853,178 @@ TEST(BitRle, Overflow) {
}
}
-template <typename Type>
-void CheckRoundTripSpaced(const Array& data, int bit_width) {
+/// Check RleBitPacked encoding/decoding round trip.
+///
+/// \tparam kSpaced If set to false, treat Nulls in the input array as regular data.
+/// \tparam kParts The number of parts in which the data will be decoded.
+/// For number greater than one, this ensure that the decoder intermediary state
+/// is valid.
+template <typename Type, bool kSpaced, int32_t kParts>
Review Comment:
Why pass `kSpaced` and `kParts` as template parameters? We should try to minimize compile times in tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2319564938
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ ARROW_DCHECK_LT(values_read, batch_size);
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+ auto const read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
return values_read;
}
-template <typename T, typename RunType, typename Converter>
-inline int RleDecoder::GetSpaced(Converter converter, int batch_size, int null_count,
- const uint8_t* valid_bits, int64_t valid_bits_offset,
- T* out) {
- if (ARROW_PREDICT_FALSE(null_count == batch_size)) {
- converter.FillZero(out, out + batch_size);
- return batch_size;
+namespace internal {
+
+/// Utility class to safely handle values and null count without too error-prone
+/// verbosity.
+class BatchCounter {
+ public:
+ using size_type = int32_t;
+
+ [[nodiscard]] static constexpr BatchCounter FromBatchSizeAndNulls(
+ size_type batch_size, size_type null_count) {
+ ARROW_DCHECK_LE(null_count, batch_size);
+ return {batch_size - null_count, null_count};
}
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
- int values_remaining = batch_size - null_count;
+ constexpr BatchCounter(size_type values_count, size_type null_count) noexcept
+ : values_count_(values_count), null_count_(null_count) {}
- // Assume no bits to start.
- arrow::internal::BitRunReader bit_reader(valid_bits, valid_bits_offset,
- /*length=*/batch_size);
- arrow::internal::BitRun valid_run = bit_reader.NextRun();
- while (values_read < batch_size) {
- if (ARROW_PREDICT_FALSE(valid_run.length == 0)) {
- valid_run = bit_reader.NextRun();
+ [[nodiscard]] constexpr size_type ValuesCount() const noexcept { return values_count_; }
+
+ [[nodiscard]] constexpr size_type ValuesRead() const noexcept { return values_read_; }
+
+ [[nodiscard]] constexpr size_type ValuesRemaining() const noexcept {
+ ARROW_DCHECK_LE(values_read_, values_count_);
+ return values_count_ - values_read_;
+ }
+
+ constexpr void AccrueReadValues(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, ValuesRemaining());
+ values_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type NullCount() const noexcept { return null_count_; }
+
+ [[nodiscard]] constexpr size_type NullRead() const noexcept { return null_read_; }
+
+ [[nodiscard]] constexpr size_type NullRemaining() const noexcept {
+ ARROW_DCHECK_LE(null_read_, null_count_);
+ return null_count_ - null_read_;
+ }
+
+ constexpr void AccrueReadNulls(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, NullRemaining());
+ null_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type TotalRemaining() const noexcept {
+ return ValuesRemaining() + NullRemaining();
+ }
+
+ [[nodiscard]] constexpr size_type TotalRead() const noexcept {
+ return values_read_ + null_read_;
+ }
+
+ [[nodiscard]] constexpr bool IsFullyNull() const noexcept {
+ return ValuesRemaining() == 0;
+ }
+
+ [[nodiscard]] constexpr bool IsDone() const noexcept { return TotalRemaining() == 0; }
+
+ private:
+ size_type values_count_ = 0;
+ size_type values_read_ = 0;
+ size_type null_count_ = 0;
+ size_type null_read_ = 0;
+};
+
+// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+/// Overload for GetSpaced for a single run in a RleDecoder
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ RleDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto values_remaining_run = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // Consume as much as possible from the repeated run.
+ // We only need to count the number of nulls and non-nulls because we can fill in the
+ // same value for nulls and non-nulls.
+ // This proves to be a big efficiency win.
+ while (values_remaining_run() > 0 && !batch.IsDone()) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const& validity_run_size = static_cast<values_count_type>(validity_run.length);
+
+ if (validity_run.set) {
+ // We may end the current RLE run in the middle of the validity run
+ auto update_size = std::min(validity_run_size, values_remaining_run());
+ batch.AccrueReadValues(update_size);
+ validity_run.length -= update_size;
+ } else {
+ // We can consume all nulls here because it does not matter if we consume on this
+ // RLE run, or an a next encoded run. The value filled does not matter.
+ auto update_size = std::min(validity_run_size, batch.NullRemaining());
+ batch.AccrueReadNulls(update_size);
+ validity_run.length -= update_size;
+ }
+
+ if (ARROW_PREDICT_TRUE(validity_run.length == 0)) {
+ validity_run = validity_reader.NextRun();
}
+ }
+
+ value_type const value = decoder.Value();
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(value))) {
+ return {0, 0};
+ }
+ converter.WriteRepeated(out, out + batch.TotalRead(), value);
+ auto const actual_values_read = decoder.Advance(batch.ValuesRead());
+ // We always cropped the number of values_read by the remaining values in the run.
+ // What's more the RLE decoder should not encounter any errors.
+ ARROW_DCHECK_EQ(actual_values_read, batch.ValuesRead());
- ARROW_DCHECK_GT(batch_size, 0);
- ARROW_DCHECK_GT(valid_run.length, 0);
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+template <typename T, typename... Ts>
+[[nodiscard]] constexpr T min(T x, Ts... ys) {
+ ((x = std::min(x, ys)), ...);
+ return x;
+}
+
+static_assert(min(5) == 5);
+static_assert(min(5, 4, -1) == -1);
+static_assert(min(5, 41) == 5);
+
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ BitPackedDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto run_values_remaining = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ while (run_values_remaining() > 0 && batch.ValuesRemaining() > 0) {
+ // TODO should this size be tune depending on sizeof(value_size)? cpu cache size?
+ // Pull a batch of values from the bit packed encoded data and store it in a local
+ // buffer to benefit from unpacking intrinsics and data locality.
+ static constexpr values_count_type kBufferCapacity = 1024;
+ std::array<value_type, kBufferCapacity> buffer = {};
+
+ values_count_type buffer_start = 0;
+ values_count_type buffer_end = 0;
+ auto buffer_size = [&]() {
+ auto out = buffer_end - buffer_start;
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // buffer_start is 0 at this point so size is end
+ buffer_end = min(run_values_remaining(), batch.ValuesRemaining(), kBufferCapacity);
+ buffer_end = decoder.GetBatch(buffer.data(), buffer_size());
+ ARROW_DCHECK_LE(buffer_size(), kBufferCapacity);
+
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(buffer.data(), buffer_size()))) {
+ return {batch.ValuesRead(), batch.NullRead()};
+ }
- if (valid_run.set) {
- if ((repeat_count_ == 0) && (literal_count_ == 0)) {
- if (!NextCounts<RunType>()) return values_read;
- ARROW_DCHECK((repeat_count_ > 0) ^ (literal_count_ > 0));
+ // Copy chunks of valid values into the output, while adjusting spacing for null
+ // values.
+ while (buffer_size() > 0) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const validity_run_length =
+ static_cast<values_count_type>(validity_run.length);
+
+ // Copy as much as possible from the buffer into the output while not exceeding
+ // validity run
+ if (validity_run.set) {
+ auto const update_size = std::min(validity_run_length, buffer_size());
+ converter.WriteRange(out, buffer.data() + buffer_start, update_size);
+ buffer_start += update_size;
+ batch.AccrueReadValues(update_size);
+ out += update_size;
+ validity_run.length -= update_size;
+ // Simply write zeros in the output
+ } else {
+ auto const update_size = std::min(validity_run_length, batch.NullRemaining());
+ converter.WriteZero(out, out + update_size);
+ batch.AccrueReadNulls(update_size);
+ out += update_size;
+ validity_run.length -= update_size;
}
- if (repeat_count_ > 0) {
- int repeat_batch = 0;
- // Consume the entire repeat counts incrementing repeat_batch to
- // be the total of nulls + values consumed, we only need to
- // get the total count because we can fill in the same value for
- // nulls and non-nulls. This proves to be a big efficiency win.
- while (repeat_count_ > 0 && (values_read + repeat_batch) < batch_size) {
- ARROW_DCHECK_GT(valid_run.length, 0);
- if (valid_run.set) {
- int update_size = std::min(static_cast<int>(valid_run.length), repeat_count_);
- repeat_count_ -= update_size;
- repeat_batch += update_size;
- valid_run.length -= update_size;
- values_remaining -= update_size;
- } else {
- // We can consume all nulls here because we would do so on
- // the next loop anyways.
- repeat_batch += static_cast<int>(valid_run.length);
- valid_run.length = 0;
- }
- if (valid_run.length == 0) {
- valid_run = bit_reader.NextRun();
- }
- }
- RunType current_value = static_cast<RunType>(current_value_);
- if (ARROW_PREDICT_FALSE(!converter.IsValid(current_value))) {
- return values_read;
- }
- converter.Fill(out, out + repeat_batch, current_value);
- out += repeat_batch;
- values_read += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(values_remaining, literal_count_);
- ARROW_DCHECK_GT(literal_batch, 0);
-
- // Decode the literals
- constexpr int kBufferSize = 1024;
- RunType indices[kBufferSize];
- literal_batch = std::min(literal_batch, kBufferSize);
- int actual_read = bit_reader_.GetBatch(bit_width_, indices, literal_batch);
- if (ARROW_PREDICT_FALSE(actual_read != literal_batch)) {
- return values_read;
- }
- if (!converter.IsValid(indices, /*length=*/actual_read)) {
- return values_read;
- }
- int skipped = 0;
- int literals_read = 0;
- while (literals_read < literal_batch) {
- if (valid_run.set) {
- int update_size = std::min(literal_batch - literals_read,
- static_cast<int>(valid_run.length));
- converter.Copy(out, indices + literals_read, update_size);
- literals_read += update_size;
- out += update_size;
- valid_run.length -= update_size;
- } else {
- converter.FillZero(out, out + valid_run.length);
- out += valid_run.length;
- skipped += static_cast<int>(valid_run.length);
- valid_run.length = 0;
- }
- if (valid_run.length == 0) {
- valid_run = bit_reader.NextRun();
- }
- }
- literal_count_ -= literal_batch;
- values_remaining -= literal_batch;
- values_read += literal_batch + skipped;
+ if (validity_run.length == 0) {
+ validity_run = validity_reader.NextRun();
}
- } else {
- converter.FillZero(out, out + valid_run.length);
- out += valid_run.length;
- values_read += static_cast<int>(valid_run.length);
- valid_run.length = 0;
}
+
+ ARROW_DCHECK_EQ(buffer_size(), 0);
}
- ARROW_DCHECK_EQ(valid_run.length, 0);
- ARROW_DCHECK_EQ(values_remaining, 0);
- return values_read;
+
+ ARROW_DCHECK_EQ(values_available - decoder.Remaining(), batch.ValuesRead());
+ ARROW_DCHECK_LE(batch.TotalRead(), batch_size);
+ ARROW_DCHECK_LE(batch.NullRead(), batch.NullCount());
+
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+/// Overload for GetSpaced for a single run in a decoder variant
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(
+ Converter& converter, typename Converter::out_type* out, values_count_type batch_size,
+ values_count_type null_count, BitRunReader&& validity_reader, BitRun&& validity_run,
+ std::variant<RleDecoder<value_type>, BitPackedDecoder<value_type>>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ return std::visit(
+ [&](auto& dec) {
+ ARROW_DCHECK_GT(dec.Remaining(), 0);
+ return RunGetSpaced(converter, out, batch_size, null_count, validity_reader,
+ validity_run, dec);
+ },
+ decoder);
}
+} // namespace internal
+
+template <typename T>
+template <typename Converter>
+auto RleBitPackedDecoder<T>::GetSpaced(
+ Converter converter, typename Converter::out_type* out, values_count_type batch_size,
+ const byte* validity_bits, int64_t validity_bits_offset, values_count_type null_count)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
+
+ ARROW_DCHECK_GT(batch_size, 0);
+
+ auto batch = internal::BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ if (ARROW_PREDICT_FALSE(batch.IsFullyNull())) {
+ converter.WriteZero(out, out + batch.NullRemaining());
+ return batch.NullRemaining();
+ }
+
+ arrow::internal::BitRunReader validity_reader(validity_bits, validity_bits_offset,
+ /*length=*/batch.TotalRemaining());
+ arrow::internal::BitRun validity_run = validity_reader.NextRun();
+
+ auto const check_and_handle_fully_null_remaining = [&]() {
+ if (batch.IsFullyNull()) {
+ ARROW_DCHECK(validity_run.length == 0 || !validity_run.set);
+ ARROW_DCHECK_GE(validity_run.length, batch.NullRemaining());
+
+ converter.WriteZero(out, out + batch.NullRemaining());
+ out += batch.NullRemaining();
+ batch.AccrueReadNulls(batch.NullRemaining());
+ }
+ };
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const [values_read, null_read] =
+ RunGetSpaced(converter, out, batch.TotalRemaining(), batch.NullRemaining(),
+ validity_reader, validity_run, decoder_);
+
+ batch.AccrueReadNulls(null_read);
+ batch.AccrueReadValues(values_read);
+ out += values_read + null_read;
+
+ // Either we fulfilled all the batch values to be read
+ if (ARROW_PREDICT_FALSE(batch.ValuesRemaining() == 0)) {
+ // There may be remaining null if they are not greedily filled
+ check_and_handle_fully_null_remaining();
+ return batch.TotalRead();
+ }
+
+ /// We finished the remaining run
+ ARROW_DCHECK(RunRemaining() == 0);
+ }
+
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+
+ auto const [values_read, null_read] = internal::RunGetSpaced(
+ converter, out, batch.TotalRemaining(), batch.NullRemaining(),
+ validity_reader, validity_run, decoder);
+
+ batch.AccrueReadNulls(null_read);
+ batch.AccrueReadValues(values_read);
+ out += values_read + null_read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == 0 || batch.ValuesRemaining() == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
+ // There may be remaining null if they are not greedily filled by either decoder calls
+ check_and_handle_fully_null_remaining();
+
+ ARROW_DCHECK(batch.IsDone() || Exhausted());
+ // batch.Done() => batch.NullRemaining() == 0
+ ARROW_DCHECK(!batch.IsDone() || (batch.NullRemaining() == 0));
+ return batch.TotalRead();
+}
+
+namespace internal {
+
// Converter for GetSpaced that handles runs that get returned
// directly as output.
template <typename T>
-struct PlainRleConverter {
- T kZero = {};
- inline bool IsValid(const T& values) const { return true; }
- inline bool IsValid(const T* values, int32_t length) const { return true; }
- inline void Fill(T* begin, T* end, const T& run_value) const {
+struct NoOpConverter {
+ using in_type = T;
+ using out_type = T;
+ using size_type = int32_t;
+
+ static constexpr bool kIsIdentity = true;
+
+ [[nodiscard]] static constexpr bool InputIsValid(const in_type& values) { return true; }
+
+ [[nodiscard]] static constexpr bool InputIsValid(const in_type* values,
+ size_type length) {
+ return true;
+ }
+
+ static void WriteRepeated(out_type* begin, out_type* end, in_type run_value) {
std::fill(begin, end, run_value);
}
- inline void FillZero(T* begin, T* end) { std::fill(begin, end, kZero); }
- inline void Copy(T* out, const T* values, int length) const {
- std::memcpy(out, values, length * sizeof(T));
+
+ static void WriteZero(out_type* begin, out_type* end) {
+ std::fill(begin, end, out_type{});
+ }
+
+ static void WriteRange(out_type* out, const in_type* values, size_type length) {
+ std::memcpy(out, values, length * sizeof(out_type));
}
};
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatchSpaced(int batch_size, int null_count,
- const uint8_t* valid_bits,
- int64_t valid_bits_offset, T* out) {
+auto RleBitPackedDecoder<T>::GetBatchSpaced(values_count_type batch_size,
+ values_count_type null_count,
+ const byte* valid_bits,
+ int64_t valid_bits_offset, value_type* out)
+ -> values_count_type {
if (null_count == 0) {
- return GetBatch<T>(out, batch_size);
+ return GetBatch(out, batch_size);
}
- PlainRleConverter<T> converter;
- arrow::internal::BitBlockCounter block_counter(valid_bits, valid_bits_offset,
- batch_size);
-
- int total_processed = 0;
- int processed = 0;
- arrow::internal::BitBlockCount block;
+ internal::NoOpConverter<value_type> converter;
- do {
- block = block_counter.NextFourWords();
- if (block.length == 0) {
- break;
- }
- if (block.AllSet()) {
- processed = GetBatch<T>(out, block.length);
- } else if (block.NoneSet()) {
- converter.FillZero(out, out + block.length);
- processed = block.length;
- } else {
- processed = GetSpaced<T, /*RunType=*/T, PlainRleConverter<T>>(
- converter, block.length, block.length - block.popcount, valid_bits,
- valid_bits_offset, out);
- }
- total_processed += processed;
- out += block.length;
- valid_bits_offset += block.length;
- } while (processed == block.length);
- return total_processed;
+ return GetSpaced(converter, out, batch_size, valid_bits, valid_bits_offset, null_count);
}
-static inline bool IndexInRange(int32_t idx, int32_t dictionary_length) {
- return idx >= 0 && idx < dictionary_length;
+namespace internal {
+
+template <typename I>
+bool IndexInRange(I idx, int32_t dictionary_length) {
+ ARROW_DCHECK_GT(dictionary_length, 0);
+ using T = std::common_type_t<decltype(idx), decltype(dictionary_length)>;
+ return idx >= 0 && static_cast<T>(idx) < static_cast<T>(dictionary_length);
}
// Converter for GetSpaced that handles runs of returned dictionary
// indices.
-template <typename T>
+template <typename V, typename I>
struct DictionaryConverter {
- T kZero = {};
- const T* dictionary;
- int32_t dictionary_length;
-
- inline bool IsValid(int32_t value) { return IndexInRange(value, dictionary_length); }
-
- inline bool IsValid(const int32_t* values, int32_t length) const {
- using IndexType = int32_t;
- IndexType min_index = std::numeric_limits<IndexType>::max();
- IndexType max_index = std::numeric_limits<IndexType>::min();
- for (int x = 0; x < length; x++) {
- min_index = std::min(values[x], min_index);
- max_index = std::max(values[x], max_index);
+ using out_type = V;
+ using in_type = I;
+ using size_type = int32_t;
+
+ static constexpr bool kIsIdentity = false;
+
+ const out_type* dictionary;
+ size_type dictionary_length;
+
+ [[nodiscard]] bool InputIsValid(in_type idx) const {
+ return IndexInRange(idx, dictionary_length);
+ }
+
+ [[nodiscard]] bool InputIsValid(const in_type* indices, size_type length) const {
Review Comment:
I suppose this is only called with `length > 0`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #47294:
URL: https://github.com/apache/arrow/pull/47294#issuecomment-3167929345
:warning: GitHub issue #47112 **has been automatically assigned in GitHub** to PR creator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "voltrondatabot (via GitHub)" <gi...@apache.org>.
voltrondatabot commented on PR #47294:
URL: https://github.com/apache/arrow/pull/47294#issuecomment-3323472873
Benchmark runs are scheduled for commit fc4f31108625857d372ec3ff4248226af812e050. Watch https://buildkite.com/apache-arrow and https://conbench.ursa.dev for updates. A comment will be posted here when the runs are complete.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #47294:
URL: https://github.com/apache/arrow/pull/47294#issuecomment-3305909237
@ursabot please benchmark
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2349473825
##########
cpp/src/arrow/util/bit_util.h:
##########
@@ -365,5 +365,118 @@ void PackBits(const uint32_t* values, uint8_t* out) {
}
}
+constexpr int64_t MaxLEB128ByteLen(int64_t n_bits) { return CeilDiv(n_bits, 7); }
+
+template <typename Int>
+constexpr int64_t MaxLEB128ByteLenFor = MaxLEB128ByteLen(sizeof(Int) * 8);
+
+/// Write a integer as LEB128
+///
+/// Write the input value as LEB128 into the outptu buffer and return the number of bytes
+/// written.
+/// If the output buffer size is insufficient, return 0 but the output may have been
+/// written to.
+///
+/// \see https://en.wikipedia.org/wiki/LEB128
+/// \see MaxLEB128ByteLenFor
+template <typename Int>
+constexpr int32_t WriteLEB128(Int value, uint8_t* out, int32_t max_out_size) {
+ constexpr Int kLow7Mask = Int(0x7F);
+ constexpr Int kHigh7Mask = ~kLow7Mask;
+ constexpr uint8_t kContinuationBit = 0x80;
+
+ auto const out_first = out;
+
+ // Write as many bytes as the could be for the given input
+ while ((value & kHigh7Mask) != Int(0)) {
+ // We do not have enough room to write the LEB128
+ if (out - out_first >= max_out_size) {
+ return 0;
+ }
+
+ // Write the encoded byte with continuation bit
+ *out = static_cast<uint8_t>(value & kLow7Mask) | kContinuationBit;
+ ++out;
+ // Shift remaining data
+ value >>= 7;
+ }
+
+ // We do not have enough room to write the LEB128
+ if (out - out_first >= max_out_size) {
+ return 0;
+ }
+
+ // Write last non-continuing byte
+ *out = static_cast<uint8_t>(value & kLow7Mask);
+ ++out;
+
+ return static_cast<int32_t>(out - out_first);
+}
+
+/// Parse a leading LEB128
+///
+/// Take as input a data pointer and the maximum number of bytes that can be read from it
+/// (typically the array size).
+/// When a valid LEB128 is found at the start of the data, the function writes it to the
+/// out pointer and return the number of bytes read.
+/// Otherwise, the out pointer is unmodified and zero is returned.
+///
+/// \see https://en.wikipedia.org/wiki/LEB128
+/// \see MaxLEB128ByteLenFor
+template <typename Int>
+constexpr int32_t ParseLeadingLEB128(uint8_t const* data, int32_t max_data_size,
+ Int* out) {
+ constexpr auto kMaxBytes = static_cast<int32_t>(MaxLEB128ByteLenFor<Int>);
+ static_assert(kMaxBytes >= 1);
+ constexpr uint8_t kLow7Mask = 0x7F;
+ constexpr uint8_t kContinuationBit = 0x80;
+ constexpr int32_t kSignBitCount = std::is_signed_v<Int> ? 1 : 0;
+ // Number of bits allowed for encoding data on the last byte to avoid overflow
+ constexpr uint8_t kHighBitCount = (8 * sizeof(Int) - kSignBitCount) % 7;
+ // kHighBitCount least significant `0` bits and the rest with `1`
+ constexpr uint8_t kHighForbiddenMask = ~((1 << kHighBitCount) - 1);
+
+ // Iteratively building the value
+ std::make_unsigned_t<Int> value = 0;
+
+ // Read as many bytes as the could be for the give output
+ for (int32_t i = 0; i < kMaxBytes - 1; i++) {
+ // We have not finished reading a valid LEB128, yet we run out of data
Review Comment:
I had done a branch-less version of the LEB128 reading for in32 on the fast path that we can read all needed bytes at once. Turned out to be slower.
My hypothesis is that with `PREDICT_FALSE` on cases with not enough bytes + prefetching the next bytes (I assume), this implementation is not as bad as it looks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2363212254
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -82,86 +81,400 @@ namespace util {
/// 200 ints = 25 groups of 8
/// <varint((25 << 1) | 1)> <25 bytes of values, bitpacked>
/// (total 26 bytes, 1 byte overhead)
-//
-/// Decoder class for RLE encoded data.
-class RleDecoder {
+/// The type for an encoded Rle of BitPacked run size, between 1 and 2^31-1 as per Parquet
+/// spec.
+/// This is also pragmatically used for other integer used in the Rle and BitPacked runs
+/// and decoder to avoid conversions.
+/// It can therefore be referred to as a "typical" size for Rle and BitPacked logic.
+using rle_size_t = int32_t;
+
+template <typename T>
+class RleRunDecoder;
+
+class RleRun {
public:
- /// Create a decoder object. buffer/buffer_len is the decoded data.
- /// bit_width is the width of each value (before encoding).
- RleDecoder(const uint8_t* buffer, int buffer_len, int bit_width)
- : bit_reader_(buffer, buffer_len),
- bit_width_(bit_width),
- current_value_(0),
- repeat_count_(0),
- literal_count_(0) {
- ARROW_DCHECK_GE(bit_width_, 0);
- ARROW_DCHECK_LE(bit_width_, 64);
+ /// The decoder class used to decode a single run in the given type.
+ template <typename T>
+ using DecoderType = RleRunDecoder<T>;
+
+ constexpr RleRun() noexcept = default;
+
+ explicit RleRun(const uint8_t* data, rle_size_t values_count,
+ rle_size_t value_bit_width) noexcept
+ : values_count_(values_count), value_bit_width_(value_bit_width) {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_GE(values_count, 0);
+ std::copy(data, data + raw_data_size(), data_.begin());
}
- RleDecoder() : bit_width_(-1) {}
+ /// The number of repeated values in this run.
+ constexpr rle_size_t values_count() const noexcept { return values_count_; }
- void Reset(const uint8_t* buffer, int buffer_len, int bit_width) {
- ARROW_DCHECK_GE(bit_width, 0);
- ARROW_DCHECK_LE(bit_width, 64);
- bit_reader_.Reset(buffer, buffer_len);
- bit_width_ = bit_width;
- current_value_ = 0;
- repeat_count_ = 0;
- literal_count_ = 0;
+ /// The size in bits of each encoded value.
+ constexpr rle_size_t values_bit_width() const noexcept { return value_bit_width_; }
+
+ /// A pointer to the repeated value raw bytes.
+ constexpr const uint8_t* raw_data_ptr() const noexcept { return data_.data(); }
+
+ /// The number of bytes used for the raw repeated value.
+ constexpr rle_size_t raw_data_size() const noexcept {
+ auto out = bit_util::BytesForBits(value_bit_width_);
+ ARROW_DCHECK_LE(out, std::numeric_limits<rle_size_t>::max());
+ return static_cast<rle_size_t>(out);
}
- /// Gets the next value. Returns false if there are no more.
+ private:
+ /// The repeated value raw bytes stored inside the class with enough space to store
+ /// up to a 64 bit value.
+ std::array<uint8_t, 8> data_ = {};
+ /// The number of time the value is repeated.
+ rle_size_t values_count_ = 0;
+ /// The size in bit of a packed value in the run.
+ rle_size_t value_bit_width_ = 0;
+};
+
+template <typename T>
+class BitPackedRunDecoder;
+
+class BitPackedRun {
+ public:
+ /// The decoder class used to decode a single run in the given type.
+ template <typename T>
+ using DecoderType = BitPackedRunDecoder<T>;
+
+ constexpr BitPackedRun() noexcept = default;
+
+ constexpr BitPackedRun(const uint8_t* data, rle_size_t values_count,
+ rle_size_t value_bit_width) noexcept
+ : data_(data), values_count_(values_count), value_bit_width_(value_bit_width) {
+ ARROW_CHECK_GE(value_bit_width_, 0);
+ ARROW_CHECK_GE(values_count_, 0);
+ }
+
+ constexpr rle_size_t values_count() const noexcept { return values_count_; }
+
+ /// The size in bits of each encoded value.
+ constexpr rle_size_t values_bit_width() const noexcept { return value_bit_width_; }
+
+ constexpr const uint8_t* raw_data_ptr() const noexcept { return data_; }
+
+ constexpr rle_size_t raw_data_size() const noexcept {
+ auto out = bit_util::BytesForBits(static_cast<int64_t>(value_bit_width_) *
+ static_cast<int64_t>(values_count_));
+ ARROW_CHECK_LE(out, std::numeric_limits<rle_size_t>::max());
+ return static_cast<rle_size_t>(out);
+ }
+
+ private:
+ /// The pointer to the beginning of the run
+ const uint8_t* data_ = nullptr;
+ /// Number of values in this run.
+ rle_size_t values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ rle_size_t value_bit_width_ = 0;
+};
+
+/// A parser that emits either a ``BitPackedRun`` or a ``RleRun``.
+class RleBitPackedParser {
+ public:
+ /// The different types of runs emitted by the parser
+ using dynamic_run_type = std::variant<RleRun, BitPackedRun>;
+
+ constexpr RleBitPackedParser() noexcept = default;
+
+ constexpr RleBitPackedParser(const uint8_t* data, rle_size_t data_size,
+ rle_size_t value_bit_width) noexcept
+ : data_(data), data_size_(data_size), value_bit_width_(value_bit_width) {}
+
+ constexpr void Reset(const uint8_t* data, rle_size_t data_size,
+ rle_size_t value_bit_width) noexcept {
+ *this = {data, data_size, value_bit_width};
+ }
+
+ /// Whether there is still runs to iterate over.
+ ///
+ /// WARN: Due to simplistic error handling, iteration with Next and Peek could
+ /// fail to return data while the parser is not exhausted.
+ /// This is how one can check for errors.
+ bool exhausted() const { return data_size_ == 0; }
+
+ /// Enum to return from an ``Parse`` handler.
+ ///
+ /// Since a callback has no way to know when to stop, the handler must return
+ /// a value indicating to the ``Parse`` function whether to stop or continue.
+ enum class ControlFlow {
+ Continue,
+ Break,
+ };
+
+ /// A callback approach to parsing.
+ ///
+ /// This approach is used to reduce the number of dynamic lookups involved with using a
+ /// variant.
+ ///
+ /// The handler must be of the form
+ /// ```cpp
+ /// struct Handler {
+ /// ControlFlow OnBitPackedRun(BitPackedRun run);
+ ///
+ /// ControlFlow OnRleRun(RleRun run);
+ /// };
+ /// ```
+ template <typename Handler>
+ void Parse(Handler&& handler);
+
+ private:
+ /// The pointer to the beginning of the run
+ const uint8_t* data_ = nullptr;
+ /// Size in bytes of the run.
+ rle_size_t data_size_ = 0;
+ /// The size in bit of a packed value in the run
+ rle_size_t value_bit_width_ = 0;
Review Comment:
That would invalidate `RleBitPackedParser::Reset`, though we could also reconstruct a parser `RleBitPackedDecoder::Reset`. What do you prefer?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #47294:
URL: https://github.com/apache/arrow/pull/47294#issuecomment-3299477850
@ursabot please benchmark command=cpp-micro --suite-filter=parquet
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #47294:
URL: https://github.com/apache/arrow/pull/47294#issuecomment-3299609415
@ursabot please benchmark lang=C++
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2355096459
##########
cpp/src/arrow/util/bit_stream_utils_internal.h:
##########
@@ -439,91 +424,84 @@ inline bool BitReader::Advance(int64_t num_bits) {
return true;
}
-inline bool BitWriter::PutVlqInt(uint32_t v) {
- bool result = true;
- while ((v & 0xFFFFFF80UL) != 0UL) {
- result &= PutAligned<uint8_t>(static_cast<uint8_t>((v & 0x7F) | 0x80), 1);
- v >>= 7;
- }
- result &= PutAligned<uint8_t>(static_cast<uint8_t>(v & 0x7F), 1);
- return result;
-}
+template <typename Int>
+inline bool BitWriter::PutVlqInt(Int v) {
+ static_assert(std::is_integral_v<Int>);
Review Comment:
We could also write a positive number from an unsigned integer type
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2355358556
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -602,20 +1038,126 @@ struct GetBatchSpacedTestCase {
int bit_width;
};
-TEST(RleDecoder, GetBatchSpaced) {
- uint32_t kSeed = 1337;
- ::arrow::random::RandomArrayGenerator rand(kSeed);
-
- std::vector<GetBatchSpacedTestCase<int32_t>> int32_cases{
- {1, 100000, 0.01, 1}, {1, 100000, 0.1, 1}, {1, 100000, 0.5, 1},
- {4, 100000, 0.05, 3}, {100, 100000, 0.05, 7},
+template <typename T>
+void DoTestGetBatchSpacedRoundtrip() {
+ using Data = DataTestRleBitPacked<T>;
+ using ArrowType = typename Data::ArrowType;
+ using RandomPart = typename Data::RandomPart;
+ using NullPart = typename Data::NullPart;
+ using RepeatPart = typename Data::RepeatPart;
+
+ std::vector<Data> test_cases = {
+ {
+ {RandomPart{/* max=*/1, /* size=*/400, /* null_proba= */ 0.1}},
+ /* bit_width= */ 1,
+ },
+ {
+ {
+ RandomPart{/* max=*/7, /* size=*/10037, /* null_proba= */ 0.0},
+ NullPart{/* size= */ 1153},
+ RandomPart{/* max=*/7, /* size=*/800, /* null_proba= */ 0.5},
+ },
+ /* bit_width= */ 3,
+ },
+ {
+ {
+ NullPart{/* size= */ 80},
+ RandomPart{/* max=*/static_cast<T>(1023), /* size=*/800,
+ /* null_proba= */ 0.01},
+ NullPart{/* size= */ 1023},
+ },
+ /* bit_width= */ 11,
+ },
+ {
+ {RepeatPart{/* value=*/13, /* size=*/100000, /* null_proba= */ 0.01}},
+ /* bit_width= */ 10,
+ },
+ {
+ {
+ NullPart{/* size= */ 1024},
+ RepeatPart{/* value=*/static_cast<T>(10000), /* size=*/100000,
+ /* null_proba= */ 0.1},
+ NullPart{/* size= */ 77},
+ },
+ /* bit_width= */ 23,
+ },
+ {
+ {
+ RepeatPart{/* value=*/13, /* size=*/100000, /* null_proba= */ 0.0},
+ NullPart{/* size= */ 1153},
+ RepeatPart{/* value=*/72, /* size=*/100799, /* null_proba= */ 0.5},
+ },
+ /* bit_width= */ 10,
+ },
+ {
+ {
+ RandomPart{/* max=*/1, /* size=*/1013, /* null_proba= */ 0.01},
+ NullPart{/* size=*/8},
+ RepeatPart{1, /* size= */ 256, /* null_proba= */ 0.1},
+ NullPart{/* size=*/128},
+ RepeatPart{0, /* size= */ 256, /* null_proba= */ 0.0},
+ NullPart{/* size=*/15},
+ RandomPart{/* max=*/1, /* size=*/8 * 1024, /* null_proba= */ 0.01},
+ },
+ /* bit_width= */ 1,
+ },
};
- for (auto case_ : int32_cases) {
- auto arr = rand.Int32(case_.size, /*min=*/0, case_.max_value, case_.null_probability);
- CheckRoundTripSpaced<Int32Type>(*arr, case_.bit_width);
- CheckRoundTripSpaced<Int32Type>(*arr->Slice(1), case_.bit_width);
+
+ ::arrow::random::RandomArrayGenerator rand(/* seed= */ 12);
+ // FRAGILE: we create a dictionary large enough so that any encoded value from the
+ // previous test cases can be used as an index in the dictionary.
+ // Its size must be increased accordingly if larger values are encoded in the test
+ // cases.
+ auto dict = std::static_pointer_cast<arrow::FloatArray>(rand.Float32(20000, -1.0, 1.0));
+
+ // Number of bits available in T to write a positive integer.
+ constexpr int kBitsAvailable = 8 * sizeof(T) - (std::is_signed_v<T> ? 1 : 0);
Review Comment:
No, for a given bit_width, we test with all possible integer type that can fit that contain it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "HuaHuaY (via GitHub)" <gi...@apache.org>.
HuaHuaY commented on PR #47294:
URL: https://github.com/apache/arrow/pull/47294#issuecomment-4052114256
> > We tested using a Parquet file with data generated from SSB Flat. We found that the less the number of distinct values, the more the performance regression.
> >
> > * `parquet-scan-main` is compiled from the latest commit of the main branch.
> > * `parquet-scan-old` is compiled from `64f2055ffb68e5077420f4253e76d78952438cab` which is the previous commit of this PR on the main branch.
> >
> > Both are compiled in release mode with `-DARROW_RUNTIME_SIMD_LEVEL=AVX2` cmake flag.
> > ```shell
> > ❯ env LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$LLVM_ROOT/lib/x86_64-unknown-linux-gnu ARROW_RUNTIME_SIMD_LEVEL=AVX2 hyperfine -w 5 -r 20 --sort mean-time "cpp/out/build/ninja-release/release/parquet-scan-main --columns=6 /dev/shm/TableSink0" "cpp/out/build/ninja-release/release/parquet-scan-old --columns=6 /dev/shm/TableSink0"
> > Benchmark 1: cpp/out/build/ninja-release/release/parquet-scan-main --columns=6 /dev/shm/TableSink0
> > Time (mean ± σ): 31.1 ms ± 0.3 ms [User: 27.2 ms, System: 3.7 ms]
> > Range (min … max): 30.7 ms … 31.8 ms 20 runs
> >
> > Benchmark 2: cpp/out/build/ninja-release/release/parquet-scan-old --columns=6 /dev/shm/TableSink0
> > Time (mean ± σ): 22.8 ms ± 0.4 ms [User: 19.3 ms, System: 3.3 ms]
> > Range (min … max): 22.2 ms … 23.3 ms 20 runs
> >
> > Summary
> > cpp/out/build/ninja-release/release/parquet-scan-old --columns=6 /dev/shm/TableSink0 ran
> > 1.37 ± 0.02 times faster than cpp/out/build/ninja-release/release/parquet-scan-main --columns=6 /dev/shm/TableSink0
> >
> > ❯ env LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$LLVM_ROOT/lib/x86_64-unknown-linux-gnu cpp/out/build/ninja-release/release/parquet-reader --only-metadata /dev/shm/TableSink0
> > File Name: /dev/shm/TableSink0
> > Version: 2.6
> > Created By: cz-cpp version BuildInfo:GitBranch:release/20240820_rc8,GitVersion:73a4383,BuildTime:1725298762,CloudEnv:ALIYUN
> > Total rows: 6250733
> > Number of RowGroups: 1
> > Number of Real Columns: 40
> > Number of Columns: 40
> > Number of Selected Columns: 40
> > ......
> > Column 6: lo_orderpriority (BYTE_ARRAY / String / UTF8)
> > ......
> > --- Row Group: 0 ---
> > --- Total Bytes: 1016215218 ---
> > --- Total Compressed Bytes: 552911018 ---
> > --- Sort Columns:
> > column_idx: 5, descending: 0, nulls_first: 1
> > column_idx: 0, descending: 0, nulls_first: 1
> > --- Rows: 6250733 ---
> > ......
> > Column 6
> > Values: 6250733, Null Values: 0, Distinct Values: 5
> > Max (exact: unknown): 5-LOW, Min (exact: unknown): 1-URGENT
> > Compression: LZ4_RAW, Encodings: PLAIN(DICT_PAGE) RLE_DICTIONARY
> > Uncompressed Size: 2267694, Compressed Size: 2092132
> > ......
> > ```
>
> Update from @HuaHuaY's comment: we just located the performance regression comes from the API change of the internal `unpack` function introduced by #47994. The quick fix is to explicitly set `UnpackOptions::max_read_bytes` to not use its default value -1. @AntoinePrv @pitrou
I tested by arrow's `parquet-scan` target and `cpp/src/arrow/util/rle_encoding_internal.h` has already set `max_read_bytes_ - bytes_fully_read`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on PR #47294:
URL: https://github.com/apache/arrow/pull/47294#issuecomment-3998517789
@HuaHuaY it is possible this PR (and some follow-up changes found by fuzzing) introduce light regressions in some cases but it should not be much (let me know what you find).
This goal of this PR is for longer term gain where we will be able to completely short-circuit the decoding in some cases, so that will more than offset it.
Recently, GH-47994 should also be a gain.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2337137830
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -118,50 +365,56 @@ class RleDecoder {
/// input with zeros. Since the encoding does not differentiate between
/// input values and padding, Get() returns true even for these padding
/// values.
- template <typename T>
- bool Get(T* val);
+ [[nodiscard]] bool Get(value_type* val);
Review Comment:
Yes
https://github.com/search?q=repo%3Aapache%2Farrow%20idx_decoder_.Get(&type=code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2337193315
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -207,12 +209,310 @@ TEST(BitUtil, RoundTripIntValues) {
}
}
+/// A Rle run is a simple class owning some data and a repetition count.
+/// It does not know how to read such data.
+TEST(Rle, RleRun) {
+ const std::array<RleRun::byte, 4> value = {21, 2, 0, 0};
+
+ RleRun::values_count_type value_count = 12;
+
+ // 12 times the value 21 fitting over 5 bits
+ auto const run_5 = RleRun(value.data(), value_count, /* value_bit_width= */ 5);
+ EXPECT_EQ(run_5.ValuesCount(), value_count);
+ EXPECT_EQ(run_5.ValuesBitWidth(), 5);
+ EXPECT_EQ(run_5.RawDataSize(), 1); // 5 bits fit in one byte
+ EXPECT_EQ(*run_5.RawDataPtr(), 21);
+
+ // 12 times the value 21 fitting over 16 bits
+ auto const run_8 = RleRun(value.data(), value_count, /* value_bit_width= */ 8);
+ EXPECT_EQ(run_8.ValuesCount(), value_count);
+ EXPECT_EQ(run_8.ValuesBitWidth(), 8);
+ EXPECT_EQ(run_8.RawDataSize(), 1); // 8 bits fit in 1 byte
+ EXPECT_EQ(*run_8.RawDataPtr(), 21);
+
+ // 12 times the value {21, 2} fitting over 10 bits
+ auto const run_10 = RleRun(value.data(), value_count, /* value_bit_width= */ 10);
+
+ EXPECT_EQ(run_10.ValuesCount(), value_count);
+ EXPECT_EQ(run_10.ValuesBitWidth(), 10);
+ EXPECT_EQ(run_10.RawDataSize(), 2); // 10 bits fit in 2 bytes
+ EXPECT_EQ(*(run_10.RawDataPtr() + 0), 21);
+ EXPECT_EQ(*(run_10.RawDataPtr() + 1), 2);
+
+ // 12 times the value {21, 2} fitting over 32 bits
+ auto const run_32 = RleRun(value.data(), value_count, /* value_bit_width= */ 32);
+ EXPECT_EQ(run_32.ValuesCount(), value_count);
+ EXPECT_EQ(run_32.ValuesBitWidth(), 32);
+ EXPECT_EQ(run_32.RawDataSize(), 4); // 32 bits fit in 4 bytes
+ EXPECT_EQ(*(run_32.RawDataPtr() + 0), 21);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 1), 2);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 2), 0);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 3), 0);
+}
+
+/// A BitPacked run is a simple class owning some data and its size.
+/// It does not know how to read such data.
+TEST(BitPacked, BitPackedRun) {
+ const std::array<BitPackedRun::byte, 4> value = {0b10101010, 0, 0, 0b1111111};
+
+ /// 16 values of 1 bit for a total of 16 bits
+ BitPackedRun::values_count_type value_count_1 = 16;
+ auto const run_1 = BitPackedRun(value.data(), value_count_1, /* value_bit_width= */ 1);
+ EXPECT_EQ(run_1.ValuesCount(), value_count_1);
+ EXPECT_EQ(run_1.ValuesBitWidth(), 1);
+ EXPECT_EQ(run_1.RawDataSize(), 2); // 16 bits fit in 2 bytes
+ for (BitPackedRun::raw_data_size_type i = 0; i < run_1.RawDataSize(); ++i) {
+ EXPECT_EQ(*(run_1.RawDataPtr() + i), value[i]);
+ }
+
+ /// 8 values of 3 bits for a total of 24 bits
+ BitPackedRun::values_count_type value_count_3 = 8;
+ auto const run_3 = BitPackedRun(value.data(), value_count_3, /* value_bit_width= */ 3);
+ EXPECT_EQ(run_3.ValuesCount(), value_count_3);
+ EXPECT_EQ(run_3.ValuesBitWidth(), 3);
+ EXPECT_EQ(run_3.RawDataSize(), 3); // 24 bits fit in 3 bytes
+ for (BitPackedRun::raw_data_size_type i = 0; i < run_3.RawDataSize(); ++i) {
+ EXPECT_EQ(*(run_3.RawDataPtr() + i), value[i]);
+ }
+}
+
+template <typename T>
+void TestRleDecoder(std::vector<RleRun::byte> bytes,
+ RleRun::values_count_type value_count,
+ RleRun::bit_size_type bit_width) {
+ // Pre-requisite for this test
+ EXPECT_GT(value_count, 6);
+
+ // Compute value associated with bytes encoded as little endian
+ T value = 0;
+ for (std::size_t i = 0; i < bytes.size(); ++i) {
+ value += static_cast<T>(bytes.at(i)) << (8 * i);
+ }
+
+ auto const run = RleRun(bytes.data(), value_count, bit_width);
+
+ auto decoder = RleDecoder<T>(run);
+ std::vector<T> vals = {0, 0};
+
+ EXPECT_EQ(decoder.Remaining(), value_count);
+
+ typename decltype(decoder)::values_count_type read = 0;
+ EXPECT_EQ(decoder.Get(vals.data()), 1);
+ read += 1;
+ EXPECT_EQ(vals.at(0), value);
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ EXPECT_EQ(decoder.Advance(3), 3);
+ read += 3;
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ vals = {0, 0};
+ EXPECT_EQ(decoder.GetBatch(vals.data(), 2), vals.size());
+ EXPECT_EQ(vals.at(0), value);
+ EXPECT_EQ(vals.at(1), value);
+ read += static_cast<decltype(read)>(vals.size());
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ // Exhaust iteration
+ EXPECT_EQ(decoder.Advance(value_count - read), value_count - read);
+ EXPECT_EQ(decoder.Remaining(), 0);
+ EXPECT_EQ(decoder.Advance(1), 0);
+ vals = {0, 0};
+ EXPECT_EQ(decoder.Get(vals.data()), 0);
+ EXPECT_EQ(vals.at(0), 0);
+
+ // Reset the decoder
+ decoder.Reset(run);
+ EXPECT_EQ(decoder.Remaining(), value_count);
+ vals = {0, 0};
+ EXPECT_EQ(decoder.GetBatch(vals.data(), 2), vals.size());
+ EXPECT_EQ(vals.at(0), value);
+ EXPECT_EQ(vals.at(1), value);
+}
+
+TEST(Rle, RleDecoder) {
+ TestRleDecoder<uint32_t>({21, 0, 0}, /* value_count= */ 21, /* bit_width= */ 5);
+ TestRleDecoder<uint16_t>({1, 0}, /* value_count= */ 13, /* bit_width= */ 1);
+ TestRleDecoder<uint64_t>({21, 2, 0, 1}, /* value_count= */ 20, /* bit_width= */ 30);
+}
+
+template <typename T>
+void TestBitPackedDecoder(std::vector<RleRun::byte> bytes,
+ BitPackedRun::values_count_type value_count,
+ BitPackedRun::bit_size_type bit_width,
+ std::vector<T> expected) {
+ // Pre-requisite for this test
+ EXPECT_GT(value_count, 6);
+
+ auto const run = BitPackedRun(bytes.data(), value_count, bit_width);
+
+ auto decoder = BitPackedDecoder<T>(run);
+ std::vector<T> vals = {0, 0};
+
+ EXPECT_EQ(decoder.Remaining(), value_count);
+
+ typename decltype(decoder)::values_count_type read = 0;
+ EXPECT_EQ(decoder.Get(vals.data()), 1);
+ EXPECT_EQ(vals.at(0), expected.at(0 + read));
+ read += 1;
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ EXPECT_EQ(decoder.Advance(3), 3);
+ read += 3;
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ vals = {0, 0};
+ EXPECT_EQ(decoder.GetBatch(vals.data(), 2), vals.size());
+ EXPECT_EQ(vals.at(0), expected.at(0 + read));
+ EXPECT_EQ(vals.at(1), expected.at(1 + read));
+ read += static_cast<decltype(read)>(vals.size());
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ // Exhaust iteration
+ EXPECT_EQ(decoder.Advance(value_count - read), value_count - read);
+ EXPECT_EQ(decoder.Remaining(), 0);
+ EXPECT_EQ(decoder.Advance(1), 0);
+ vals = {0, 0};
+ EXPECT_EQ(decoder.Get(vals.data()), 0);
+ EXPECT_EQ(vals.at(0), 0);
+
+ // Reset the decoder
+ decoder.Reset(run);
+ read = 0;
+ EXPECT_EQ(decoder.Remaining(), value_count);
+ vals = {0, 0};
+ EXPECT_EQ(decoder.GetBatch(vals.data(), 2), vals.size());
+ EXPECT_EQ(vals.at(0), expected.at(0 + read));
+ EXPECT_EQ(vals.at(1), expected.at(1 + read));
+}
+
+TEST(BitPacked, BitPackedDecoder) {
+ /// See parquet encoding for bytes layout
+ TestBitPackedDecoder<uint16_t>(
+ /* bytes= */ {0x88, 0xc6, 0xfa},
+ /* values_count= */ 8,
+ /* bit_width= */ 3,
+ /* expected= */ {0, 1, 2, 3, 4, 5, 6, 7});
+ TestBitPackedDecoder<uint8_t>(
+ /* bytes= */ {0x0, 0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7},
+ /* values_count= */ 8,
+ /* bit_width= */ 8,
+ /* expected= */ {0, 1, 2, 3, 4, 5, 6, 7});
+ TestBitPackedDecoder<uint32_t>(
+ /* bytes= */ {0x47, 0xc, 0x10, 0x35},
+ /* values_count= */ 8,
+ /* bit_width= */ 4,
+ /* expected= */ {7, 4, 12, 0, 0, 1, 5, 3});
+ TestBitPackedDecoder<uint64_t>(
+ /* bytes= */ {0xe8, 0x7, 0x20, 0xc0, 0x0, 0x4, 0x14, 0x60, 0xc0, 0x1},
+ /* values_count= */ 8,
+ /* bit_width= */ 10,
+ /* expected= */ {1000, 1, 2, 3, 4, 5, 6, 7});
+}
+
+template <typename T>
+void TestRleBitPackedParser(std::vector<RleRun::byte> bytes,
+ RleBitPackedParser::bit_size_type bit_width,
+ std::vector<T> expected) {
+ auto parser = RleBitPackedParser(
+ bytes.data(), static_cast<BitPackedRun::raw_data_size_type>(bytes.size()),
+ bit_width);
+ EXPECT_FALSE(parser.Exhausted());
+
+ // Peek return the same data
+ auto run1 = parser.Peek();
+ EXPECT_TRUE(run1.has_value());
+ auto run2 = parser.Peek();
+ EXPECT_TRUE(run2.has_value());
+ auto ptr1 = std::visit([](auto const& r) { return r.RawDataPtr(); }, run1.value());
+ auto size1 = std::visit([](auto const& r) { return r.RawDataSize(); }, run1.value());
+ auto ptr2 = std::visit([](auto const& r) { return r.RawDataPtr(); }, run2.value());
+ auto size2 = std::visit([](auto const& r) { return r.RawDataSize(); }, run2.value());
+ EXPECT_TRUE(std::equal(ptr1, ptr1 + size1, ptr2, ptr2 + size2));
+ EXPECT_FALSE(parser.Exhausted());
+
+ // Try to decode all data of all runs in the decoded vector
+ decltype(expected) decoded = {};
+ auto rle_decoder = RleDecoder<T>();
+ auto bit_packed_decoder = BitPackedDecoder<T>();
+ // Iterate over all runs
+ while (auto run = parser.Next()) {
+ EXPECT_TRUE(run.has_value());
+
+ if (std::holds_alternative<RleRun>(run.value())) {
+ rle_decoder.Reset(std::get<RleRun>(run.value()));
Review Comment:
Yes, on the other hand extending its lifetime also has the added benefit of testing that it is properly Reset.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2336690910
##########
cpp/src/arrow/util/bit_util.h:
##########
@@ -365,5 +365,118 @@ void PackBits(const uint32_t* values, uint8_t* out) {
}
}
+constexpr int64_t MaxLEB128ByteLen(int64_t n_bits) { return CeilDiv(n_bits, 7); }
+
+template <typename Int>
+constexpr int64_t MaxLEB128ByteLenFor = MaxLEB128ByteLen(sizeof(Int) * 8);
+
+/// Write a integer as LEB128
+///
+/// Write the input value as LEB128 into the outptu buffer and return the number of bytes
+/// written.
+/// If the output buffer size is insufficient, return 0 but the output may have been
+/// written to.
+///
+/// \see https://en.wikipedia.org/wiki/LEB128
+/// \see MaxLEB128ByteLenFor
+template <typename Int>
+constexpr int32_t WriteLEB128(Int value, uint8_t* out, int32_t max_out_size) {
+ constexpr Int kLow7Mask = Int(0x7F);
+ constexpr Int kHigh7Mask = ~kLow7Mask;
+ constexpr uint8_t kContinuationBit = 0x80;
+
+ auto const out_first = out;
+
+ // Write as many bytes as the could be for the given input
+ while ((value & kHigh7Mask) != Int(0)) {
+ // We do not have enough room to write the LEB128
+ if (out - out_first >= max_out_size) {
+ return 0;
+ }
+
+ // Write the encoded byte with continuation bit
+ *out = static_cast<uint8_t>(value & kLow7Mask) | kContinuationBit;
+ ++out;
+ // Shift remaining data
+ value >>= 7;
+ }
+
+ // We do not have enough room to write the LEB128
+ if (out - out_first >= max_out_size) {
+ return 0;
+ }
+
+ // Write last non-continuing byte
+ *out = static_cast<uint8_t>(value & kLow7Mask);
+ ++out;
+
+ return static_cast<int32_t>(out - out_first);
+}
+
+/// Parse a leading LEB128
+///
+/// Take as input a data pointer and the maximum number of bytes that can be read from it
+/// (typically the array size).
+/// When a valid LEB128 is found at the start of the data, the function writes it to the
+/// out pointer and return the number of bytes read.
+/// Otherwise, the out pointer is unmodified and zero is returned.
+///
+/// \see https://en.wikipedia.org/wiki/LEB128
+/// \see MaxLEB128ByteLenFor
+template <typename Int>
+constexpr int32_t ParseLeadingLEB128(uint8_t const* data, int32_t max_data_size,
+ Int* out) {
+ constexpr auto kMaxBytes = static_cast<int32_t>(MaxLEB128ByteLenFor<Int>);
+ static_assert(kMaxBytes >= 1);
+ constexpr uint8_t kLow7Mask = 0x7F;
+ constexpr uint8_t kContinuationBit = 0x80;
+ constexpr int32_t kSignBitCount = std::is_signed_v<Int> ? 1 : 0;
Review Comment:
This decoding function allows reading into a signed integer type, but the encoded value can only represent non-negatives anyways (the encoding function refuses negative inputs).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2367144653
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +612,663 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/************************
+ * RleBitPackedParser *
+ ************************/
+
+template <typename Handler>
+void RleBitPackedParser::Parse(Handler&& handler) {
+ while (!exhausted()) {
+ auto [read, control] = PeekImpl(handler);
+ data_ += read;
+ data_size_ -= read;
+ if (ARROW_PREDICT_FALSE(control == ControlFlow::Break)) {
+ break;
+ }
+ }
+}
+
+namespace internal {
+/// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+} // namespace internal
+
+template <typename Handler>
+auto RleBitPackedParser::PeekImpl(Handler&& handler) const
+ -> std::pair<rle_size_t, ControlFlow> {
+ ARROW_DCHECK(!exhausted());
+
+ constexpr auto kMaxSize = bit_util::kMaxLEB128ByteLenFor<uint32_t>;
+ uint32_t run_len_type = 0;
+ const auto header_bytes = bit_util::ParseLeadingLEB128(data_, kMaxSize, &run_len_type);
+
+ if (ARROW_PREDICT_FALSE(header_bytes == 0)) {
+ // Malfomrmed LEB128 data
+ return {};
+ }
+
+ const bool is_bit_packed = run_len_type & 1;
+ const uint32_t count = run_len_type >> 1;
+ if (is_bit_packed) {
+ constexpr auto kMaxCount = bit_util::CeilDiv(internal::max_size_for_v<rle_size_t>, 8);
+ if (ARROW_PREDICT_FALSE(count == 0 || count > kMaxCount)) {
+ // Illegal number of encoded values
+ return {0, ControlFlow::Break};
+ }
+
+ const auto values_count = static_cast<rle_size_t>(count * 8);
+ ARROW_DCHECK_LT(count, internal::max_size_for_v<rle_size_t>);
+ // Count Already divided by 8
+ const auto bytes_read =
+ header_bytes + static_cast<rle_size_t>(count) * value_bit_width_;
+
+ auto control = handler.OnBitPackedRun(
+ BitPackedRun(data_ + header_bytes, values_count, value_bit_width_));
+
+ return {bytes_read, control};
+ }
+
+ if (ARROW_PREDICT_FALSE(
+ count == 0 ||
+ count > static_cast<uint32_t>(std::numeric_limits<rle_size_t>::max()))) {
+ // Illegal number of encoded values
+ return {0, ControlFlow::Break};
+ }
+
+ const auto values_count = static_cast<rle_size_t>(count);
+ const auto value_bytes = bit_util::BytesForBits(value_bit_width_);
+ ARROW_DCHECK_LT(value_bytes, internal::max_size_for_v<rle_size_t>);
+ const auto bytes_read = header_bytes + static_cast<rle_size_t>(value_bytes);
+
+ auto control =
+ handler.OnRleRun(RleRun(data_ + header_bytes, values_count, value_bit_width_));
+
+ return {bytes_read, control};
+}
+
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+template <typename Callable>
+void RleBitPackedDecoder<T>::ParseWithCallable(Callable&& func) {
+ struct {
+ Callable func;
+ auto OnBitPackedRun(BitPackedRun run) { return func(std::move(run)); }
+ auto OnRleRun(RleRun run) { return func(std::move(run)); }
+ } handler{std::move(func)};
+
+ parser_.Parse(std::move(handler));
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, rle_size_t batch_size)
+ -> rle_size_t {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ rle_size_t values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(run_remaining() > 0)) {
+ const auto read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(run_remaining() == 0);
}
+ ParseWithCallable([&](auto run) {
+ using RunDecoder = typename decltype(run)::template DecoderType<value_type>;
+
+ ARROW_DCHECK_LT(values_read, batch_size);
+ RunDecoder decoder(run);
+ const auto read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
Review Comment:
Ok, but then, when is `read == 0` supposed to occur? The only possibility is that the `decoder`'s length is 0, but `PeekImpl` is careful to avoid that.
I think we can change that to a debug assertion:
```c++
// Stop reading and store remaining decoder
if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
decoder_ = std::move(decoder);
return ControlFlow::Break;
}
// We ensure that `values_read != batch_size` before calling `RunDecoder::GetBatch`
// and `ParseWithCallable` ensures the run isn't empty.
ARROW_DCHECK(read != 0);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "HuaHuaY (via GitHub)" <gi...@apache.org>.
HuaHuaY commented on PR #47294:
URL: https://github.com/apache/arrow/pull/47294#issuecomment-3997703287
We're experiencing a performance regression with the Parquet reader on our internal program. We are still locating the regression and suspecting it's related to this PR. I checked the full Conbench report, which mentions several unstable performance issues, all seemingly related to the changes in this PR. However, the URLs in the report is no longer accessible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2319483141
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ ARROW_DCHECK_LT(values_read, batch_size);
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+ auto const read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
return values_read;
}
-template <typename T, typename RunType, typename Converter>
-inline int RleDecoder::GetSpaced(Converter converter, int batch_size, int null_count,
- const uint8_t* valid_bits, int64_t valid_bits_offset,
- T* out) {
- if (ARROW_PREDICT_FALSE(null_count == batch_size)) {
- converter.FillZero(out, out + batch_size);
- return batch_size;
+namespace internal {
+
+/// Utility class to safely handle values and null count without too error-prone
+/// verbosity.
+class BatchCounter {
+ public:
+ using size_type = int32_t;
+
+ [[nodiscard]] static constexpr BatchCounter FromBatchSizeAndNulls(
+ size_type batch_size, size_type null_count) {
+ ARROW_DCHECK_LE(null_count, batch_size);
+ return {batch_size - null_count, null_count};
}
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
- int values_remaining = batch_size - null_count;
+ constexpr BatchCounter(size_type values_count, size_type null_count) noexcept
+ : values_count_(values_count), null_count_(null_count) {}
- // Assume no bits to start.
- arrow::internal::BitRunReader bit_reader(valid_bits, valid_bits_offset,
- /*length=*/batch_size);
- arrow::internal::BitRun valid_run = bit_reader.NextRun();
- while (values_read < batch_size) {
- if (ARROW_PREDICT_FALSE(valid_run.length == 0)) {
- valid_run = bit_reader.NextRun();
+ [[nodiscard]] constexpr size_type ValuesCount() const noexcept { return values_count_; }
+
+ [[nodiscard]] constexpr size_type ValuesRead() const noexcept { return values_read_; }
+
+ [[nodiscard]] constexpr size_type ValuesRemaining() const noexcept {
+ ARROW_DCHECK_LE(values_read_, values_count_);
+ return values_count_ - values_read_;
+ }
+
+ constexpr void AccrueReadValues(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, ValuesRemaining());
+ values_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type NullCount() const noexcept { return null_count_; }
+
+ [[nodiscard]] constexpr size_type NullRead() const noexcept { return null_read_; }
+
+ [[nodiscard]] constexpr size_type NullRemaining() const noexcept {
+ ARROW_DCHECK_LE(null_read_, null_count_);
+ return null_count_ - null_read_;
+ }
+
+ constexpr void AccrueReadNulls(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, NullRemaining());
+ null_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type TotalRemaining() const noexcept {
+ return ValuesRemaining() + NullRemaining();
+ }
+
+ [[nodiscard]] constexpr size_type TotalRead() const noexcept {
+ return values_read_ + null_read_;
+ }
+
+ [[nodiscard]] constexpr bool IsFullyNull() const noexcept {
+ return ValuesRemaining() == 0;
+ }
+
+ [[nodiscard]] constexpr bool IsDone() const noexcept { return TotalRemaining() == 0; }
+
+ private:
+ size_type values_count_ = 0;
+ size_type values_read_ = 0;
+ size_type null_count_ = 0;
+ size_type null_read_ = 0;
+};
+
+// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+/// Overload for GetSpaced for a single run in a RleDecoder
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ RleDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto values_remaining_run = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // Consume as much as possible from the repeated run.
+ // We only need to count the number of nulls and non-nulls because we can fill in the
+ // same value for nulls and non-nulls.
+ // This proves to be a big efficiency win.
+ while (values_remaining_run() > 0 && !batch.IsDone()) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const& validity_run_size = static_cast<values_count_type>(validity_run.length);
+
+ if (validity_run.set) {
+ // We may end the current RLE run in the middle of the validity run
+ auto update_size = std::min(validity_run_size, values_remaining_run());
+ batch.AccrueReadValues(update_size);
+ validity_run.length -= update_size;
+ } else {
+ // We can consume all nulls here because it does not matter if we consume on this
+ // RLE run, or an a next encoded run. The value filled does not matter.
+ auto update_size = std::min(validity_run_size, batch.NullRemaining());
+ batch.AccrueReadNulls(update_size);
+ validity_run.length -= update_size;
+ }
+
+ if (ARROW_PREDICT_TRUE(validity_run.length == 0)) {
+ validity_run = validity_reader.NextRun();
}
+ }
+
+ value_type const value = decoder.Value();
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(value))) {
+ return {0, 0};
+ }
+ converter.WriteRepeated(out, out + batch.TotalRead(), value);
+ auto const actual_values_read = decoder.Advance(batch.ValuesRead());
+ // We always cropped the number of values_read by the remaining values in the run.
+ // What's more the RLE decoder should not encounter any errors.
+ ARROW_DCHECK_EQ(actual_values_read, batch.ValuesRead());
- ARROW_DCHECK_GT(batch_size, 0);
- ARROW_DCHECK_GT(valid_run.length, 0);
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+template <typename T, typename... Ts>
+[[nodiscard]] constexpr T min(T x, Ts... ys) {
+ ((x = std::min(x, ys)), ...);
+ return x;
+}
+
+static_assert(min(5) == 5);
+static_assert(min(5, 4, -1) == -1);
+static_assert(min(5, 41) == 5);
+
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ BitPackedDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto run_values_remaining = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ while (run_values_remaining() > 0 && batch.ValuesRemaining() > 0) {
+ // TODO should this size be tune depending on sizeof(value_size)? cpu cache size?
+ // Pull a batch of values from the bit packed encoded data and store it in a local
+ // buffer to benefit from unpacking intrinsics and data locality.
+ static constexpr values_count_type kBufferCapacity = 1024;
+ std::array<value_type, kBufferCapacity> buffer = {};
+
+ values_count_type buffer_start = 0;
+ values_count_type buffer_end = 0;
+ auto buffer_size = [&]() {
+ auto out = buffer_end - buffer_start;
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // buffer_start is 0 at this point so size is end
+ buffer_end = min(run_values_remaining(), batch.ValuesRemaining(), kBufferCapacity);
+ buffer_end = decoder.GetBatch(buffer.data(), buffer_size());
+ ARROW_DCHECK_LE(buffer_size(), kBufferCapacity);
+
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(buffer.data(), buffer_size()))) {
+ return {batch.ValuesRead(), batch.NullRead()};
+ }
- if (valid_run.set) {
- if ((repeat_count_ == 0) && (literal_count_ == 0)) {
- if (!NextCounts<RunType>()) return values_read;
- ARROW_DCHECK((repeat_count_ > 0) ^ (literal_count_ > 0));
+ // Copy chunks of valid values into the output, while adjusting spacing for null
+ // values.
+ while (buffer_size() > 0) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const validity_run_length =
+ static_cast<values_count_type>(validity_run.length);
+
+ // Copy as much as possible from the buffer into the output while not exceeding
+ // validity run
+ if (validity_run.set) {
+ auto const update_size = std::min(validity_run_length, buffer_size());
+ converter.WriteRange(out, buffer.data() + buffer_start, update_size);
+ buffer_start += update_size;
+ batch.AccrueReadValues(update_size);
+ out += update_size;
+ validity_run.length -= update_size;
+ // Simply write zeros in the output
+ } else {
+ auto const update_size = std::min(validity_run_length, batch.NullRemaining());
+ converter.WriteZero(out, out + update_size);
+ batch.AccrueReadNulls(update_size);
+ out += update_size;
+ validity_run.length -= update_size;
}
- if (repeat_count_ > 0) {
- int repeat_batch = 0;
- // Consume the entire repeat counts incrementing repeat_batch to
- // be the total of nulls + values consumed, we only need to
- // get the total count because we can fill in the same value for
- // nulls and non-nulls. This proves to be a big efficiency win.
- while (repeat_count_ > 0 && (values_read + repeat_batch) < batch_size) {
- ARROW_DCHECK_GT(valid_run.length, 0);
- if (valid_run.set) {
- int update_size = std::min(static_cast<int>(valid_run.length), repeat_count_);
- repeat_count_ -= update_size;
- repeat_batch += update_size;
- valid_run.length -= update_size;
- values_remaining -= update_size;
- } else {
- // We can consume all nulls here because we would do so on
- // the next loop anyways.
- repeat_batch += static_cast<int>(valid_run.length);
- valid_run.length = 0;
- }
- if (valid_run.length == 0) {
- valid_run = bit_reader.NextRun();
- }
- }
- RunType current_value = static_cast<RunType>(current_value_);
- if (ARROW_PREDICT_FALSE(!converter.IsValid(current_value))) {
- return values_read;
- }
- converter.Fill(out, out + repeat_batch, current_value);
- out += repeat_batch;
- values_read += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(values_remaining, literal_count_);
- ARROW_DCHECK_GT(literal_batch, 0);
-
- // Decode the literals
- constexpr int kBufferSize = 1024;
- RunType indices[kBufferSize];
- literal_batch = std::min(literal_batch, kBufferSize);
- int actual_read = bit_reader_.GetBatch(bit_width_, indices, literal_batch);
- if (ARROW_PREDICT_FALSE(actual_read != literal_batch)) {
- return values_read;
- }
- if (!converter.IsValid(indices, /*length=*/actual_read)) {
- return values_read;
- }
- int skipped = 0;
- int literals_read = 0;
- while (literals_read < literal_batch) {
- if (valid_run.set) {
- int update_size = std::min(literal_batch - literals_read,
- static_cast<int>(valid_run.length));
- converter.Copy(out, indices + literals_read, update_size);
- literals_read += update_size;
- out += update_size;
- valid_run.length -= update_size;
- } else {
- converter.FillZero(out, out + valid_run.length);
- out += valid_run.length;
- skipped += static_cast<int>(valid_run.length);
- valid_run.length = 0;
- }
- if (valid_run.length == 0) {
- valid_run = bit_reader.NextRun();
- }
- }
- literal_count_ -= literal_batch;
- values_remaining -= literal_batch;
- values_read += literal_batch + skipped;
+ if (validity_run.length == 0) {
+ validity_run = validity_reader.NextRun();
}
- } else {
- converter.FillZero(out, out + valid_run.length);
- out += valid_run.length;
- values_read += static_cast<int>(valid_run.length);
- valid_run.length = 0;
}
+
+ ARROW_DCHECK_EQ(buffer_size(), 0);
}
- ARROW_DCHECK_EQ(valid_run.length, 0);
- ARROW_DCHECK_EQ(values_remaining, 0);
- return values_read;
+
+ ARROW_DCHECK_EQ(values_available - decoder.Remaining(), batch.ValuesRead());
+ ARROW_DCHECK_LE(batch.TotalRead(), batch_size);
+ ARROW_DCHECK_LE(batch.NullRead(), batch.NullCount());
+
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+/// Overload for GetSpaced for a single run in a decoder variant
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(
+ Converter& converter, typename Converter::out_type* out, values_count_type batch_size,
+ values_count_type null_count, BitRunReader&& validity_reader, BitRun&& validity_run,
+ std::variant<RleDecoder<value_type>, BitPackedDecoder<value_type>>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ return std::visit(
+ [&](auto& dec) {
+ ARROW_DCHECK_GT(dec.Remaining(), 0);
+ return RunGetSpaced(converter, out, batch_size, null_count, validity_reader,
+ validity_run, dec);
+ },
+ decoder);
}
+} // namespace internal
+
+template <typename T>
+template <typename Converter>
+auto RleBitPackedDecoder<T>::GetSpaced(
+ Converter converter, typename Converter::out_type* out, values_count_type batch_size,
+ const byte* validity_bits, int64_t validity_bits_offset, values_count_type null_count)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
+
+ ARROW_DCHECK_GT(batch_size, 0);
+
+ auto batch = internal::BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ if (ARROW_PREDICT_FALSE(batch.IsFullyNull())) {
+ converter.WriteZero(out, out + batch.NullRemaining());
+ return batch.NullRemaining();
+ }
+
+ arrow::internal::BitRunReader validity_reader(validity_bits, validity_bits_offset,
+ /*length=*/batch.TotalRemaining());
+ arrow::internal::BitRun validity_run = validity_reader.NextRun();
+
+ auto const check_and_handle_fully_null_remaining = [&]() {
+ if (batch.IsFullyNull()) {
+ ARROW_DCHECK(validity_run.length == 0 || !validity_run.set);
+ ARROW_DCHECK_GE(validity_run.length, batch.NullRemaining());
+
+ converter.WriteZero(out, out + batch.NullRemaining());
+ out += batch.NullRemaining();
+ batch.AccrueReadNulls(batch.NullRemaining());
+ }
+ };
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const [values_read, null_read] =
+ RunGetSpaced(converter, out, batch.TotalRemaining(), batch.NullRemaining(),
+ validity_reader, validity_run, decoder_);
+
+ batch.AccrueReadNulls(null_read);
+ batch.AccrueReadValues(values_read);
+ out += values_read + null_read;
+
+ // Either we fulfilled all the batch values to be read
+ if (ARROW_PREDICT_FALSE(batch.ValuesRemaining() == 0)) {
+ // There may be remaining null if they are not greedily filled
+ check_and_handle_fully_null_remaining();
+ return batch.TotalRead();
+ }
+
+ /// We finished the remaining run
+ ARROW_DCHECK(RunRemaining() == 0);
+ }
+
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+
+ auto const [values_read, null_read] = internal::RunGetSpaced(
+ converter, out, batch.TotalRemaining(), batch.NullRemaining(),
+ validity_reader, validity_run, decoder);
+
+ batch.AccrueReadNulls(null_read);
+ batch.AccrueReadValues(values_read);
+ out += values_read + null_read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == 0 || batch.ValuesRemaining() == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
+ // There may be remaining null if they are not greedily filled by either decoder calls
+ check_and_handle_fully_null_remaining();
+
+ ARROW_DCHECK(batch.IsDone() || Exhausted());
+ // batch.Done() => batch.NullRemaining() == 0
+ ARROW_DCHECK(!batch.IsDone() || (batch.NullRemaining() == 0));
+ return batch.TotalRead();
+}
+
+namespace internal {
+
// Converter for GetSpaced that handles runs that get returned
// directly as output.
template <typename T>
-struct PlainRleConverter {
- T kZero = {};
- inline bool IsValid(const T& values) const { return true; }
- inline bool IsValid(const T* values, int32_t length) const { return true; }
- inline void Fill(T* begin, T* end, const T& run_value) const {
+struct NoOpConverter {
+ using in_type = T;
+ using out_type = T;
+ using size_type = int32_t;
+
+ static constexpr bool kIsIdentity = true;
+
+ [[nodiscard]] static constexpr bool InputIsValid(const in_type& values) { return true; }
+
+ [[nodiscard]] static constexpr bool InputIsValid(const in_type* values,
+ size_type length) {
+ return true;
+ }
+
+ static void WriteRepeated(out_type* begin, out_type* end, in_type run_value) {
std::fill(begin, end, run_value);
}
- inline void FillZero(T* begin, T* end) { std::fill(begin, end, kZero); }
- inline void Copy(T* out, const T* values, int length) const {
- std::memcpy(out, values, length * sizeof(T));
+
+ static void WriteZero(out_type* begin, out_type* end) {
+ std::fill(begin, end, out_type{});
+ }
+
+ static void WriteRange(out_type* out, const in_type* values, size_type length) {
+ std::memcpy(out, values, length * sizeof(out_type));
}
};
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatchSpaced(int batch_size, int null_count,
- const uint8_t* valid_bits,
- int64_t valid_bits_offset, T* out) {
+auto RleBitPackedDecoder<T>::GetBatchSpaced(values_count_type batch_size,
+ values_count_type null_count,
+ const byte* valid_bits,
+ int64_t valid_bits_offset, value_type* out)
+ -> values_count_type {
if (null_count == 0) {
- return GetBatch<T>(out, batch_size);
+ return GetBatch(out, batch_size);
}
- PlainRleConverter<T> converter;
- arrow::internal::BitBlockCounter block_counter(valid_bits, valid_bits_offset,
- batch_size);
-
- int total_processed = 0;
- int processed = 0;
- arrow::internal::BitBlockCount block;
+ internal::NoOpConverter<value_type> converter;
- do {
- block = block_counter.NextFourWords();
- if (block.length == 0) {
- break;
- }
- if (block.AllSet()) {
- processed = GetBatch<T>(out, block.length);
- } else if (block.NoneSet()) {
- converter.FillZero(out, out + block.length);
- processed = block.length;
- } else {
- processed = GetSpaced<T, /*RunType=*/T, PlainRleConverter<T>>(
- converter, block.length, block.length - block.popcount, valid_bits,
- valid_bits_offset, out);
- }
- total_processed += processed;
- out += block.length;
- valid_bits_offset += block.length;
- } while (processed == block.length);
- return total_processed;
+ return GetSpaced(converter, out, batch_size, valid_bits, valid_bits_offset, null_count);
}
-static inline bool IndexInRange(int32_t idx, int32_t dictionary_length) {
- return idx >= 0 && idx < dictionary_length;
+namespace internal {
+
+template <typename I>
+bool IndexInRange(I idx, int32_t dictionary_length) {
+ ARROW_DCHECK_GT(dictionary_length, 0);
+ using T = std::common_type_t<decltype(idx), decltype(dictionary_length)>;
+ return idx >= 0 && static_cast<T>(idx) < static_cast<T>(dictionary_length);
}
// Converter for GetSpaced that handles runs of returned dictionary
// indices.
-template <typename T>
+template <typename V, typename I>
struct DictionaryConverter {
- T kZero = {};
- const T* dictionary;
- int32_t dictionary_length;
-
- inline bool IsValid(int32_t value) { return IndexInRange(value, dictionary_length); }
-
- inline bool IsValid(const int32_t* values, int32_t length) const {
- using IndexType = int32_t;
- IndexType min_index = std::numeric_limits<IndexType>::max();
- IndexType max_index = std::numeric_limits<IndexType>::min();
- for (int x = 0; x < length; x++) {
- min_index = std::min(values[x], min_index);
- max_index = std::max(values[x], max_index);
+ using out_type = V;
+ using in_type = I;
+ using size_type = int32_t;
+
+ static constexpr bool kIsIdentity = false;
+
+ const out_type* dictionary;
+ size_type dictionary_length;
+
+ [[nodiscard]] bool InputIsValid(in_type idx) const {
+ return IndexInRange(idx, dictionary_length);
+ }
+
+ [[nodiscard]] bool InputIsValid(const in_type* indices, size_type length) const {
+ in_type min_index = std::numeric_limits<in_type>::max();
+ in_type max_index = std::numeric_limits<in_type>::min();
+ for (size_type x = 0; x < length; x++) {
+ min_index = std::min(indices[x], min_index);
+ max_index = std::max(indices[x], max_index);
}
return IndexInRange(min_index, dictionary_length) &&
IndexInRange(max_index, dictionary_length);
}
- inline void Fill(T* begin, T* end, const int32_t& run_value) const {
+
+ void WriteRepeated(out_type* begin, out_type* end, in_type run_value) const {
std::fill(begin, end, dictionary[run_value]);
}
- inline void FillZero(T* begin, T* end) { std::fill(begin, end, kZero); }
- inline void Copy(T* out, const int32_t* values, int length) const {
- for (int x = 0; x < length; x++) {
+ static void WriteZero(out_type* begin, out_type* end) {
+ std::fill(begin, end, out_type{});
+ }
+
+ void WriteRange(out_type* out, const in_type* values, size_type length) const {
+ for (size_type x = 0; x < length; x++) {
out[x] = dictionary[values[x]];
}
}
};
+/// Dummy imitation of BitRun that is all set.
+struct AllSetBitRun {
+ static constexpr bool set = true;
+ int64_t length = 0;
+};
+
+/// Dummy imitation of BitRunReader that should never be called.
+struct UnreachableBitRunReader {
+ constexpr static AllSetBitRun NextRun() { return {}; }
+};
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatchWithDict(const T* dictionary, int32_t dictionary_length,
- T* values, int batch_size) {
- // Per https://github.com/apache/parquet-format/blob/master/Encodings.md,
- // the maximum dictionary index width in Parquet is 32 bits.
- using IndexType = int32_t;
- DictionaryConverter<T> converter;
- converter.dictionary = dictionary;
- converter.dictionary_length = dictionary_length;
-
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) {
- auto idx = static_cast<IndexType>(current_value_);
- if (ARROW_PREDICT_FALSE(!IndexInRange(idx, dictionary_length))) {
- return values_read;
- }
- T val = dictionary[idx];
+template <typename V>
+auto RleBitPackedDecoder<T>::GetBatchWithDict(const V* dictionary,
+ int32_t dictionary_length, V* out,
+ values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
+
+ if (ARROW_PREDICT_FALSE(batch_size <= 0)) {
+ return 0;
+ }
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, val);
+ internal::DictionaryConverter<V, value_type> converter{dictionary, dictionary_length};
- /* Upkeep counters */
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- constexpr int kBufferSize = 1024;
- IndexType indices[kBufferSize];
+ // Make lightweight BitRun class to reuse previous methods.
+ constexpr internal::UnreachableBitRunReader validity_reader{};
+ internal::AllSetBitRun validity_run = {batch_size};
- int literal_batch = std::min(remaining, literal_count_);
- literal_batch = std::min(literal_batch, kBufferSize);
+ values_count_type values_read = 0;
+ auto batch_values_remaining = [&]() {
+ ARROW_DCHECK_LE(values_read, batch_size);
+ return batch_size - values_read;
+ };
- int actual_read = bit_reader_.GetBatch(bit_width_, indices, literal_batch);
- if (ARROW_PREDICT_FALSE(actual_read != literal_batch)) {
- return values_read;
- }
- if (ARROW_PREDICT_FALSE(!converter.IsValid(indices, /*length=*/literal_batch))) {
- return values_read;
- }
- converter.Copy(out, indices, literal_batch);
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const [run_values_read, run_null_read] =
+ RunGetSpaced(converter, out, batch_size, /* null_count= */ 0, validity_reader,
+ validity_run, decoder_);
- /* Upkeep counters */
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<IndexType>()) return values_read;
+ ARROW_DCHECK_EQ(run_null_read, 0);
+ values_read += run_values_read;
+ out += run_values_read;
+
+ // Either we fulfilled all the batch values to be read
+ if (ARROW_PREDICT_FALSE(values_read >= batch_size)) {
+ // There may be remaining null if they are not greedily filled
+ return values_read;
}
+
+ /// We finished the remaining run
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+
+ auto const [run_values_read, run_null_read] = internal::RunGetSpaced(
+ converter, out, batch_values_remaining(), /* null_count= */ 0,
+ validity_reader, validity_run, decoder);
+
+ ARROW_DCHECK_EQ(run_null_read, 0);
+ values_read += run_values_read;
+ out += run_values_read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(run_values_read == 0 || values_read == batch_size)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
return values_read;
}
template <typename T>
-inline int RleDecoder::GetBatchWithDictSpaced(const T* dictionary,
- int32_t dictionary_length, T* out,
- int batch_size, int null_count,
- const uint8_t* valid_bits,
- int64_t valid_bits_offset) {
+template <typename V>
+auto RleBitPackedDecoder<T>::GetBatchWithDictSpaced(
+ const V* dictionary, int32_t dictionary_length, V* out, values_count_type batch_size,
+ values_count_type null_count, const uint8_t* valid_bits, int64_t valid_bits_offset)
+ -> values_count_type {
if (null_count == 0) {
- return GetBatchWithDict<T>(dictionary, dictionary_length, out, batch_size);
- }
- arrow::internal::BitBlockCounter block_counter(valid_bits, valid_bits_offset,
- batch_size);
- using IndexType = int32_t;
- DictionaryConverter<T> converter;
- converter.dictionary = dictionary;
- converter.dictionary_length = dictionary_length;
-
- int total_processed = 0;
- int processed = 0;
- arrow::internal::BitBlockCount block;
- do {
- block = block_counter.NextFourWords();
- if (block.length == 0) {
- break;
+ return GetBatchWithDict<V>(dictionary, dictionary_length, out, batch_size);
+ }
+ internal::DictionaryConverter<V, value_type> converter{dictionary, dictionary_length};
+
+ return GetSpaced(converter, out, batch_size, valid_bits, valid_bits_offset, null_count);
+}
+
+/************
+ * RleRun *
+ ************/
+
+inline RleRun::RleRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept
+ : values_count_(values_count), value_bit_width_(value_bit_width) {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_GE(values_count, 0);
+ std::copy(data, data + RawDataSize(), data_.begin());
+}
+
+constexpr auto RleRun::ValuesCount() const noexcept -> values_count_type {
+ return values_count_;
+}
+
+constexpr auto RleRun::ValuesBitWidth() const noexcept -> bit_size_type {
+ return value_bit_width_;
+}
+
+constexpr auto RleRun::RawDataPtr() const noexcept -> raw_data_const_pointer {
+ return data_.data();
+}
+
+constexpr auto RleRun::RawDataSize() const noexcept -> raw_data_size_type {
+ auto out = bit_util::BytesForBits(value_bit_width_);
+ ARROW_DCHECK_LE(out, std::numeric_limits<raw_data_size_type>::max());
+ return static_cast<raw_data_size_type>(out);
+};
+
+/******************
+ * BitPackedRun *
+ ******************/
+
+constexpr BitPackedRun::BitPackedRun(raw_data_const_pointer data,
+ values_count_type values_count,
+ bit_size_type value_bit_width) noexcept
+ : data_(data), values_count_(values_count), value_bit_width_(value_bit_width) {
+ ARROW_CHECK_GE(value_bit_width_, 0);
+ ARROW_CHECK_GE(values_count_, 0);
+}
+
+constexpr auto BitPackedRun::ValuesCount() const noexcept -> values_count_type {
+ return values_count_;
+}
+
+constexpr auto BitPackedRun::ValuesBitWidth() const noexcept -> bit_size_type {
+ return value_bit_width_;
+}
+
+constexpr auto BitPackedRun::RawDataPtr() const noexcept -> raw_data_const_pointer {
+ return data_;
+}
+
+constexpr auto BitPackedRun::RawDataSize() const noexcept -> raw_data_size_type {
+ auto out = bit_util::BytesForBits(static_cast<int64_t>(value_bit_width_) *
+ static_cast<int64_t>(values_count_));
+ ARROW_CHECK_LE(out, std::numeric_limits<raw_data_size_type>::max());
+ return static_cast<raw_data_size_type>(out);
+}
+
+/************************
+ * RleBitPackedParser *
+ ************************/
+
+constexpr RleBitPackedParser::RleBitPackedParser(raw_data_const_pointer data,
+ raw_data_size_type size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, size, value_bit_width);
+}
+
+constexpr void RleBitPackedParser::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ data_ = data;
+ data_size_ = data_size;
+ value_bit_width_ = value_bit_width;
+}
+
+inline auto RleBitPackedParser::Peek() const -> std::optional<dynamic_run_type> {
+ if (ARROW_PREDICT_FALSE(Exhausted())) {
+ return {};
+ }
+
+ auto out = std::optional<dynamic_run_type>{};
+ auto handler = internal::LambdaHandler{[&](auto run) {
+ out = run;
+ return ControlFlow::Break;
+ }};
+ PeekImpl(handler);
+ return out;
+}
+
+inline auto RleBitPackedParser::Next() -> std::optional<dynamic_run_type> {
+ if (ARROW_PREDICT_FALSE(Exhausted())) {
+ return {};
+ }
+
+ auto out = std::optional<dynamic_run_type>{};
+ auto handler = internal::LambdaHandler{[&](auto run) {
+ out = run;
+ return ControlFlow::Break;
+ }};
+ PeekImpl(handler);
+ auto [read, control] = PeekImpl(handler);
+ data_ += read;
+ data_size_ -= read;
+ return out;
+}
+
+inline bool RleBitPackedParser::Advance() { return Next().has_value(); }
+
+inline bool RleBitPackedParser::Exhausted() const { return data_size_ == 0; }
+
+template <typename Handler>
+auto RleBitPackedParser::PeekImpl(Handler&& handler) const
+ -> std::pair<raw_data_size_type, ControlFlow> {
+ ARROW_DCHECK(!Exhausted());
+
+ constexpr auto kMaxSize = bit_util::MaxLEB128ByteLenFor<uint32_t>;
+ uint32_t run_len_type = 0;
+ auto const header_bytes = bit_util::ParseLeadingLEB128(data_, kMaxSize, &run_len_type);
+
+ if (ARROW_PREDICT_FALSE(header_bytes == 0)) {
+ // Malfomrmed LEB128 data
+ return {};
+ }
+
+ bool const is_bit_packed = run_len_type & 1;
+ uint32_t const count = run_len_type >> 1;
+ if (is_bit_packed) {
+ using values_count_type = BitPackedRun::values_count_type;
+ constexpr auto kMaxCount =
+ bit_util::CeilDiv(internal::max_size_for_v<values_count_type>, 8);
+ if (ARROW_PREDICT_FALSE(count == 0 || count > kMaxCount)) {
+ /// Illegal number of encoded values
+ return {};
}
- if (block.AllSet()) {
- processed = GetBatchWithDict<T>(dictionary, dictionary_length, out, block.length);
- } else if (block.NoneSet()) {
- converter.FillZero(out, out + block.length);
- processed = block.length;
- } else {
- processed = GetSpaced<T, /*RunType=*/IndexType, DictionaryConverter<T>>(
- converter, block.length, block.length - block.popcount, valid_bits,
- valid_bits_offset, out);
+
+ auto const values_count = static_cast<values_count_type>(count * 8);
+ ARROW_DCHECK_LT(count, internal::max_size_for_v<raw_data_size_type>);
+ // Count Already divided by 8
+ auto const bytes_read =
+ header_bytes + static_cast<raw_data_size_type>(count) * value_bit_width_;
+
+ auto control = handler.OnBitPackedRun(
+ BitPackedRun(data_ + header_bytes, values_count, value_bit_width_));
+
+ return {bytes_read, control};
+ }
+
+ using values_count_type = RleRun::values_count_type;
+ if (ARROW_PREDICT_FALSE(
+ count == 0 ||
+ count > static_cast<uint32_t>(std::numeric_limits<values_count_type>::max()))) {
+ /// Illegal number of encoded values
+ return {};
+ }
+
+ auto const values_count = static_cast<values_count_type>(count);
+ auto const value_bytes = bit_util::BytesForBits(value_bit_width_);
+ ARROW_DCHECK_LT(value_bytes, internal::max_size_for_v<raw_data_size_type>);
+ auto const bytes_read = header_bytes + static_cast<raw_data_size_type>(value_bytes);
+
+ auto control =
+ handler.OnRleRun(RleRun(data_ + header_bytes, values_count, value_bit_width_));
+
+ return {bytes_read, control};
+}
+
+template <typename Handler>
+void RleBitPackedParser::Parse(Handler&& handler) {
+ while (!Exhausted()) {
+ auto [read, control] = PeekImpl(handler);
+ data_ += read;
+ data_size_ -= read;
+ if (ARROW_PREDICT_FALSE(control == ControlFlow::Break)) {
+ break;
}
- total_processed += processed;
- out += block.length;
- valid_bits_offset += block.length;
- } while (processed == block.length);
- return total_processed;
+ }
}
+/****************
+ * RleDecoder *
+ ****************/
+
template <typename T>
-bool RleDecoder::NextCounts() {
- // Read the next run's indicator int, it could be a literal or repeated run.
- // The int is encoded as a vlq-encoded value.
- uint32_t indicator_value = 0;
- if (!bit_reader_.GetVlqInt(&indicator_value)) return false;
-
- // lsb indicates if it is a literal run or repeated run
- bool is_literal = indicator_value & 1;
- uint32_t count = indicator_value >> 1;
- if (is_literal) {
- if (ARROW_PREDICT_FALSE(count == 0 || count > static_cast<uint32_t>(INT32_MAX) / 8)) {
- return false;
- }
- literal_count_ = count * 8;
+RleDecoder<T>::RleDecoder(run_type const& run) noexcept {
Review Comment:
By the way, there seems to be no reason for these short/tiny methods to be defined outside of the class declaration, and inlining the definition there would probably make the code more readable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "HuaHuaY (via GitHub)" <gi...@apache.org>.
HuaHuaY commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2306492567
##########
cpp/src/arrow/util/bit_util_test.cc:
##########
@@ -1997,11 +1997,119 @@ TEST(BitUtil, RoundUpToPowerOf2) {
#undef U64
#undef S64
+/// Test the maximum number of bytes needed to write a LEB128 of a give size.
+TEST(BitStreamUtil, MaxLEB128ByteLenFor) {
Review Comment:
Will it be better to ensure correctness through `static_assert`?
##########
cpp/src/arrow/util/bit_util.h:
##########
@@ -365,5 +365,118 @@ void PackBits(const uint32_t* values, uint8_t* out) {
}
}
+constexpr int64_t MaxLEB128ByteLen(int64_t n_bits) { return CeilDiv(n_bits, 7); }
+
+template <typename Int>
+constexpr int64_t MaxLEB128ByteLenFor = MaxLEB128ByteLen(sizeof(Int) * 8);
+
+/// Write a integer as LEB128
+///
+/// Write the input value as LEB128 into the outptu buffer and return the number of bytes
+/// written.
+/// If the output buffer size is insufficient, return 0 but the output may have been
+/// written to.
+///
+/// \see https://en.wikipedia.org/wiki/LEB128
+/// \see MaxLEB128ByteLenFor
+template <typename Int>
+constexpr int32_t WriteLEB128(Int value, uint8_t* out, int32_t max_out_size) {
+ constexpr Int kLow7Mask = Int(0x7F);
+ constexpr Int kHigh7Mask = ~kLow7Mask;
+ constexpr uint8_t kContinuationBit = 0x80;
+
+ auto const out_first = out;
+
+ // Write as many bytes as the could be for the given input
+ while ((value & kHigh7Mask) != Int(0)) {
+ // We do not have enough room to write the LEB128
+ if (out - out_first >= max_out_size) {
+ return 0;
+ }
+
+ // Write the encoded byte with continuation bit
+ *out = static_cast<uint8_t>(value & kLow7Mask) | kContinuationBit;
+ ++out;
+ // Shift remaining data
+ value >>= 7;
+ }
+
+ // We do not have enough room to write the LEB128
+ if (out - out_first >= max_out_size) {
+ return 0;
+ }
+
+ // Write last non-continuing byte
+ *out = static_cast<uint8_t>(value & kLow7Mask);
+ ++out;
+
+ return static_cast<int32_t>(out - out_first);
+}
+
+/// Parse a leading LEB128
+///
+/// Take as input a data pointer and the maximum number of bytes that can be read from it
+/// (typically the array size).
+/// When a valid LEB128 is found at the start of the data, the function writes it to the
+/// out pointer and return the number of bytes read.
+/// Otherwise, the out pointer is unmodified and zero is returned.
+///
+/// \see https://en.wikipedia.org/wiki/LEB128
+/// \see MaxLEB128ByteLenFor
+template <typename Int>
+constexpr int32_t ParseLeadingLEB128(uint8_t const* data, int32_t max_data_size,
+ Int* out) {
+ constexpr auto kMaxBytes = static_cast<int32_t>(MaxLEB128ByteLenFor<Int>);
+ static_assert(kMaxBytes >= 1);
+ constexpr uint8_t kLow7Mask = 0x7F;
+ constexpr uint8_t kContinuationBit = 0x80;
+ constexpr int32_t kSignBitCount = std::is_signed_v<Int> ? 1 : 0;
+ // Number of bits allowed for encoding data on the last byte to avoid overflow
+ constexpr uint8_t kHighBitCount = (8 * sizeof(Int) - kSignBitCount) % 7;
+ // kHighBitCount least significant `0` bits and the rest with `1`
+ constexpr uint8_t kHighForbiddenMask = ~((1 << kHighBitCount) - 1);
+
+ // Iteratively building the value
+ std::make_unsigned_t<Int> value = 0;
+
+ // Read as many bytes as the could be for the give output
+ for (int32_t i = 0; i < kMaxBytes - 1; i++) {
+ // We have not finished reading a valid LEB128, yet we run out of data
+ if (ARROW_PREDICT_FALSE(i >= max_data_size)) {
+ return 0;
+ }
+
+ // Read the byte and set its 7 LSB to in the final value
+ uint8_t const byte = data[i];
+ value |= static_cast<Int>(byte & kLow7Mask) << (7 * i);
+
+ // Check for lack of continuation flag in MSB
+ if ((byte & kContinuationBit) == 0) {
+ *out = value;
+ return i + 1;
+ }
+ }
+
+ // Process the last index avoiding overflowing
+ constexpr int32_t last = kMaxBytes - 1;
+
+ // We have not finished reading a valid LEB128, yet we run out of data
+ if (ARROW_PREDICT_FALSE(last >= max_data_size)) {
+ return 0;
+ }
+
+ uint8_t const byte = data[last];
+
+ // Need to check if there are bits that would overflow the output.
+ // Also checks that there is no continuation.
+ if (ARROW_PREDICT_FALSE((byte & kHighForbiddenMask) != 0)) {
Review Comment:
In my opinion, high bit remains bit0 when original `BitWriter::PutVlqInt(uint64_t v)` writes the last byte due to right shift of `uint64_t`. In which case should we treat the last byte specially in `WriteLEB128` and `ParseLeadingLEB128 `?
##########
cpp/src/arrow/util/bit_stream_utils_internal.h:
##########
@@ -483,13 +496,17 @@ inline bool BitReader::GetZigZagVlqInt(int32_t* v) {
}
inline bool BitWriter::PutVlqInt(uint64_t v) {
- bool result = true;
- while ((v & 0xFFFFFFFFFFFFFF80ULL) != 0ULL) {
- result &= PutAligned<uint8_t>(static_cast<uint8_t>((v & 0x7F) | 0x80), 1);
- v >>= 7;
+ constexpr auto kMaxBytes = bit_util::MaxLEB128ByteLenFor<decltype(v)>;
Review Comment:
```suggestion
constexpr auto kMaxBytes = kMaxVlqByteLengthForInt64;
```
##########
cpp/src/arrow/util/bit_util.h:
##########
@@ -365,5 +365,118 @@ void PackBits(const uint32_t* values, uint8_t* out) {
}
}
+constexpr int64_t MaxLEB128ByteLen(int64_t n_bits) { return CeilDiv(n_bits, 7); }
+
+template <typename Int>
+constexpr int64_t MaxLEB128ByteLenFor = MaxLEB128ByteLen(sizeof(Int) * 8);
+
+/// Write a integer as LEB128
+///
+/// Write the input value as LEB128 into the outptu buffer and return the number of bytes
Review Comment:
```suggestion
/// Write the input value as LEB128 into the output buffer and return the number of bytes
```
##########
cpp/src/arrow/util/bit_util.h:
##########
@@ -365,5 +365,118 @@ void PackBits(const uint32_t* values, uint8_t* out) {
}
}
+constexpr int64_t MaxLEB128ByteLen(int64_t n_bits) { return CeilDiv(n_bits, 7); }
+
+template <typename Int>
+constexpr int64_t MaxLEB128ByteLenFor = MaxLEB128ByteLen(sizeof(Int) * 8);
+
+/// Write a integer as LEB128
+///
+/// Write the input value as LEB128 into the outptu buffer and return the number of bytes
+/// written.
+/// If the output buffer size is insufficient, return 0 but the output may have been
+/// written to.
+///
+/// \see https://en.wikipedia.org/wiki/LEB128
+/// \see MaxLEB128ByteLenFor
+template <typename Int>
+constexpr int32_t WriteLEB128(Int value, uint8_t* out, int32_t max_out_size) {
Review Comment:
Why do we write two new functions instead of moving the previous function here? If we want a generic function through template, why is `BitWriter::PutVlqInt(uint32_t v)` not replaced?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2322169639
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -602,20 +1033,106 @@ struct GetBatchSpacedTestCase {
int bit_width;
};
-TEST(RleDecoder, GetBatchSpaced) {
- uint32_t kSeed = 1337;
- ::arrow::random::RandomArrayGenerator rand(kSeed);
-
- std::vector<GetBatchSpacedTestCase<int32_t>> int32_cases{
- {1, 100000, 0.01, 1}, {1, 100000, 0.1, 1}, {1, 100000, 0.5, 1},
- {4, 100000, 0.05, 3}, {100, 100000, 0.05, 7},
+template <typename T>
+void DoTestGetBatchSpacedRoundtrip() {
+ using Data = DataTestRleBitPacked<T>;
+ using ArrowType = typename Data::ArrowType;
+ using RandomPart = typename Data::RandomPart;
+ using NullPart = typename Data::NullPart;
+ using RepeatPart = typename Data::RepeatPart;
+
+ std::vector<Data> test_cases = {
+ {
+ {RandomPart{/* max=*/1, /* size=*/400, /* null_proba= */ 0.1}},
+ /* bit_width= */ 1,
+ },
+ {
+ {
+ RandomPart{/* max=*/7, /* size=*/10037, /* null_proba= */ 0.1},
+ NullPart{/* size= */ 1153},
+ RandomPart{/* max=*/7, /* size=*/800, /* null_proba= */ 0.5},
+ },
+ /* bit_width= */ 3,
+ },
+ {
+ {
+ NullPart{/* size= */ 80},
+ RandomPart{/* max=*/1023, /* size=*/800, /* null_proba= */ 0.01},
+ NullPart{/* size= */ 1023},
+ },
+ /* bit_width= */ 11,
+ },
+ {
+ {RepeatPart{/* value=*/13, /* size=*/100000}},
+ /* bit_width= */ 10,
+ },
+ {
+ {
+ NullPart{/* size= */ 1024},
+ RepeatPart{/* value=*/10000, /* size=*/100000},
+ NullPart{/* size= */ 77},
+ },
+ /* bit_width= */ 23,
+ },
+ {
+ {
+ RepeatPart{/* value=*/13, /* size=*/100000},
+ NullPart{/* size= */ 1153},
+ RepeatPart{/* value=*/72, /* size=*/100799},
+ },
+ /* bit_width= */ 10,
+ },
+ {
+ {
+ RandomPart{/* max=*/1, /* size=*/1013, /* null_proba= */ 0.01},
+ NullPart{/* size=*/8},
+ RepeatPart{1, /* size= */ 256},
+ NullPart{/* size=*/128},
+ RepeatPart{0, /* size= */ 256},
+ NullPart{/* size=*/15},
+ RandomPart{/* max=*/1, /* size=*/8 * 1024, /* null_proba= */ 0.01},
+ },
+ /* bit_width= */ 1,
+ },
};
- for (auto case_ : int32_cases) {
- auto arr = rand.Int32(case_.size, /*min=*/0, case_.max_value, case_.null_probability);
- CheckRoundTripSpaced<Int32Type>(*arr, case_.bit_width);
- CheckRoundTripSpaced<Int32Type>(*arr->Slice(1), case_.bit_width);
+
+ ::arrow::random::RandomArrayGenerator rand(/* seed= */ 12);
+ // FRAGILE: Large enough so that it can be indexed by any value in all cases
Review Comment:
Why "FRAGILE"? I don't really understand this comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2322049113
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -207,12 +209,310 @@ TEST(BitUtil, RoundTripIntValues) {
}
}
+/// A Rle run is a simple class owning some data and a repetition count.
+/// It does not know how to read such data.
+TEST(Rle, RleRun) {
+ const std::array<RleRun::byte, 4> value = {21, 2, 0, 0};
+
+ RleRun::values_count_type value_count = 12;
+
+ // 12 times the value 21 fitting over 5 bits
+ auto const run_5 = RleRun(value.data(), value_count, /* value_bit_width= */ 5);
+ EXPECT_EQ(run_5.ValuesCount(), value_count);
+ EXPECT_EQ(run_5.ValuesBitWidth(), 5);
+ EXPECT_EQ(run_5.RawDataSize(), 1); // 5 bits fit in one byte
+ EXPECT_EQ(*run_5.RawDataPtr(), 21);
+
+ // 12 times the value 21 fitting over 16 bits
+ auto const run_8 = RleRun(value.data(), value_count, /* value_bit_width= */ 8);
+ EXPECT_EQ(run_8.ValuesCount(), value_count);
+ EXPECT_EQ(run_8.ValuesBitWidth(), 8);
+ EXPECT_EQ(run_8.RawDataSize(), 1); // 8 bits fit in 1 byte
+ EXPECT_EQ(*run_8.RawDataPtr(), 21);
+
+ // 12 times the value {21, 2} fitting over 10 bits
+ auto const run_10 = RleRun(value.data(), value_count, /* value_bit_width= */ 10);
+
+ EXPECT_EQ(run_10.ValuesCount(), value_count);
+ EXPECT_EQ(run_10.ValuesBitWidth(), 10);
+ EXPECT_EQ(run_10.RawDataSize(), 2); // 10 bits fit in 2 bytes
+ EXPECT_EQ(*(run_10.RawDataPtr() + 0), 21);
+ EXPECT_EQ(*(run_10.RawDataPtr() + 1), 2);
+
+ // 12 times the value {21, 2} fitting over 32 bits
+ auto const run_32 = RleRun(value.data(), value_count, /* value_bit_width= */ 32);
+ EXPECT_EQ(run_32.ValuesCount(), value_count);
+ EXPECT_EQ(run_32.ValuesBitWidth(), 32);
+ EXPECT_EQ(run_32.RawDataSize(), 4); // 32 bits fit in 4 bytes
+ EXPECT_EQ(*(run_32.RawDataPtr() + 0), 21);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 1), 2);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 2), 0);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 3), 0);
+}
+
+/// A BitPacked run is a simple class owning some data and its size.
+/// It does not know how to read such data.
+TEST(BitPacked, BitPackedRun) {
+ const std::array<BitPackedRun::byte, 4> value = {0b10101010, 0, 0, 0b1111111};
+
+ /// 16 values of 1 bit for a total of 16 bits
+ BitPackedRun::values_count_type value_count_1 = 16;
+ auto const run_1 = BitPackedRun(value.data(), value_count_1, /* value_bit_width= */ 1);
+ EXPECT_EQ(run_1.ValuesCount(), value_count_1);
+ EXPECT_EQ(run_1.ValuesBitWidth(), 1);
+ EXPECT_EQ(run_1.RawDataSize(), 2); // 16 bits fit in 2 bytes
+ for (BitPackedRun::raw_data_size_type i = 0; i < run_1.RawDataSize(); ++i) {
+ EXPECT_EQ(*(run_1.RawDataPtr() + i), value[i]);
+ }
+
+ /// 8 values of 3 bits for a total of 24 bits
+ BitPackedRun::values_count_type value_count_3 = 8;
+ auto const run_3 = BitPackedRun(value.data(), value_count_3, /* value_bit_width= */ 3);
+ EXPECT_EQ(run_3.ValuesCount(), value_count_3);
+ EXPECT_EQ(run_3.ValuesBitWidth(), 3);
+ EXPECT_EQ(run_3.RawDataSize(), 3); // 24 bits fit in 3 bytes
+ for (BitPackedRun::raw_data_size_type i = 0; i < run_3.RawDataSize(); ++i) {
+ EXPECT_EQ(*(run_3.RawDataPtr() + i), value[i]);
+ }
+}
+
+template <typename T>
+void TestRleDecoder(std::vector<RleRun::byte> bytes,
+ RleRun::values_count_type value_count,
+ RleRun::bit_size_type bit_width) {
+ // Pre-requisite for this test
+ EXPECT_GT(value_count, 6);
+
+ // Compute value associated with bytes encoded as little endian
+ T value = 0;
+ for (std::size_t i = 0; i < bytes.size(); ++i) {
+ value += static_cast<T>(bytes.at(i)) << (8 * i);
+ }
+
+ auto const run = RleRun(bytes.data(), value_count, bit_width);
+
+ auto decoder = RleDecoder<T>(run);
+ std::vector<T> vals = {0, 0};
+
+ EXPECT_EQ(decoder.Remaining(), value_count);
+
+ typename decltype(decoder)::values_count_type read = 0;
+ EXPECT_EQ(decoder.Get(vals.data()), 1);
+ read += 1;
+ EXPECT_EQ(vals.at(0), value);
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ EXPECT_EQ(decoder.Advance(3), 3);
+ read += 3;
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ vals = {0, 0};
+ EXPECT_EQ(decoder.GetBatch(vals.data(), 2), vals.size());
+ EXPECT_EQ(vals.at(0), value);
+ EXPECT_EQ(vals.at(1), value);
+ read += static_cast<decltype(read)>(vals.size());
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ // Exhaust iteration
+ EXPECT_EQ(decoder.Advance(value_count - read), value_count - read);
+ EXPECT_EQ(decoder.Remaining(), 0);
+ EXPECT_EQ(decoder.Advance(1), 0);
+ vals = {0, 0};
+ EXPECT_EQ(decoder.Get(vals.data()), 0);
+ EXPECT_EQ(vals.at(0), 0);
+
+ // Reset the decoder
+ decoder.Reset(run);
+ EXPECT_EQ(decoder.Remaining(), value_count);
+ vals = {0, 0};
+ EXPECT_EQ(decoder.GetBatch(vals.data(), 2), vals.size());
+ EXPECT_EQ(vals.at(0), value);
+ EXPECT_EQ(vals.at(1), value);
+}
+
+TEST(Rle, RleDecoder) {
+ TestRleDecoder<uint32_t>({21, 0, 0}, /* value_count= */ 21, /* bit_width= */ 5);
+ TestRleDecoder<uint16_t>({1, 0}, /* value_count= */ 13, /* bit_width= */ 1);
+ TestRleDecoder<uint64_t>({21, 2, 0, 1}, /* value_count= */ 20, /* bit_width= */ 30);
Review Comment:
Does `RleDecoder` support `uint8_t` output? If yes, can we test that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2355358556
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -602,20 +1038,126 @@ struct GetBatchSpacedTestCase {
int bit_width;
};
-TEST(RleDecoder, GetBatchSpaced) {
- uint32_t kSeed = 1337;
- ::arrow::random::RandomArrayGenerator rand(kSeed);
-
- std::vector<GetBatchSpacedTestCase<int32_t>> int32_cases{
- {1, 100000, 0.01, 1}, {1, 100000, 0.1, 1}, {1, 100000, 0.5, 1},
- {4, 100000, 0.05, 3}, {100, 100000, 0.05, 7},
+template <typename T>
+void DoTestGetBatchSpacedRoundtrip() {
+ using Data = DataTestRleBitPacked<T>;
+ using ArrowType = typename Data::ArrowType;
+ using RandomPart = typename Data::RandomPart;
+ using NullPart = typename Data::NullPart;
+ using RepeatPart = typename Data::RepeatPart;
+
+ std::vector<Data> test_cases = {
+ {
+ {RandomPart{/* max=*/1, /* size=*/400, /* null_proba= */ 0.1}},
+ /* bit_width= */ 1,
+ },
+ {
+ {
+ RandomPart{/* max=*/7, /* size=*/10037, /* null_proba= */ 0.0},
+ NullPart{/* size= */ 1153},
+ RandomPart{/* max=*/7, /* size=*/800, /* null_proba= */ 0.5},
+ },
+ /* bit_width= */ 3,
+ },
+ {
+ {
+ NullPart{/* size= */ 80},
+ RandomPart{/* max=*/static_cast<T>(1023), /* size=*/800,
+ /* null_proba= */ 0.01},
+ NullPart{/* size= */ 1023},
+ },
+ /* bit_width= */ 11,
+ },
+ {
+ {RepeatPart{/* value=*/13, /* size=*/100000, /* null_proba= */ 0.01}},
+ /* bit_width= */ 10,
+ },
+ {
+ {
+ NullPart{/* size= */ 1024},
+ RepeatPart{/* value=*/static_cast<T>(10000), /* size=*/100000,
+ /* null_proba= */ 0.1},
+ NullPart{/* size= */ 77},
+ },
+ /* bit_width= */ 23,
+ },
+ {
+ {
+ RepeatPart{/* value=*/13, /* size=*/100000, /* null_proba= */ 0.0},
+ NullPart{/* size= */ 1153},
+ RepeatPart{/* value=*/72, /* size=*/100799, /* null_proba= */ 0.5},
+ },
+ /* bit_width= */ 10,
+ },
+ {
+ {
+ RandomPart{/* max=*/1, /* size=*/1013, /* null_proba= */ 0.01},
+ NullPart{/* size=*/8},
+ RepeatPart{1, /* size= */ 256, /* null_proba= */ 0.1},
+ NullPart{/* size=*/128},
+ RepeatPart{0, /* size= */ 256, /* null_proba= */ 0.0},
+ NullPart{/* size=*/15},
+ RandomPart{/* max=*/1, /* size=*/8 * 1024, /* null_proba= */ 0.01},
+ },
+ /* bit_width= */ 1,
+ },
};
- for (auto case_ : int32_cases) {
- auto arr = rand.Int32(case_.size, /*min=*/0, case_.max_value, case_.null_probability);
- CheckRoundTripSpaced<Int32Type>(*arr, case_.bit_width);
- CheckRoundTripSpaced<Int32Type>(*arr->Slice(1), case_.bit_width);
+
+ ::arrow::random::RandomArrayGenerator rand(/* seed= */ 12);
+ // FRAGILE: we create a dictionary large enough so that any encoded value from the
+ // previous test cases can be used as an index in the dictionary.
+ // Its size must be increased accordingly if larger values are encoded in the test
+ // cases.
+ auto dict = std::static_pointer_cast<arrow::FloatArray>(rand.Float32(20000, -1.0, 1.0));
+
+ // Number of bits available in T to write a positive integer.
+ constexpr int kBitsAvailable = 8 * sizeof(T) - (std::is_signed_v<T> ? 1 : 0);
Review Comment:
No, for a given bit_width, we test with all possible integer type that can contain it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "voltrondatabot (via GitHub)" <gi...@apache.org>.
voltrondatabot commented on PR #47294:
URL: https://github.com/apache/arrow/pull/47294#issuecomment-3305909644
Benchmark runs are scheduled for commit c6a6c2140003c3a56ffde37d561e98a8103f5333. Watch https://buildkite.com/apache-arrow and https://conbench.ursa.dev for updates. A comment will be posted here when the runs are complete.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2349117516
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -602,20 +1033,106 @@ struct GetBatchSpacedTestCase {
int bit_width;
};
-TEST(RleDecoder, GetBatchSpaced) {
- uint32_t kSeed = 1337;
- ::arrow::random::RandomArrayGenerator rand(kSeed);
-
- std::vector<GetBatchSpacedTestCase<int32_t>> int32_cases{
- {1, 100000, 0.01, 1}, {1, 100000, 0.1, 1}, {1, 100000, 0.5, 1},
- {4, 100000, 0.05, 3}, {100, 100000, 0.05, 7},
+template <typename T>
+void DoTestGetBatchSpacedRoundtrip() {
+ using Data = DataTestRleBitPacked<T>;
+ using ArrowType = typename Data::ArrowType;
+ using RandomPart = typename Data::RandomPart;
+ using NullPart = typename Data::NullPart;
+ using RepeatPart = typename Data::RepeatPart;
+
+ std::vector<Data> test_cases = {
+ {
+ {RandomPart{/* max=*/1, /* size=*/400, /* null_proba= */ 0.1}},
+ /* bit_width= */ 1,
+ },
+ {
+ {
+ RandomPart{/* max=*/7, /* size=*/10037, /* null_proba= */ 0.1},
+ NullPart{/* size= */ 1153},
+ RandomPart{/* max=*/7, /* size=*/800, /* null_proba= */ 0.5},
+ },
+ /* bit_width= */ 3,
+ },
+ {
+ {
+ NullPart{/* size= */ 80},
+ RandomPart{/* max=*/1023, /* size=*/800, /* null_proba= */ 0.01},
+ NullPart{/* size= */ 1023},
+ },
+ /* bit_width= */ 11,
+ },
+ {
+ {RepeatPart{/* value=*/13, /* size=*/100000}},
+ /* bit_width= */ 10,
+ },
+ {
+ {
+ NullPart{/* size= */ 1024},
+ RepeatPart{/* value=*/10000, /* size=*/100000},
+ NullPart{/* size= */ 77},
+ },
+ /* bit_width= */ 23,
+ },
+ {
+ {
+ RepeatPart{/* value=*/13, /* size=*/100000},
+ NullPart{/* size= */ 1153},
+ RepeatPart{/* value=*/72, /* size=*/100799},
+ },
+ /* bit_width= */ 10,
+ },
+ {
+ {
+ RandomPart{/* max=*/1, /* size=*/1013, /* null_proba= */ 0.01},
+ NullPart{/* size=*/8},
+ RepeatPart{1, /* size= */ 256},
+ NullPart{/* size=*/128},
+ RepeatPart{0, /* size= */ 256},
+ NullPart{/* size=*/15},
+ RandomPart{/* max=*/1, /* size=*/8 * 1024, /* null_proba= */ 0.01},
+ },
+ /* bit_width= */ 1,
+ },
};
- for (auto case_ : int32_cases) {
- auto arr = rand.Int32(case_.size, /*min=*/0, case_.max_value, case_.null_probability);
- CheckRoundTripSpaced<Int32Type>(*arr, case_.bit_width);
- CheckRoundTripSpaced<Int32Type>(*arr->Slice(1), case_.bit_width);
+
+ ::arrow::random::RandomArrayGenerator rand(/* seed= */ 12);
+ // FRAGILE: Large enough so that it can be indexed by any value in all cases
+ auto dict = std::static_pointer_cast<arrow::FloatArray>(rand.Float32(20000, -1.0, 1.0));
+
+ for (auto case_ : test_cases) {
+ if (static_cast<std::size_t>(case_.bit_width) > sizeof(T)) {
+ continue;
+ }
+
+ auto array = case_.MakeArray(rand);
+
+ // Tests for GetBatch
+ CheckRoundTrip<ArrowType, false, 1>(*array, case_.bit_width);
+ CheckRoundTrip<ArrowType, false, 3>(*array, case_.bit_width);
+
+ // Tests for GetBatchSpaced
+ CheckRoundTrip<ArrowType, true, 1>(*array, case_.bit_width);
+ CheckRoundTrip<ArrowType, true, 7>(*array, case_.bit_width);
+ CheckRoundTrip<ArrowType, true, 1>(*array->Slice(1), case_.bit_width);
+
+ // Cannot test GetBatchWithDict with this method since unknown null values
+
+ // Tests for GetBatchWithDictSpaced
+ CheckRoundTrip<ArrowType, true, 1>(*array, case_.bit_width, dict);
+ CheckRoundTrip<ArrowType, true, 5>(*array, case_.bit_width, dict);
}
}
+TEST(RleBitPacked, GetBatchSpacedRoundtripUint16) {
+ DoTestGetBatchSpacedRoundtrip<uint16_t>();
Review Comment:
We already have a test for `int32_t` here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2349580245
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ ARROW_DCHECK_LT(values_read, batch_size);
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+ auto const read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
return values_read;
}
-template <typename T, typename RunType, typename Converter>
-inline int RleDecoder::GetSpaced(Converter converter, int batch_size, int null_count,
- const uint8_t* valid_bits, int64_t valid_bits_offset,
- T* out) {
- if (ARROW_PREDICT_FALSE(null_count == batch_size)) {
- converter.FillZero(out, out + batch_size);
- return batch_size;
+namespace internal {
+
+/// Utility class to safely handle values and null count without too error-prone
+/// verbosity.
+class BatchCounter {
+ public:
+ using size_type = int32_t;
+
+ [[nodiscard]] static constexpr BatchCounter FromBatchSizeAndNulls(
+ size_type batch_size, size_type null_count) {
+ ARROW_DCHECK_LE(null_count, batch_size);
+ return {batch_size - null_count, null_count};
}
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
- int values_remaining = batch_size - null_count;
+ constexpr BatchCounter(size_type values_count, size_type null_count) noexcept
+ : values_count_(values_count), null_count_(null_count) {}
- // Assume no bits to start.
- arrow::internal::BitRunReader bit_reader(valid_bits, valid_bits_offset,
- /*length=*/batch_size);
- arrow::internal::BitRun valid_run = bit_reader.NextRun();
- while (values_read < batch_size) {
- if (ARROW_PREDICT_FALSE(valid_run.length == 0)) {
- valid_run = bit_reader.NextRun();
+ [[nodiscard]] constexpr size_type ValuesCount() const noexcept { return values_count_; }
+
+ [[nodiscard]] constexpr size_type ValuesRead() const noexcept { return values_read_; }
+
+ [[nodiscard]] constexpr size_type ValuesRemaining() const noexcept {
+ ARROW_DCHECK_LE(values_read_, values_count_);
+ return values_count_ - values_read_;
+ }
+
+ constexpr void AccrueReadValues(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, ValuesRemaining());
+ values_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type NullCount() const noexcept { return null_count_; }
+
+ [[nodiscard]] constexpr size_type NullRead() const noexcept { return null_read_; }
+
+ [[nodiscard]] constexpr size_type NullRemaining() const noexcept {
+ ARROW_DCHECK_LE(null_read_, null_count_);
+ return null_count_ - null_read_;
+ }
+
+ constexpr void AccrueReadNulls(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, NullRemaining());
+ null_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type TotalRemaining() const noexcept {
+ return ValuesRemaining() + NullRemaining();
+ }
+
+ [[nodiscard]] constexpr size_type TotalRead() const noexcept {
+ return values_read_ + null_read_;
+ }
+
+ [[nodiscard]] constexpr bool IsFullyNull() const noexcept {
+ return ValuesRemaining() == 0;
+ }
+
+ [[nodiscard]] constexpr bool IsDone() const noexcept { return TotalRemaining() == 0; }
+
+ private:
+ size_type values_count_ = 0;
+ size_type values_read_ = 0;
+ size_type null_count_ = 0;
+ size_type null_read_ = 0;
+};
+
+// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+/// Overload for GetSpaced for a single run in a RleDecoder
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ RleDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto values_remaining_run = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // Consume as much as possible from the repeated run.
+ // We only need to count the number of nulls and non-nulls because we can fill in the
+ // same value for nulls and non-nulls.
+ // This proves to be a big efficiency win.
+ while (values_remaining_run() > 0 && !batch.IsDone()) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const& validity_run_size = static_cast<values_count_type>(validity_run.length);
+
+ if (validity_run.set) {
+ // We may end the current RLE run in the middle of the validity run
+ auto update_size = std::min(validity_run_size, values_remaining_run());
+ batch.AccrueReadValues(update_size);
+ validity_run.length -= update_size;
+ } else {
+ // We can consume all nulls here because it does not matter if we consume on this
+ // RLE run, or an a next encoded run. The value filled does not matter.
+ auto update_size = std::min(validity_run_size, batch.NullRemaining());
+ batch.AccrueReadNulls(update_size);
+ validity_run.length -= update_size;
+ }
+
+ if (ARROW_PREDICT_TRUE(validity_run.length == 0)) {
+ validity_run = validity_reader.NextRun();
}
+ }
+
+ value_type const value = decoder.Value();
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(value))) {
+ return {0, 0};
+ }
+ converter.WriteRepeated(out, out + batch.TotalRead(), value);
+ auto const actual_values_read = decoder.Advance(batch.ValuesRead());
+ // We always cropped the number of values_read by the remaining values in the run.
+ // What's more the RLE decoder should not encounter any errors.
+ ARROW_DCHECK_EQ(actual_values_read, batch.ValuesRead());
- ARROW_DCHECK_GT(batch_size, 0);
- ARROW_DCHECK_GT(valid_run.length, 0);
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+template <typename T, typename... Ts>
+[[nodiscard]] constexpr T min(T x, Ts... ys) {
+ ((x = std::min(x, ys)), ...);
+ return x;
+}
+
+static_assert(min(5) == 5);
+static_assert(min(5, 4, -1) == -1);
+static_assert(min(5, 41) == 5);
+
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ BitPackedDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto run_values_remaining = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ while (run_values_remaining() > 0 && batch.ValuesRemaining() > 0) {
+ // TODO should this size be tune depending on sizeof(value_size)? cpu cache size?
Review Comment:
`1024` was the [previous](https://github.com/apache/arrow/blob/eb6b637cc8857419ced5379dd5693c57dafa2458/cpp/src/arrow/util/rle_encoding_internal.h#L409) size.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2349077797
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -602,20 +1033,106 @@ struct GetBatchSpacedTestCase {
int bit_width;
};
-TEST(RleDecoder, GetBatchSpaced) {
- uint32_t kSeed = 1337;
- ::arrow::random::RandomArrayGenerator rand(kSeed);
-
- std::vector<GetBatchSpacedTestCase<int32_t>> int32_cases{
- {1, 100000, 0.01, 1}, {1, 100000, 0.1, 1}, {1, 100000, 0.5, 1},
- {4, 100000, 0.05, 3}, {100, 100000, 0.05, 7},
+template <typename T>
+void DoTestGetBatchSpacedRoundtrip() {
+ using Data = DataTestRleBitPacked<T>;
+ using ArrowType = typename Data::ArrowType;
+ using RandomPart = typename Data::RandomPart;
+ using NullPart = typename Data::NullPart;
+ using RepeatPart = typename Data::RepeatPart;
+
+ std::vector<Data> test_cases = {
+ {
+ {RandomPart{/* max=*/1, /* size=*/400, /* null_proba= */ 0.1}},
+ /* bit_width= */ 1,
+ },
+ {
+ {
+ RandomPart{/* max=*/7, /* size=*/10037, /* null_proba= */ 0.1},
+ NullPart{/* size= */ 1153},
+ RandomPart{/* max=*/7, /* size=*/800, /* null_proba= */ 0.5},
+ },
+ /* bit_width= */ 3,
+ },
+ {
+ {
+ NullPart{/* size= */ 80},
+ RandomPart{/* max=*/1023, /* size=*/800, /* null_proba= */ 0.01},
+ NullPart{/* size= */ 1023},
+ },
+ /* bit_width= */ 11,
+ },
+ {
+ {RepeatPart{/* value=*/13, /* size=*/100000}},
+ /* bit_width= */ 10,
+ },
+ {
+ {
+ NullPart{/* size= */ 1024},
+ RepeatPart{/* value=*/10000, /* size=*/100000},
+ NullPart{/* size= */ 77},
+ },
+ /* bit_width= */ 23,
+ },
+ {
+ {
+ RepeatPart{/* value=*/13, /* size=*/100000},
+ NullPart{/* size= */ 1153},
+ RepeatPart{/* value=*/72, /* size=*/100799},
+ },
+ /* bit_width= */ 10,
+ },
+ {
+ {
+ RandomPart{/* max=*/1, /* size=*/1013, /* null_proba= */ 0.01},
+ NullPart{/* size=*/8},
+ RepeatPart{1, /* size= */ 256},
+ NullPart{/* size=*/128},
+ RepeatPart{0, /* size= */ 256},
+ NullPart{/* size=*/15},
+ RandomPart{/* max=*/1, /* size=*/8 * 1024, /* null_proba= */ 0.01},
+ },
+ /* bit_width= */ 1,
+ },
};
- for (auto case_ : int32_cases) {
- auto arr = rand.Int32(case_.size, /*min=*/0, case_.max_value, case_.null_probability);
- CheckRoundTripSpaced<Int32Type>(*arr, case_.bit_width);
- CheckRoundTripSpaced<Int32Type>(*arr->Slice(1), case_.bit_width);
+
+ ::arrow::random::RandomArrayGenerator rand(/* seed= */ 12);
+ // FRAGILE: Large enough so that it can be indexed by any value in all cases
Review Comment:
I changed the comment to
```
// FRAGILE: we create a dictionary large enough so that any encoded value from the
// previous test cases can be used as an index in the dictionary.
// Its size must be increased accordingly if larger values are encoded in the test
// cases.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2349548583
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -84,32 +85,278 @@ namespace util {
/// (total 26 bytes, 1 byte overhead)
//
+class RleRun {
+ public:
+ using byte = uint8_t;
+ /// Enough space to store a 64bit value
+ using raw_data_storage = std::array<byte, 8>;
+ using raw_data_const_pointer = const byte*;
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr RleRun() noexcept = default;
+ constexpr RleRun(RleRun const&) noexcept = default;
+ constexpr RleRun(RleRun&&) noexcept = default;
+
+ explicit RleRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr RleRun& operator=(RleRun const&) noexcept = default;
+ constexpr RleRun& operator=(RleRun&&) noexcept = default;
+
+ /// The number of repeated values in this run.
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ /// A pointer to the repeated value raw bytes.
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ /// The number of bytes used for the raw repeated value.
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The repeated value raw bytes stored inside the class
+ raw_data_storage data_ = {};
+ /// The number of time the value is repeated
+ values_count_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+class BitPackedRun {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// According to the Parquet thrift definition the page size can be written into an
+ /// int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr BitPackedRun() noexcept = default;
+ constexpr BitPackedRun(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun(BitPackedRun&&) noexcept = default;
+
+ constexpr BitPackedRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr BitPackedRun& operator=(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun& operator=(BitPackedRun&&) noexcept = default;
+
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Number of values in this run.
+ raw_data_size_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+/// A parser that emits either a ``BitPackedRun`` or a ``RleRun``.
+class RleBitPackedParser {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// By Parquet thrift definition the page size can be written into an int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+ /// The different types of runs emitted by the parser
+ using dynamic_run_type = std::variant<RleRun, BitPackedRun>;
+
+ constexpr RleBitPackedParser() noexcept = default;
+
+ constexpr RleBitPackedParser(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr void Reset(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width_) noexcept;
+
+ /// Get the current run with a small parsing cost without advancing the iteration.
+ [[nodiscard]] std::optional<dynamic_run_type> Peek() const;
+
+ /// Move to the next run.
+ [[nodiscard]] bool Advance();
+
+ /// Advance and return the current run.
+ [[nodiscard]] std::optional<dynamic_run_type> Next();
+
+ /// Whether there is still runs to iterate over.
+ ///
+ /// WARN: Due to lack of proper error handling, iteration with Next and Peek could
+ /// return not data while the parser is not exhausted.
+ /// This is how one can check for errors.
+ [[nodiscard]] bool Exhausted() const;
+
+ /// Enum to return from an ``Parse`` handler.
+ ///
+ /// Since a callback has no way to know when to stop, the handler must return
+ /// a value indicating to the ``Parse`` function whether to stop or continue.
+ enum class ControlFlow {
+ Continue,
+ Break,
+ };
+
+ /// A callback approach to parsing.
+ ///
+ /// This approach is used to reduce the number of dynamic lookups involved with using a
+ /// variant.
+ ///
+ /// The handler must be of the form
+ /// ```cpp`
+ /// struct Handler {
+ /// ControlFlow OnBitPackedRun(BitPackedRun run);
+ ///
+ /// ControlFlow OnRleRun(RleRun run);
+ /// };
+ /// ```
+ template <typename Handler>
+ void Parse(Handler&& handler);
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Size in bytes of the run.
+ raw_data_size_type data_size_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+
+ /// Run the handler on the run read and return the number of values read.
+ /// Does not advance the parser.
+ template <typename Handler>
+ std::pair<raw_data_size_type, ControlFlow> PeekImpl(Handler&&) const;
+};
+
/// Decoder class for RLE encoded data.
+template <typename T>
class RleDecoder {
public:
- /// Create a decoder object. buffer/buffer_len is the decoded data.
- /// bit_width is the width of each value (before encoding).
- RleDecoder(const uint8_t* buffer, int buffer_len, int bit_width)
- : bit_reader_(buffer, buffer_len),
- bit_width_(bit_width),
- current_value_(0),
- repeat_count_(0),
- literal_count_(0) {
- ARROW_DCHECK_GE(bit_width_, 0);
- ARROW_DCHECK_LE(bit_width_, 64);
- }
+ /// The type in which the data should be decoded.
+ using value_type = T;
+ /// The type of run that can be decoded.
+ using run_type = RleRun;
+ using values_count_type = run_type::values_count_type;
Review Comment:
Though it's mostly a matter of preference, I agree that we ended up with too many aliases.
They were useful in tracking the many `int`/`int32` *etc*. in the previous implementation.
The thing I think that remain very useful in reading in the future is answering questions like "*why does this only has 32 bits?*", "*Why is it signed?*"...
We could keep a few namespace level alias like `using RleBitPackedSize = int32_t;` along with their documentation. What do you think?
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -84,32 +85,278 @@ namespace util {
/// (total 26 bytes, 1 byte overhead)
//
+class RleRun {
+ public:
+ using byte = uint8_t;
+ /// Enough space to store a 64bit value
+ using raw_data_storage = std::array<byte, 8>;
+ using raw_data_const_pointer = const byte*;
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr RleRun() noexcept = default;
+ constexpr RleRun(RleRun const&) noexcept = default;
+ constexpr RleRun(RleRun&&) noexcept = default;
+
+ explicit RleRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr RleRun& operator=(RleRun const&) noexcept = default;
+ constexpr RleRun& operator=(RleRun&&) noexcept = default;
+
+ /// The number of repeated values in this run.
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ /// A pointer to the repeated value raw bytes.
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ /// The number of bytes used for the raw repeated value.
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The repeated value raw bytes stored inside the class
+ raw_data_storage data_ = {};
+ /// The number of time the value is repeated
+ values_count_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+class BitPackedRun {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// According to the Parquet thrift definition the page size can be written into an
+ /// int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr BitPackedRun() noexcept = default;
+ constexpr BitPackedRun(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun(BitPackedRun&&) noexcept = default;
+
+ constexpr BitPackedRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr BitPackedRun& operator=(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun& operator=(BitPackedRun&&) noexcept = default;
+
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Number of values in this run.
+ raw_data_size_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+/// A parser that emits either a ``BitPackedRun`` or a ``RleRun``.
+class RleBitPackedParser {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// By Parquet thrift definition the page size can be written into an int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+ /// The different types of runs emitted by the parser
+ using dynamic_run_type = std::variant<RleRun, BitPackedRun>;
+
+ constexpr RleBitPackedParser() noexcept = default;
+
+ constexpr RleBitPackedParser(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr void Reset(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width_) noexcept;
+
+ /// Get the current run with a small parsing cost without advancing the iteration.
+ [[nodiscard]] std::optional<dynamic_run_type> Peek() const;
+
+ /// Move to the next run.
+ [[nodiscard]] bool Advance();
+
+ /// Advance and return the current run.
+ [[nodiscard]] std::optional<dynamic_run_type> Next();
+
+ /// Whether there is still runs to iterate over.
+ ///
+ /// WARN: Due to lack of proper error handling, iteration with Next and Peek could
+ /// return not data while the parser is not exhausted.
+ /// This is how one can check for errors.
+ [[nodiscard]] bool Exhausted() const;
+
+ /// Enum to return from an ``Parse`` handler.
+ ///
+ /// Since a callback has no way to know when to stop, the handler must return
+ /// a value indicating to the ``Parse`` function whether to stop or continue.
+ enum class ControlFlow {
+ Continue,
+ Break,
+ };
+
+ /// A callback approach to parsing.
+ ///
+ /// This approach is used to reduce the number of dynamic lookups involved with using a
+ /// variant.
+ ///
+ /// The handler must be of the form
+ /// ```cpp`
+ /// struct Handler {
+ /// ControlFlow OnBitPackedRun(BitPackedRun run);
+ ///
+ /// ControlFlow OnRleRun(RleRun run);
+ /// };
+ /// ```
+ template <typename Handler>
+ void Parse(Handler&& handler);
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Size in bytes of the run.
+ raw_data_size_type data_size_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+
+ /// Run the handler on the run read and return the number of values read.
+ /// Does not advance the parser.
+ template <typename Handler>
+ std::pair<raw_data_size_type, ControlFlow> PeekImpl(Handler&&) const;
+};
+
/// Decoder class for RLE encoded data.
+template <typename T>
class RleDecoder {
public:
- /// Create a decoder object. buffer/buffer_len is the decoded data.
- /// bit_width is the width of each value (before encoding).
- RleDecoder(const uint8_t* buffer, int buffer_len, int bit_width)
- : bit_reader_(buffer, buffer_len),
- bit_width_(bit_width),
- current_value_(0),
- repeat_count_(0),
- literal_count_(0) {
- ARROW_DCHECK_GE(bit_width_, 0);
- ARROW_DCHECK_LE(bit_width_, 64);
- }
+ /// The type in which the data should be decoded.
+ using value_type = T;
+ /// The type of run that can be decoded.
+ using run_type = RleRun;
+ using values_count_type = run_type::values_count_type;
Review Comment:
Although it's mostly a matter of preference, I agree that we ended up with too many aliases.
They were useful in tracking the many `int`/`int32` *etc*. in the previous implementation.
The thing I think that remain very useful in reading in the future is answering questions like "*why does this only has 32 bits?*", "*Why is it signed?*"...
We could keep a few namespace level alias like `using RleBitPackedSize = int32_t;` along with their documentation. What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2349465815
##########
cpp/src/arrow/util/bit_util.h:
##########
@@ -365,5 +365,118 @@ void PackBits(const uint32_t* values, uint8_t* out) {
}
}
+constexpr int64_t MaxLEB128ByteLen(int64_t n_bits) { return CeilDiv(n_bits, 7); }
+
+template <typename Int>
+constexpr int64_t MaxLEB128ByteLenFor = MaxLEB128ByteLen(sizeof(Int) * 8);
+
+/// Write a integer as LEB128
+///
+/// Write the input value as LEB128 into the outptu buffer and return the number of bytes
+/// written.
+/// If the output buffer size is insufficient, return 0 but the output may have been
+/// written to.
+///
+/// \see https://en.wikipedia.org/wiki/LEB128
+/// \see MaxLEB128ByteLenFor
+template <typename Int>
+constexpr int32_t WriteLEB128(Int value, uint8_t* out, int32_t max_out_size) {
+ constexpr Int kLow7Mask = Int(0x7F);
+ constexpr Int kHigh7Mask = ~kLow7Mask;
+ constexpr uint8_t kContinuationBit = 0x80;
+
+ auto const out_first = out;
+
+ // Write as many bytes as the could be for the given input
+ while ((value & kHigh7Mask) != Int(0)) {
+ // We do not have enough room to write the LEB128
+ if (out - out_first >= max_out_size) {
Review Comment:
I had done a branch-less version of the LEB128 reading for in32 on the fast path that we can read all needed bytes at once. Turned out to be slower.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2355368675
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -602,20 +1038,126 @@ struct GetBatchSpacedTestCase {
int bit_width;
};
-TEST(RleDecoder, GetBatchSpaced) {
- uint32_t kSeed = 1337;
- ::arrow::random::RandomArrayGenerator rand(kSeed);
-
- std::vector<GetBatchSpacedTestCase<int32_t>> int32_cases{
- {1, 100000, 0.01, 1}, {1, 100000, 0.1, 1}, {1, 100000, 0.5, 1},
- {4, 100000, 0.05, 3}, {100, 100000, 0.05, 7},
+template <typename T>
+void DoTestGetBatchSpacedRoundtrip() {
+ using Data = DataTestRleBitPacked<T>;
+ using ArrowType = typename Data::ArrowType;
+ using RandomPart = typename Data::RandomPart;
+ using NullPart = typename Data::NullPart;
+ using RepeatPart = typename Data::RepeatPart;
+
+ std::vector<Data> test_cases = {
+ {
+ {RandomPart{/* max=*/1, /* size=*/400, /* null_proba= */ 0.1}},
+ /* bit_width= */ 1,
+ },
+ {
+ {
+ RandomPart{/* max=*/7, /* size=*/10037, /* null_proba= */ 0.0},
+ NullPart{/* size= */ 1153},
+ RandomPart{/* max=*/7, /* size=*/800, /* null_proba= */ 0.5},
+ },
+ /* bit_width= */ 3,
+ },
+ {
+ {
+ NullPart{/* size= */ 80},
+ RandomPart{/* max=*/static_cast<T>(1023), /* size=*/800,
+ /* null_proba= */ 0.01},
+ NullPart{/* size= */ 1023},
+ },
+ /* bit_width= */ 11,
+ },
+ {
+ {RepeatPart{/* value=*/13, /* size=*/100000, /* null_proba= */ 0.01}},
+ /* bit_width= */ 10,
+ },
+ {
+ {
+ NullPart{/* size= */ 1024},
+ RepeatPart{/* value=*/static_cast<T>(10000), /* size=*/100000,
+ /* null_proba= */ 0.1},
+ NullPart{/* size= */ 77},
+ },
+ /* bit_width= */ 23,
+ },
+ {
+ {
+ RepeatPart{/* value=*/13, /* size=*/100000, /* null_proba= */ 0.0},
+ NullPart{/* size= */ 1153},
+ RepeatPart{/* value=*/72, /* size=*/100799, /* null_proba= */ 0.5},
+ },
+ /* bit_width= */ 10,
+ },
+ {
+ {
+ RandomPart{/* max=*/1, /* size=*/1013, /* null_proba= */ 0.01},
+ NullPart{/* size=*/8},
+ RepeatPart{1, /* size= */ 256, /* null_proba= */ 0.1},
+ NullPart{/* size=*/128},
+ RepeatPart{0, /* size= */ 256, /* null_proba= */ 0.0},
+ NullPart{/* size=*/15},
+ RandomPart{/* max=*/1, /* size=*/8 * 1024, /* null_proba= */ 0.01},
+ },
+ /* bit_width= */ 1,
+ },
};
- for (auto case_ : int32_cases) {
- auto arr = rand.Int32(case_.size, /*min=*/0, case_.max_value, case_.null_probability);
- CheckRoundTripSpaced<Int32Type>(*arr, case_.bit_width);
- CheckRoundTripSpaced<Int32Type>(*arr->Slice(1), case_.bit_width);
+
+ ::arrow::random::RandomArrayGenerator rand(/* seed= */ 12);
+ // FRAGILE: we create a dictionary large enough so that any encoded value from the
+ // previous test cases can be used as an index in the dictionary.
+ // Its size must be increased accordingly if larger values are encoded in the test
+ // cases.
+ auto dict = std::static_pointer_cast<arrow::FloatArray>(rand.Float32(20000, -1.0, 1.0));
+
+ // Number of bits available in T to write a positive integer.
+ constexpr int kBitsAvailable = 8 * sizeof(T) - (std::is_signed_v<T> ? 1 : 0);
Review Comment:
Oh, yes, you're right, sorry.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #47294:
URL: https://github.com/apache/arrow/pull/47294#issuecomment-3299464450
@ursabot please benchmark --help
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "conbench-apache-arrow[bot] (via GitHub)" <gi...@apache.org>.
conbench-apache-arrow[bot] commented on PR #47294:
URL: https://github.com/apache/arrow/pull/47294#issuecomment-3329887895
After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 79f97640d0d79af928a300cb53c5ce6fad3dabe4.
There were no benchmark performance regressions. 🎉
The [full Conbench report](https://github.com/apache/arrow/runs/51157326214) has more details. It also includes information about 5 possible false positives for unstable benchmarks that are known to sometimes produce them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2367097752
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -82,86 +81,400 @@ namespace util {
/// 200 ints = 25 groups of 8
/// <varint((25 << 1) | 1)> <25 bytes of values, bitpacked>
/// (total 26 bytes, 1 byte overhead)
-//
-/// Decoder class for RLE encoded data.
-class RleDecoder {
+/// The type for an encoded Rle of BitPacked run size, between 1 and 2^31-1 as per Parquet
+/// spec.
+/// This is also pragmatically used for other integer used in the Rle and BitPacked runs
+/// and decoder to avoid conversions.
+/// It can therefore be referred to as a "typical" size for Rle and BitPacked logic.
+using rle_size_t = int32_t;
+
+template <typename T>
+class RleRunDecoder;
+
+class RleRun {
public:
- /// Create a decoder object. buffer/buffer_len is the decoded data.
- /// bit_width is the width of each value (before encoding).
- RleDecoder(const uint8_t* buffer, int buffer_len, int bit_width)
- : bit_reader_(buffer, buffer_len),
- bit_width_(bit_width),
- current_value_(0),
- repeat_count_(0),
- literal_count_(0) {
- ARROW_DCHECK_GE(bit_width_, 0);
- ARROW_DCHECK_LE(bit_width_, 64);
+ /// The decoder class used to decode a single run in the given type.
+ template <typename T>
+ using DecoderType = RleRunDecoder<T>;
+
+ constexpr RleRun() noexcept = default;
+
+ explicit RleRun(const uint8_t* data, rle_size_t values_count,
+ rle_size_t value_bit_width) noexcept
+ : values_count_(values_count), value_bit_width_(value_bit_width) {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_GE(values_count, 0);
+ std::copy(data, data + raw_data_size(), data_.begin());
}
- RleDecoder() : bit_width_(-1) {}
+ /// The number of repeated values in this run.
+ constexpr rle_size_t values_count() const noexcept { return values_count_; }
- void Reset(const uint8_t* buffer, int buffer_len, int bit_width) {
- ARROW_DCHECK_GE(bit_width, 0);
- ARROW_DCHECK_LE(bit_width, 64);
- bit_reader_.Reset(buffer, buffer_len);
- bit_width_ = bit_width;
- current_value_ = 0;
- repeat_count_ = 0;
- literal_count_ = 0;
+ /// The size in bits of each encoded value.
+ constexpr rle_size_t values_bit_width() const noexcept { return value_bit_width_; }
+
+ /// A pointer to the repeated value raw bytes.
+ constexpr const uint8_t* raw_data_ptr() const noexcept { return data_.data(); }
+
+ /// The number of bytes used for the raw repeated value.
+ constexpr rle_size_t raw_data_size() const noexcept {
+ auto out = bit_util::BytesForBits(value_bit_width_);
+ ARROW_DCHECK_LE(out, std::numeric_limits<rle_size_t>::max());
+ return static_cast<rle_size_t>(out);
}
- /// Gets the next value. Returns false if there are no more.
+ private:
+ /// The repeated value raw bytes stored inside the class with enough space to store
+ /// up to a 64 bit value.
+ std::array<uint8_t, 8> data_ = {};
+ /// The number of time the value is repeated.
+ rle_size_t values_count_ = 0;
+ /// The size in bit of a packed value in the run.
+ rle_size_t value_bit_width_ = 0;
Review Comment:
I think the main question for me is whether the optimizer is able to deduce that the bit width is constant across the entire decoding process. If it is, then fine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2336677731
##########
cpp/src/arrow/util/bit_util.h:
##########
@@ -365,5 +365,118 @@ void PackBits(const uint32_t* values, uint8_t* out) {
}
}
+constexpr int64_t MaxLEB128ByteLen(int64_t n_bits) { return CeilDiv(n_bits, 7); }
+
+template <typename Int>
+constexpr int64_t MaxLEB128ByteLenFor = MaxLEB128ByteLen(sizeof(Int) * 8);
+
+/// Write a integer as LEB128
+///
+/// Write the input value as LEB128 into the outptu buffer and return the number of bytes
+/// written.
+/// If the output buffer size is insufficient, return 0 but the output may have been
+/// written to.
+///
+/// \see https://en.wikipedia.org/wiki/LEB128
+/// \see MaxLEB128ByteLenFor
+template <typename Int>
+constexpr int32_t WriteLEB128(Int value, uint8_t* out, int32_t max_out_size) {
+ constexpr Int kLow7Mask = Int(0x7F);
+ constexpr Int kHigh7Mask = ~kLow7Mask;
+ constexpr uint8_t kContinuationBit = 0x80;
+
+ auto const out_first = out;
+
+ // Write as many bytes as the could be for the given input
+ while ((value & kHigh7Mask) != Int(0)) {
+ // We do not have enough room to write the LEB128
+ if (out - out_first >= max_out_size) {
+ return 0;
+ }
+
+ // Write the encoded byte with continuation bit
+ *out = static_cast<uint8_t>(value & kLow7Mask) | kContinuationBit;
+ ++out;
+ // Shift remaining data
+ value >>= 7;
Review Comment:
It need not be unsigned but must be non negative. This is handled explicitly now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2322154575
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -553,47 +853,178 @@ TEST(BitRle, Overflow) {
}
}
-template <typename Type>
-void CheckRoundTripSpaced(const Array& data, int bit_width) {
+/// Check RleBitPacked encoding/decoding round trip.
+///
+/// \tparam kSpaced If set to false, treat Nulls in the input array as regular data.
+/// \tparam kParts The number of parts in which the data will be decoded.
+/// For number greater than one, this ensure that the decoder intermediary state
+/// is valid.
+template <typename Type, bool kSpaced, int32_t kParts>
+void CheckRoundTrip(const Array& data, int bit_width,
+ std::shared_ptr<FloatArray> dict = {}) {
using ArrayType = typename TypeTraits<Type>::ArrayType;
- using T = typename Type::c_type;
+ using value_type = typename Type::c_type;
- int num_values = static_cast<int>(data.length());
- int buffer_size = RleEncoder::MaxBufferSize(bit_width, num_values);
+ int const data_size = static_cast<int>(data.length());
+ int const data_values_count =
+ static_cast<int>(data.length() - kSpaced * data.null_count());
+ int const buffer_size = RleBitPackedEncoder::MaxBufferSize(bit_width, data_size);
+ ASSERT_GE(kParts, 1);
+ ASSERT_LE(kParts, data_size);
- const T* values = static_cast<const ArrayType&>(data).raw_values();
+ const value_type* data_values = static_cast<const ArrayType&>(data).raw_values();
+ // Encode the data into ``buffer`` using the encoder.
std::vector<uint8_t> buffer(buffer_size);
- RleEncoder encoder(buffer.data(), buffer_size, bit_width);
- for (int i = 0; i < num_values; ++i) {
- if (data.IsValid(i)) {
- if (!encoder.Put(static_cast<uint64_t>(values[i]))) {
- FAIL() << "Encoding failed";
- }
+ RleBitPackedEncoder encoder(buffer.data(), buffer_size, bit_width);
+ int32_t encoded_values_size = 0;
+ for (int i = 0; i < data_size; ++i) {
+ // Depending on kSpaced we treat nulls as regular values.
+ if (data.IsValid(i) || !kSpaced) {
+ bool success = encoder.Put(static_cast<uint64_t>(data_values[i]));
+ ASSERT_TRUE(success) << "Encoding failed in pos " << i;
+ ++encoded_values_size;
}
}
- int encoded_size = encoder.Flush();
+ int encoded_byte_size = encoder.Flush();
+ ASSERT_EQ(encoded_values_size, data_values_count)
+ << "All values input were not encoded successfully by the encoder";
+
+ // On to verify batch read
+ RleBitPackedDecoder<value_type> decoder(buffer.data(), encoded_byte_size, bit_width);
+ // We will only use one of them depending on whether this is a dictonnary tests
+ std::vector<float> dict_read;
+ std::vector<value_type> values_read;
+ if (dict) {
+ dict_read.resize(data_size);
+ } else {
+ values_read.resize(data_size);
+ }
- // Verify batch read
- RleDecoder decoder(buffer.data(), encoded_size, bit_width);
- std::vector<T> values_read(num_values);
+ // We will read the data in kParts calls to make sure intermediate states are valid
+ int32_t actual_read_count = 0;
+ int32_t requested_read_count = 0;
+ while (requested_read_count < data_size) {
+ auto const remaining = data_size - requested_read_count;
+ auto to_read = data_size / kParts;
+ if (remaining / to_read == 1) {
+ to_read = remaining;
+ }
- if (num_values != decoder.GetBatchSpaced(
- num_values, static_cast<int>(data.null_count()),
- data.null_bitmap_data(), data.offset(), values_read.data())) {
- FAIL();
- }
+ auto read = 0;
+ if constexpr (kSpaced) {
+ // We need to slice the input array get the proper null count and bitmap
+ auto data_remaining = data.Slice(requested_read_count, to_read);
+
+ if (dict) {
+ auto* out = dict_read.data() + requested_read_count;
+ read = decoder.GetBatchWithDictSpaced(
+ dict->raw_values(), static_cast<int32_t>(dict->length()), out, to_read,
+ static_cast<int32_t>(data_remaining->null_count()),
+ data_remaining->null_bitmap_data(), data_remaining->offset());
+ } else {
+ auto* out = values_read.data() + requested_read_count;
+ read = decoder.GetBatchSpaced(
+ to_read, static_cast<int32_t>(data_remaining->null_count()),
+ data_remaining->null_bitmap_data(), data_remaining->offset(), out);
+ }
+ } else {
+ if (dict) {
+ auto* out = dict_read.data() + requested_read_count;
+ read = decoder.GetBatchWithDict(
+ dict->raw_values(), static_cast<int32_t>(dict->length()), out, to_read);
+ } else {
+ auto* out = values_read.data() + requested_read_count;
+ read = decoder.GetBatch(out, to_read);
+ }
+ }
+ ASSERT_EQ(read, to_read) << "Decoder did not read as many values as requested";
- for (int64_t i = 0; i < num_values; ++i) {
- if (data.IsValid(i)) {
- if (values_read[i] != values[i]) {
- FAIL() << "Index " << i << " read " << values_read[i] << " but should be "
- << values[i];
+ actual_read_count += read;
+ requested_read_count += to_read;
+ }
+ EXPECT_EQ(requested_read_count, data_size) << "This test logic is wrong";
+ EXPECT_EQ(actual_read_count, data_size) << "Total number of values read is off";
+
+ // Verify the round trip: encoded-decoded values must equal the original one
+ for (int64_t i = 0; i < data_size; ++i) {
+ if (data.IsValid(i) || !kSpaced) {
+ if (dict) {
+ EXPECT_EQ(dict_read.at(i), dict->Value(data_values[i]))
+ << "Encoded then decoded and mapped value at position " << i << " ("
+ << values_read[i] << ") differs from original value (" << data_values[i]
+ << " mapped to " << dict->Value(data_values[i]) << ")";
+ } else {
+ EXPECT_EQ(values_read.at(i), data_values[i])
+ << "Encoded then decoded value at position " << i << " (" << values_read.at(i)
+ << ") differs from original value (" << data_values[i] << ")";
}
}
}
}
+template <typename T>
+struct DataTestRleBitPackedRandomPart {
+ using value_type = T;
+
+ value_type max;
+ int32_t size;
+ double null_probability;
+};
+
+template <typename T>
+struct DataTestRleBitPackedRepeatPart {
+ using value_type = T;
+
+ value_type value;
+ int32_t size;
+};
+
+template <typename T>
+struct DataTestRleBitPackedNullPart {
+ using value_type = T;
+
+ int32_t size;
+};
+
+template <typename T>
+struct DataTestRleBitPacked {
+ using value_type = T;
+ using ArrowType = typename arrow::CTypeTraits<value_type>::ArrowType;
+ using RandomPart = DataTestRleBitPackedRandomPart<value_type>;
+ using RepeatPart = DataTestRleBitPackedRepeatPart<value_type>;
+ using NullPart = DataTestRleBitPackedNullPart<value_type>;
+
+ std::vector<std::variant<RandomPart, RepeatPart, NullPart>> parts;
+ int32_t bit_width;
+
+ std::shared_ptr<::arrow::Array> MakeArray(
+ ::arrow::random::RandomArrayGenerator& rand) const {
+ std::vector<std::shared_ptr<::arrow::Array>> arrays = {};
+
+ for (auto const& dyn_part : parts) {
+ if (auto* part = std::get_if<RandomPart>(&dyn_part)) {
+ auto arr = rand.Numeric<ArrowType>(part->size, /* min= */ value_type(0),
+ part->max, part->null_probability);
+ arrays.push_back(std::move(arr));
+
+ } else if (auto* part = std::get_if<RepeatPart>(&dyn_part)) {
+ auto scalar = ::arrow::MakeScalar(part->value);
+ arrays.push_back(::arrow::MakeArrayFromScalar(*scalar, part->size).ValueOrDie());
+
+ } else if (auto* part = std::get_if<NullPart>(&dyn_part)) {
+ using Traits = arrow::TypeTraits<ArrowType>;
+ auto null_scalar = ::arrow::MakeNullScalar(Traits::type_singleton());
+ arrays.push_back(
+ ::arrow::MakeArrayFromScalar(*null_scalar, part->size).ValueOrDie());
Review Comment:
You can simply use `MakeArrayOfNull`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2319463614
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ ARROW_DCHECK_LT(values_read, batch_size);
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+ auto const read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
return values_read;
}
-template <typename T, typename RunType, typename Converter>
-inline int RleDecoder::GetSpaced(Converter converter, int batch_size, int null_count,
- const uint8_t* valid_bits, int64_t valid_bits_offset,
- T* out) {
- if (ARROW_PREDICT_FALSE(null_count == batch_size)) {
- converter.FillZero(out, out + batch_size);
- return batch_size;
+namespace internal {
+
+/// Utility class to safely handle values and null count without too error-prone
+/// verbosity.
+class BatchCounter {
+ public:
+ using size_type = int32_t;
+
+ [[nodiscard]] static constexpr BatchCounter FromBatchSizeAndNulls(
+ size_type batch_size, size_type null_count) {
+ ARROW_DCHECK_LE(null_count, batch_size);
+ return {batch_size - null_count, null_count};
}
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
- int values_remaining = batch_size - null_count;
+ constexpr BatchCounter(size_type values_count, size_type null_count) noexcept
+ : values_count_(values_count), null_count_(null_count) {}
- // Assume no bits to start.
- arrow::internal::BitRunReader bit_reader(valid_bits, valid_bits_offset,
- /*length=*/batch_size);
- arrow::internal::BitRun valid_run = bit_reader.NextRun();
- while (values_read < batch_size) {
- if (ARROW_PREDICT_FALSE(valid_run.length == 0)) {
- valid_run = bit_reader.NextRun();
+ [[nodiscard]] constexpr size_type ValuesCount() const noexcept { return values_count_; }
+
+ [[nodiscard]] constexpr size_type ValuesRead() const noexcept { return values_read_; }
+
+ [[nodiscard]] constexpr size_type ValuesRemaining() const noexcept {
+ ARROW_DCHECK_LE(values_read_, values_count_);
+ return values_count_ - values_read_;
+ }
+
+ constexpr void AccrueReadValues(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, ValuesRemaining());
+ values_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type NullCount() const noexcept { return null_count_; }
+
+ [[nodiscard]] constexpr size_type NullRead() const noexcept { return null_read_; }
+
+ [[nodiscard]] constexpr size_type NullRemaining() const noexcept {
+ ARROW_DCHECK_LE(null_read_, null_count_);
+ return null_count_ - null_read_;
+ }
+
+ constexpr void AccrueReadNulls(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, NullRemaining());
+ null_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type TotalRemaining() const noexcept {
+ return ValuesRemaining() + NullRemaining();
+ }
+
+ [[nodiscard]] constexpr size_type TotalRead() const noexcept {
+ return values_read_ + null_read_;
+ }
+
+ [[nodiscard]] constexpr bool IsFullyNull() const noexcept {
+ return ValuesRemaining() == 0;
+ }
+
+ [[nodiscard]] constexpr bool IsDone() const noexcept { return TotalRemaining() == 0; }
+
+ private:
+ size_type values_count_ = 0;
+ size_type values_read_ = 0;
+ size_type null_count_ = 0;
+ size_type null_read_ = 0;
+};
+
+// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+/// Overload for GetSpaced for a single run in a RleDecoder
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ RleDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto values_remaining_run = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // Consume as much as possible from the repeated run.
+ // We only need to count the number of nulls and non-nulls because we can fill in the
+ // same value for nulls and non-nulls.
+ // This proves to be a big efficiency win.
+ while (values_remaining_run() > 0 && !batch.IsDone()) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const& validity_run_size = static_cast<values_count_type>(validity_run.length);
+
+ if (validity_run.set) {
+ // We may end the current RLE run in the middle of the validity run
+ auto update_size = std::min(validity_run_size, values_remaining_run());
+ batch.AccrueReadValues(update_size);
+ validity_run.length -= update_size;
+ } else {
+ // We can consume all nulls here because it does not matter if we consume on this
+ // RLE run, or an a next encoded run. The value filled does not matter.
+ auto update_size = std::min(validity_run_size, batch.NullRemaining());
+ batch.AccrueReadNulls(update_size);
+ validity_run.length -= update_size;
+ }
+
+ if (ARROW_PREDICT_TRUE(validity_run.length == 0)) {
+ validity_run = validity_reader.NextRun();
}
+ }
+
+ value_type const value = decoder.Value();
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(value))) {
+ return {0, 0};
+ }
+ converter.WriteRepeated(out, out + batch.TotalRead(), value);
+ auto const actual_values_read = decoder.Advance(batch.ValuesRead());
+ // We always cropped the number of values_read by the remaining values in the run.
+ // What's more the RLE decoder should not encounter any errors.
+ ARROW_DCHECK_EQ(actual_values_read, batch.ValuesRead());
- ARROW_DCHECK_GT(batch_size, 0);
- ARROW_DCHECK_GT(valid_run.length, 0);
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+template <typename T, typename... Ts>
+[[nodiscard]] constexpr T min(T x, Ts... ys) {
+ ((x = std::min(x, ys)), ...);
+ return x;
+}
+
+static_assert(min(5) == 5);
+static_assert(min(5, 4, -1) == -1);
+static_assert(min(5, 41) == 5);
+
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ BitPackedDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto run_values_remaining = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ while (run_values_remaining() > 0 && batch.ValuesRemaining() > 0) {
+ // TODO should this size be tune depending on sizeof(value_size)? cpu cache size?
+ // Pull a batch of values from the bit packed encoded data and store it in a local
+ // buffer to benefit from unpacking intrinsics and data locality.
+ static constexpr values_count_type kBufferCapacity = 1024;
+ std::array<value_type, kBufferCapacity> buffer = {};
+
+ values_count_type buffer_start = 0;
+ values_count_type buffer_end = 0;
+ auto buffer_size = [&]() {
+ auto out = buffer_end - buffer_start;
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // buffer_start is 0 at this point so size is end
+ buffer_end = min(run_values_remaining(), batch.ValuesRemaining(), kBufferCapacity);
+ buffer_end = decoder.GetBatch(buffer.data(), buffer_size());
+ ARROW_DCHECK_LE(buffer_size(), kBufferCapacity);
+
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(buffer.data(), buffer_size()))) {
+ return {batch.ValuesRead(), batch.NullRead()};
+ }
- if (valid_run.set) {
- if ((repeat_count_ == 0) && (literal_count_ == 0)) {
- if (!NextCounts<RunType>()) return values_read;
- ARROW_DCHECK((repeat_count_ > 0) ^ (literal_count_ > 0));
+ // Copy chunks of valid values into the output, while adjusting spacing for null
+ // values.
+ while (buffer_size() > 0) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const validity_run_length =
+ static_cast<values_count_type>(validity_run.length);
+
+ // Copy as much as possible from the buffer into the output while not exceeding
+ // validity run
+ if (validity_run.set) {
+ auto const update_size = std::min(validity_run_length, buffer_size());
+ converter.WriteRange(out, buffer.data() + buffer_start, update_size);
+ buffer_start += update_size;
+ batch.AccrueReadValues(update_size);
+ out += update_size;
+ validity_run.length -= update_size;
+ // Simply write zeros in the output
+ } else {
+ auto const update_size = std::min(validity_run_length, batch.NullRemaining());
+ converter.WriteZero(out, out + update_size);
+ batch.AccrueReadNulls(update_size);
+ out += update_size;
+ validity_run.length -= update_size;
}
- if (repeat_count_ > 0) {
- int repeat_batch = 0;
- // Consume the entire repeat counts incrementing repeat_batch to
- // be the total of nulls + values consumed, we only need to
- // get the total count because we can fill in the same value for
- // nulls and non-nulls. This proves to be a big efficiency win.
- while (repeat_count_ > 0 && (values_read + repeat_batch) < batch_size) {
- ARROW_DCHECK_GT(valid_run.length, 0);
- if (valid_run.set) {
- int update_size = std::min(static_cast<int>(valid_run.length), repeat_count_);
- repeat_count_ -= update_size;
- repeat_batch += update_size;
- valid_run.length -= update_size;
- values_remaining -= update_size;
- } else {
- // We can consume all nulls here because we would do so on
- // the next loop anyways.
- repeat_batch += static_cast<int>(valid_run.length);
- valid_run.length = 0;
- }
- if (valid_run.length == 0) {
- valid_run = bit_reader.NextRun();
- }
- }
- RunType current_value = static_cast<RunType>(current_value_);
- if (ARROW_PREDICT_FALSE(!converter.IsValid(current_value))) {
- return values_read;
- }
- converter.Fill(out, out + repeat_batch, current_value);
- out += repeat_batch;
- values_read += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(values_remaining, literal_count_);
- ARROW_DCHECK_GT(literal_batch, 0);
-
- // Decode the literals
- constexpr int kBufferSize = 1024;
- RunType indices[kBufferSize];
- literal_batch = std::min(literal_batch, kBufferSize);
- int actual_read = bit_reader_.GetBatch(bit_width_, indices, literal_batch);
- if (ARROW_PREDICT_FALSE(actual_read != literal_batch)) {
- return values_read;
- }
- if (!converter.IsValid(indices, /*length=*/actual_read)) {
- return values_read;
- }
- int skipped = 0;
- int literals_read = 0;
- while (literals_read < literal_batch) {
- if (valid_run.set) {
- int update_size = std::min(literal_batch - literals_read,
- static_cast<int>(valid_run.length));
- converter.Copy(out, indices + literals_read, update_size);
- literals_read += update_size;
- out += update_size;
- valid_run.length -= update_size;
- } else {
- converter.FillZero(out, out + valid_run.length);
- out += valid_run.length;
- skipped += static_cast<int>(valid_run.length);
- valid_run.length = 0;
- }
- if (valid_run.length == 0) {
- valid_run = bit_reader.NextRun();
- }
- }
- literal_count_ -= literal_batch;
- values_remaining -= literal_batch;
- values_read += literal_batch + skipped;
+ if (validity_run.length == 0) {
+ validity_run = validity_reader.NextRun();
}
- } else {
- converter.FillZero(out, out + valid_run.length);
- out += valid_run.length;
- values_read += static_cast<int>(valid_run.length);
- valid_run.length = 0;
}
+
+ ARROW_DCHECK_EQ(buffer_size(), 0);
}
- ARROW_DCHECK_EQ(valid_run.length, 0);
- ARROW_DCHECK_EQ(values_remaining, 0);
- return values_read;
+
+ ARROW_DCHECK_EQ(values_available - decoder.Remaining(), batch.ValuesRead());
+ ARROW_DCHECK_LE(batch.TotalRead(), batch_size);
+ ARROW_DCHECK_LE(batch.NullRead(), batch.NullCount());
+
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+/// Overload for GetSpaced for a single run in a decoder variant
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(
+ Converter& converter, typename Converter::out_type* out, values_count_type batch_size,
+ values_count_type null_count, BitRunReader&& validity_reader, BitRun&& validity_run,
+ std::variant<RleDecoder<value_type>, BitPackedDecoder<value_type>>& decoder)
Review Comment:
Ideally we want to avoid passing non-const references (use a pointer instead).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2319447403
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ ARROW_DCHECK_LT(values_read, batch_size);
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
Review Comment:
With the right definitions directly in the run classes, you could probably write something like:
```suggestion
typename decltype(run)::DecoderType decoder(run);
```
or even perhaps:
```suggestion
auto decoder = run.MakeDecoder();
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2319540511
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ ARROW_DCHECK_LT(values_read, batch_size);
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+ auto const read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
return values_read;
}
-template <typename T, typename RunType, typename Converter>
-inline int RleDecoder::GetSpaced(Converter converter, int batch_size, int null_count,
- const uint8_t* valid_bits, int64_t valid_bits_offset,
- T* out) {
- if (ARROW_PREDICT_FALSE(null_count == batch_size)) {
- converter.FillZero(out, out + batch_size);
- return batch_size;
+namespace internal {
+
+/// Utility class to safely handle values and null count without too error-prone
+/// verbosity.
+class BatchCounter {
+ public:
+ using size_type = int32_t;
+
+ [[nodiscard]] static constexpr BatchCounter FromBatchSizeAndNulls(
+ size_type batch_size, size_type null_count) {
+ ARROW_DCHECK_LE(null_count, batch_size);
+ return {batch_size - null_count, null_count};
}
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
- int values_remaining = batch_size - null_count;
+ constexpr BatchCounter(size_type values_count, size_type null_count) noexcept
+ : values_count_(values_count), null_count_(null_count) {}
- // Assume no bits to start.
- arrow::internal::BitRunReader bit_reader(valid_bits, valid_bits_offset,
- /*length=*/batch_size);
- arrow::internal::BitRun valid_run = bit_reader.NextRun();
- while (values_read < batch_size) {
- if (ARROW_PREDICT_FALSE(valid_run.length == 0)) {
- valid_run = bit_reader.NextRun();
+ [[nodiscard]] constexpr size_type ValuesCount() const noexcept { return values_count_; }
+
+ [[nodiscard]] constexpr size_type ValuesRead() const noexcept { return values_read_; }
+
+ [[nodiscard]] constexpr size_type ValuesRemaining() const noexcept {
+ ARROW_DCHECK_LE(values_read_, values_count_);
+ return values_count_ - values_read_;
+ }
+
+ constexpr void AccrueReadValues(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, ValuesRemaining());
+ values_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type NullCount() const noexcept { return null_count_; }
+
+ [[nodiscard]] constexpr size_type NullRead() const noexcept { return null_read_; }
+
+ [[nodiscard]] constexpr size_type NullRemaining() const noexcept {
+ ARROW_DCHECK_LE(null_read_, null_count_);
+ return null_count_ - null_read_;
+ }
+
+ constexpr void AccrueReadNulls(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, NullRemaining());
+ null_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type TotalRemaining() const noexcept {
+ return ValuesRemaining() + NullRemaining();
+ }
+
+ [[nodiscard]] constexpr size_type TotalRead() const noexcept {
+ return values_read_ + null_read_;
+ }
+
+ [[nodiscard]] constexpr bool IsFullyNull() const noexcept {
+ return ValuesRemaining() == 0;
+ }
+
+ [[nodiscard]] constexpr bool IsDone() const noexcept { return TotalRemaining() == 0; }
+
+ private:
+ size_type values_count_ = 0;
+ size_type values_read_ = 0;
+ size_type null_count_ = 0;
+ size_type null_read_ = 0;
+};
+
+// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+/// Overload for GetSpaced for a single run in a RleDecoder
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ RleDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto values_remaining_run = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // Consume as much as possible from the repeated run.
+ // We only need to count the number of nulls and non-nulls because we can fill in the
+ // same value for nulls and non-nulls.
+ // This proves to be a big efficiency win.
+ while (values_remaining_run() > 0 && !batch.IsDone()) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const& validity_run_size = static_cast<values_count_type>(validity_run.length);
+
+ if (validity_run.set) {
+ // We may end the current RLE run in the middle of the validity run
+ auto update_size = std::min(validity_run_size, values_remaining_run());
+ batch.AccrueReadValues(update_size);
+ validity_run.length -= update_size;
+ } else {
+ // We can consume all nulls here because it does not matter if we consume on this
+ // RLE run, or an a next encoded run. The value filled does not matter.
+ auto update_size = std::min(validity_run_size, batch.NullRemaining());
+ batch.AccrueReadNulls(update_size);
+ validity_run.length -= update_size;
+ }
+
+ if (ARROW_PREDICT_TRUE(validity_run.length == 0)) {
+ validity_run = validity_reader.NextRun();
}
+ }
+
+ value_type const value = decoder.Value();
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(value))) {
+ return {0, 0};
+ }
+ converter.WriteRepeated(out, out + batch.TotalRead(), value);
+ auto const actual_values_read = decoder.Advance(batch.ValuesRead());
+ // We always cropped the number of values_read by the remaining values in the run.
+ // What's more the RLE decoder should not encounter any errors.
+ ARROW_DCHECK_EQ(actual_values_read, batch.ValuesRead());
- ARROW_DCHECK_GT(batch_size, 0);
- ARROW_DCHECK_GT(valid_run.length, 0);
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+template <typename T, typename... Ts>
+[[nodiscard]] constexpr T min(T x, Ts... ys) {
+ ((x = std::min(x, ys)), ...);
+ return x;
+}
+
+static_assert(min(5) == 5);
+static_assert(min(5, 4, -1) == -1);
+static_assert(min(5, 41) == 5);
+
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ BitPackedDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto run_values_remaining = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ while (run_values_remaining() > 0 && batch.ValuesRemaining() > 0) {
+ // TODO should this size be tune depending on sizeof(value_size)? cpu cache size?
+ // Pull a batch of values from the bit packed encoded data and store it in a local
+ // buffer to benefit from unpacking intrinsics and data locality.
+ static constexpr values_count_type kBufferCapacity = 1024;
+ std::array<value_type, kBufferCapacity> buffer = {};
+
+ values_count_type buffer_start = 0;
+ values_count_type buffer_end = 0;
+ auto buffer_size = [&]() {
+ auto out = buffer_end - buffer_start;
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // buffer_start is 0 at this point so size is end
+ buffer_end = min(run_values_remaining(), batch.ValuesRemaining(), kBufferCapacity);
+ buffer_end = decoder.GetBatch(buffer.data(), buffer_size());
+ ARROW_DCHECK_LE(buffer_size(), kBufferCapacity);
+
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(buffer.data(), buffer_size()))) {
+ return {batch.ValuesRead(), batch.NullRead()};
+ }
- if (valid_run.set) {
- if ((repeat_count_ == 0) && (literal_count_ == 0)) {
- if (!NextCounts<RunType>()) return values_read;
- ARROW_DCHECK((repeat_count_ > 0) ^ (literal_count_ > 0));
+ // Copy chunks of valid values into the output, while adjusting spacing for null
+ // values.
+ while (buffer_size() > 0) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const validity_run_length =
+ static_cast<values_count_type>(validity_run.length);
+
+ // Copy as much as possible from the buffer into the output while not exceeding
+ // validity run
+ if (validity_run.set) {
+ auto const update_size = std::min(validity_run_length, buffer_size());
+ converter.WriteRange(out, buffer.data() + buffer_start, update_size);
+ buffer_start += update_size;
+ batch.AccrueReadValues(update_size);
+ out += update_size;
+ validity_run.length -= update_size;
+ // Simply write zeros in the output
+ } else {
+ auto const update_size = std::min(validity_run_length, batch.NullRemaining());
+ converter.WriteZero(out, out + update_size);
+ batch.AccrueReadNulls(update_size);
+ out += update_size;
+ validity_run.length -= update_size;
}
- if (repeat_count_ > 0) {
- int repeat_batch = 0;
- // Consume the entire repeat counts incrementing repeat_batch to
- // be the total of nulls + values consumed, we only need to
- // get the total count because we can fill in the same value for
- // nulls and non-nulls. This proves to be a big efficiency win.
- while (repeat_count_ > 0 && (values_read + repeat_batch) < batch_size) {
- ARROW_DCHECK_GT(valid_run.length, 0);
- if (valid_run.set) {
- int update_size = std::min(static_cast<int>(valid_run.length), repeat_count_);
- repeat_count_ -= update_size;
- repeat_batch += update_size;
- valid_run.length -= update_size;
- values_remaining -= update_size;
- } else {
- // We can consume all nulls here because we would do so on
- // the next loop anyways.
- repeat_batch += static_cast<int>(valid_run.length);
- valid_run.length = 0;
- }
- if (valid_run.length == 0) {
- valid_run = bit_reader.NextRun();
- }
- }
- RunType current_value = static_cast<RunType>(current_value_);
- if (ARROW_PREDICT_FALSE(!converter.IsValid(current_value))) {
- return values_read;
- }
- converter.Fill(out, out + repeat_batch, current_value);
- out += repeat_batch;
- values_read += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(values_remaining, literal_count_);
- ARROW_DCHECK_GT(literal_batch, 0);
-
- // Decode the literals
- constexpr int kBufferSize = 1024;
- RunType indices[kBufferSize];
- literal_batch = std::min(literal_batch, kBufferSize);
- int actual_read = bit_reader_.GetBatch(bit_width_, indices, literal_batch);
- if (ARROW_PREDICT_FALSE(actual_read != literal_batch)) {
- return values_read;
- }
- if (!converter.IsValid(indices, /*length=*/actual_read)) {
- return values_read;
- }
- int skipped = 0;
- int literals_read = 0;
- while (literals_read < literal_batch) {
- if (valid_run.set) {
- int update_size = std::min(literal_batch - literals_read,
- static_cast<int>(valid_run.length));
- converter.Copy(out, indices + literals_read, update_size);
- literals_read += update_size;
- out += update_size;
- valid_run.length -= update_size;
- } else {
- converter.FillZero(out, out + valid_run.length);
- out += valid_run.length;
- skipped += static_cast<int>(valid_run.length);
- valid_run.length = 0;
- }
- if (valid_run.length == 0) {
- valid_run = bit_reader.NextRun();
- }
- }
- literal_count_ -= literal_batch;
- values_remaining -= literal_batch;
- values_read += literal_batch + skipped;
+ if (validity_run.length == 0) {
+ validity_run = validity_reader.NextRun();
}
- } else {
- converter.FillZero(out, out + valid_run.length);
- out += valid_run.length;
- values_read += static_cast<int>(valid_run.length);
- valid_run.length = 0;
}
+
+ ARROW_DCHECK_EQ(buffer_size(), 0);
}
- ARROW_DCHECK_EQ(valid_run.length, 0);
- ARROW_DCHECK_EQ(values_remaining, 0);
- return values_read;
+
+ ARROW_DCHECK_EQ(values_available - decoder.Remaining(), batch.ValuesRead());
+ ARROW_DCHECK_LE(batch.TotalRead(), batch_size);
+ ARROW_DCHECK_LE(batch.NullRead(), batch.NullCount());
+
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+/// Overload for GetSpaced for a single run in a decoder variant
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(
+ Converter& converter, typename Converter::out_type* out, values_count_type batch_size,
+ values_count_type null_count, BitRunReader&& validity_reader, BitRun&& validity_run,
+ std::variant<RleDecoder<value_type>, BitPackedDecoder<value_type>>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ return std::visit(
+ [&](auto& dec) {
+ ARROW_DCHECK_GT(dec.Remaining(), 0);
+ return RunGetSpaced(converter, out, batch_size, null_count, validity_reader,
+ validity_run, dec);
+ },
+ decoder);
}
+} // namespace internal
+
+template <typename T>
+template <typename Converter>
+auto RleBitPackedDecoder<T>::GetSpaced(
+ Converter converter, typename Converter::out_type* out, values_count_type batch_size,
+ const byte* validity_bits, int64_t validity_bits_offset, values_count_type null_count)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
+
+ ARROW_DCHECK_GT(batch_size, 0);
+
+ auto batch = internal::BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ if (ARROW_PREDICT_FALSE(batch.IsFullyNull())) {
+ converter.WriteZero(out, out + batch.NullRemaining());
+ return batch.NullRemaining();
+ }
+
+ arrow::internal::BitRunReader validity_reader(validity_bits, validity_bits_offset,
+ /*length=*/batch.TotalRemaining());
+ arrow::internal::BitRun validity_run = validity_reader.NextRun();
+
+ auto const check_and_handle_fully_null_remaining = [&]() {
+ if (batch.IsFullyNull()) {
+ ARROW_DCHECK(validity_run.length == 0 || !validity_run.set);
+ ARROW_DCHECK_GE(validity_run.length, batch.NullRemaining());
+
+ converter.WriteZero(out, out + batch.NullRemaining());
+ out += batch.NullRemaining();
+ batch.AccrueReadNulls(batch.NullRemaining());
+ }
+ };
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
Review Comment:
Not sure about `ARROW_PREDICT_FALSE` again...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2319532082
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ ARROW_DCHECK_LT(values_read, batch_size);
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+ auto const read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
return values_read;
}
-template <typename T, typename RunType, typename Converter>
-inline int RleDecoder::GetSpaced(Converter converter, int batch_size, int null_count,
- const uint8_t* valid_bits, int64_t valid_bits_offset,
- T* out) {
- if (ARROW_PREDICT_FALSE(null_count == batch_size)) {
- converter.FillZero(out, out + batch_size);
- return batch_size;
+namespace internal {
+
+/// Utility class to safely handle values and null count without too error-prone
+/// verbosity.
+class BatchCounter {
+ public:
+ using size_type = int32_t;
+
+ [[nodiscard]] static constexpr BatchCounter FromBatchSizeAndNulls(
+ size_type batch_size, size_type null_count) {
+ ARROW_DCHECK_LE(null_count, batch_size);
+ return {batch_size - null_count, null_count};
}
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
- int values_remaining = batch_size - null_count;
+ constexpr BatchCounter(size_type values_count, size_type null_count) noexcept
+ : values_count_(values_count), null_count_(null_count) {}
- // Assume no bits to start.
- arrow::internal::BitRunReader bit_reader(valid_bits, valid_bits_offset,
- /*length=*/batch_size);
- arrow::internal::BitRun valid_run = bit_reader.NextRun();
- while (values_read < batch_size) {
- if (ARROW_PREDICT_FALSE(valid_run.length == 0)) {
- valid_run = bit_reader.NextRun();
+ [[nodiscard]] constexpr size_type ValuesCount() const noexcept { return values_count_; }
+
+ [[nodiscard]] constexpr size_type ValuesRead() const noexcept { return values_read_; }
+
+ [[nodiscard]] constexpr size_type ValuesRemaining() const noexcept {
+ ARROW_DCHECK_LE(values_read_, values_count_);
+ return values_count_ - values_read_;
+ }
+
+ constexpr void AccrueReadValues(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, ValuesRemaining());
+ values_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type NullCount() const noexcept { return null_count_; }
+
+ [[nodiscard]] constexpr size_type NullRead() const noexcept { return null_read_; }
+
+ [[nodiscard]] constexpr size_type NullRemaining() const noexcept {
+ ARROW_DCHECK_LE(null_read_, null_count_);
+ return null_count_ - null_read_;
+ }
+
+ constexpr void AccrueReadNulls(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, NullRemaining());
+ null_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type TotalRemaining() const noexcept {
+ return ValuesRemaining() + NullRemaining();
+ }
+
+ [[nodiscard]] constexpr size_type TotalRead() const noexcept {
+ return values_read_ + null_read_;
+ }
+
+ [[nodiscard]] constexpr bool IsFullyNull() const noexcept {
+ return ValuesRemaining() == 0;
+ }
+
+ [[nodiscard]] constexpr bool IsDone() const noexcept { return TotalRemaining() == 0; }
+
+ private:
+ size_type values_count_ = 0;
+ size_type values_read_ = 0;
+ size_type null_count_ = 0;
+ size_type null_read_ = 0;
+};
+
+// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+/// Overload for GetSpaced for a single run in a RleDecoder
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ RleDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto values_remaining_run = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // Consume as much as possible from the repeated run.
+ // We only need to count the number of nulls and non-nulls because we can fill in the
+ // same value for nulls and non-nulls.
+ // This proves to be a big efficiency win.
+ while (values_remaining_run() > 0 && !batch.IsDone()) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const& validity_run_size = static_cast<values_count_type>(validity_run.length);
+
+ if (validity_run.set) {
+ // We may end the current RLE run in the middle of the validity run
+ auto update_size = std::min(validity_run_size, values_remaining_run());
+ batch.AccrueReadValues(update_size);
+ validity_run.length -= update_size;
+ } else {
+ // We can consume all nulls here because it does not matter if we consume on this
+ // RLE run, or an a next encoded run. The value filled does not matter.
+ auto update_size = std::min(validity_run_size, batch.NullRemaining());
+ batch.AccrueReadNulls(update_size);
+ validity_run.length -= update_size;
+ }
+
+ if (ARROW_PREDICT_TRUE(validity_run.length == 0)) {
+ validity_run = validity_reader.NextRun();
}
+ }
+
+ value_type const value = decoder.Value();
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(value))) {
+ return {0, 0};
+ }
+ converter.WriteRepeated(out, out + batch.TotalRead(), value);
+ auto const actual_values_read = decoder.Advance(batch.ValuesRead());
+ // We always cropped the number of values_read by the remaining values in the run.
+ // What's more the RLE decoder should not encounter any errors.
+ ARROW_DCHECK_EQ(actual_values_read, batch.ValuesRead());
- ARROW_DCHECK_GT(batch_size, 0);
- ARROW_DCHECK_GT(valid_run.length, 0);
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+template <typename T, typename... Ts>
+[[nodiscard]] constexpr T min(T x, Ts... ys) {
+ ((x = std::min(x, ys)), ...);
+ return x;
+}
+
+static_assert(min(5) == 5);
+static_assert(min(5, 4, -1) == -1);
+static_assert(min(5, 41) == 5);
+
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ BitPackedDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto run_values_remaining = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ while (run_values_remaining() > 0 && batch.ValuesRemaining() > 0) {
Review Comment:
For simplification, can we instead instantiate `batch` like this:
```c++
auto batch = BatchCounter::FromBatchSizeAndNulls(
std::min(batch_size, values_available), null_count);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2319573067
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ ARROW_DCHECK_LT(values_read, batch_size);
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+ auto const read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
return values_read;
}
-template <typename T, typename RunType, typename Converter>
-inline int RleDecoder::GetSpaced(Converter converter, int batch_size, int null_count,
- const uint8_t* valid_bits, int64_t valid_bits_offset,
- T* out) {
- if (ARROW_PREDICT_FALSE(null_count == batch_size)) {
- converter.FillZero(out, out + batch_size);
- return batch_size;
+namespace internal {
+
+/// Utility class to safely handle values and null count without too error-prone
+/// verbosity.
+class BatchCounter {
+ public:
+ using size_type = int32_t;
+
+ [[nodiscard]] static constexpr BatchCounter FromBatchSizeAndNulls(
+ size_type batch_size, size_type null_count) {
+ ARROW_DCHECK_LE(null_count, batch_size);
+ return {batch_size - null_count, null_count};
}
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
- int values_remaining = batch_size - null_count;
+ constexpr BatchCounter(size_type values_count, size_type null_count) noexcept
+ : values_count_(values_count), null_count_(null_count) {}
- // Assume no bits to start.
- arrow::internal::BitRunReader bit_reader(valid_bits, valid_bits_offset,
- /*length=*/batch_size);
- arrow::internal::BitRun valid_run = bit_reader.NextRun();
- while (values_read < batch_size) {
- if (ARROW_PREDICT_FALSE(valid_run.length == 0)) {
- valid_run = bit_reader.NextRun();
+ [[nodiscard]] constexpr size_type ValuesCount() const noexcept { return values_count_; }
+
+ [[nodiscard]] constexpr size_type ValuesRead() const noexcept { return values_read_; }
+
+ [[nodiscard]] constexpr size_type ValuesRemaining() const noexcept {
+ ARROW_DCHECK_LE(values_read_, values_count_);
+ return values_count_ - values_read_;
+ }
+
+ constexpr void AccrueReadValues(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, ValuesRemaining());
+ values_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type NullCount() const noexcept { return null_count_; }
+
+ [[nodiscard]] constexpr size_type NullRead() const noexcept { return null_read_; }
+
+ [[nodiscard]] constexpr size_type NullRemaining() const noexcept {
+ ARROW_DCHECK_LE(null_read_, null_count_);
+ return null_count_ - null_read_;
+ }
+
+ constexpr void AccrueReadNulls(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, NullRemaining());
+ null_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type TotalRemaining() const noexcept {
+ return ValuesRemaining() + NullRemaining();
+ }
+
+ [[nodiscard]] constexpr size_type TotalRead() const noexcept {
+ return values_read_ + null_read_;
+ }
+
+ [[nodiscard]] constexpr bool IsFullyNull() const noexcept {
+ return ValuesRemaining() == 0;
+ }
+
+ [[nodiscard]] constexpr bool IsDone() const noexcept { return TotalRemaining() == 0; }
+
+ private:
+ size_type values_count_ = 0;
+ size_type values_read_ = 0;
+ size_type null_count_ = 0;
+ size_type null_read_ = 0;
+};
+
+// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+/// Overload for GetSpaced for a single run in a RleDecoder
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ RleDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto values_remaining_run = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // Consume as much as possible from the repeated run.
+ // We only need to count the number of nulls and non-nulls because we can fill in the
+ // same value for nulls and non-nulls.
+ // This proves to be a big efficiency win.
+ while (values_remaining_run() > 0 && !batch.IsDone()) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const& validity_run_size = static_cast<values_count_type>(validity_run.length);
+
+ if (validity_run.set) {
+ // We may end the current RLE run in the middle of the validity run
+ auto update_size = std::min(validity_run_size, values_remaining_run());
+ batch.AccrueReadValues(update_size);
+ validity_run.length -= update_size;
+ } else {
+ // We can consume all nulls here because it does not matter if we consume on this
+ // RLE run, or an a next encoded run. The value filled does not matter.
+ auto update_size = std::min(validity_run_size, batch.NullRemaining());
+ batch.AccrueReadNulls(update_size);
+ validity_run.length -= update_size;
+ }
+
+ if (ARROW_PREDICT_TRUE(validity_run.length == 0)) {
+ validity_run = validity_reader.NextRun();
}
+ }
+
+ value_type const value = decoder.Value();
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(value))) {
+ return {0, 0};
+ }
+ converter.WriteRepeated(out, out + batch.TotalRead(), value);
+ auto const actual_values_read = decoder.Advance(batch.ValuesRead());
+ // We always cropped the number of values_read by the remaining values in the run.
+ // What's more the RLE decoder should not encounter any errors.
+ ARROW_DCHECK_EQ(actual_values_read, batch.ValuesRead());
- ARROW_DCHECK_GT(batch_size, 0);
- ARROW_DCHECK_GT(valid_run.length, 0);
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+template <typename T, typename... Ts>
+[[nodiscard]] constexpr T min(T x, Ts... ys) {
+ ((x = std::min(x, ys)), ...);
+ return x;
+}
+
+static_assert(min(5) == 5);
+static_assert(min(5, 4, -1) == -1);
+static_assert(min(5, 41) == 5);
+
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ BitPackedDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto run_values_remaining = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ while (run_values_remaining() > 0 && batch.ValuesRemaining() > 0) {
+ // TODO should this size be tune depending on sizeof(value_size)? cpu cache size?
+ // Pull a batch of values from the bit packed encoded data and store it in a local
+ // buffer to benefit from unpacking intrinsics and data locality.
+ static constexpr values_count_type kBufferCapacity = 1024;
+ std::array<value_type, kBufferCapacity> buffer = {};
+
+ values_count_type buffer_start = 0;
+ values_count_type buffer_end = 0;
+ auto buffer_size = [&]() {
+ auto out = buffer_end - buffer_start;
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // buffer_start is 0 at this point so size is end
+ buffer_end = min(run_values_remaining(), batch.ValuesRemaining(), kBufferCapacity);
+ buffer_end = decoder.GetBatch(buffer.data(), buffer_size());
+ ARROW_DCHECK_LE(buffer_size(), kBufferCapacity);
+
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(buffer.data(), buffer_size()))) {
+ return {batch.ValuesRead(), batch.NullRead()};
+ }
- if (valid_run.set) {
- if ((repeat_count_ == 0) && (literal_count_ == 0)) {
- if (!NextCounts<RunType>()) return values_read;
- ARROW_DCHECK((repeat_count_ > 0) ^ (literal_count_ > 0));
+ // Copy chunks of valid values into the output, while adjusting spacing for null
+ // values.
+ while (buffer_size() > 0) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const validity_run_length =
+ static_cast<values_count_type>(validity_run.length);
+
+ // Copy as much as possible from the buffer into the output while not exceeding
+ // validity run
+ if (validity_run.set) {
+ auto const update_size = std::min(validity_run_length, buffer_size());
+ converter.WriteRange(out, buffer.data() + buffer_start, update_size);
+ buffer_start += update_size;
+ batch.AccrueReadValues(update_size);
+ out += update_size;
+ validity_run.length -= update_size;
+ // Simply write zeros in the output
+ } else {
+ auto const update_size = std::min(validity_run_length, batch.NullRemaining());
+ converter.WriteZero(out, out + update_size);
+ batch.AccrueReadNulls(update_size);
+ out += update_size;
+ validity_run.length -= update_size;
}
- if (repeat_count_ > 0) {
- int repeat_batch = 0;
- // Consume the entire repeat counts incrementing repeat_batch to
- // be the total of nulls + values consumed, we only need to
- // get the total count because we can fill in the same value for
- // nulls and non-nulls. This proves to be a big efficiency win.
- while (repeat_count_ > 0 && (values_read + repeat_batch) < batch_size) {
- ARROW_DCHECK_GT(valid_run.length, 0);
- if (valid_run.set) {
- int update_size = std::min(static_cast<int>(valid_run.length), repeat_count_);
- repeat_count_ -= update_size;
- repeat_batch += update_size;
- valid_run.length -= update_size;
- values_remaining -= update_size;
- } else {
- // We can consume all nulls here because we would do so on
- // the next loop anyways.
- repeat_batch += static_cast<int>(valid_run.length);
- valid_run.length = 0;
- }
- if (valid_run.length == 0) {
- valid_run = bit_reader.NextRun();
- }
- }
- RunType current_value = static_cast<RunType>(current_value_);
- if (ARROW_PREDICT_FALSE(!converter.IsValid(current_value))) {
- return values_read;
- }
- converter.Fill(out, out + repeat_batch, current_value);
- out += repeat_batch;
- values_read += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(values_remaining, literal_count_);
- ARROW_DCHECK_GT(literal_batch, 0);
-
- // Decode the literals
- constexpr int kBufferSize = 1024;
- RunType indices[kBufferSize];
- literal_batch = std::min(literal_batch, kBufferSize);
- int actual_read = bit_reader_.GetBatch(bit_width_, indices, literal_batch);
- if (ARROW_PREDICT_FALSE(actual_read != literal_batch)) {
- return values_read;
- }
- if (!converter.IsValid(indices, /*length=*/actual_read)) {
- return values_read;
- }
- int skipped = 0;
- int literals_read = 0;
- while (literals_read < literal_batch) {
- if (valid_run.set) {
- int update_size = std::min(literal_batch - literals_read,
- static_cast<int>(valid_run.length));
- converter.Copy(out, indices + literals_read, update_size);
- literals_read += update_size;
- out += update_size;
- valid_run.length -= update_size;
- } else {
- converter.FillZero(out, out + valid_run.length);
- out += valid_run.length;
- skipped += static_cast<int>(valid_run.length);
- valid_run.length = 0;
- }
- if (valid_run.length == 0) {
- valid_run = bit_reader.NextRun();
- }
- }
- literal_count_ -= literal_batch;
- values_remaining -= literal_batch;
- values_read += literal_batch + skipped;
+ if (validity_run.length == 0) {
+ validity_run = validity_reader.NextRun();
}
- } else {
- converter.FillZero(out, out + valid_run.length);
- out += valid_run.length;
- values_read += static_cast<int>(valid_run.length);
- valid_run.length = 0;
}
+
+ ARROW_DCHECK_EQ(buffer_size(), 0);
}
- ARROW_DCHECK_EQ(valid_run.length, 0);
- ARROW_DCHECK_EQ(values_remaining, 0);
- return values_read;
+
+ ARROW_DCHECK_EQ(values_available - decoder.Remaining(), batch.ValuesRead());
+ ARROW_DCHECK_LE(batch.TotalRead(), batch_size);
+ ARROW_DCHECK_LE(batch.NullRead(), batch.NullCount());
+
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+/// Overload for GetSpaced for a single run in a decoder variant
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(
+ Converter& converter, typename Converter::out_type* out, values_count_type batch_size,
+ values_count_type null_count, BitRunReader&& validity_reader, BitRun&& validity_run,
+ std::variant<RleDecoder<value_type>, BitPackedDecoder<value_type>>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ return std::visit(
+ [&](auto& dec) {
+ ARROW_DCHECK_GT(dec.Remaining(), 0);
+ return RunGetSpaced(converter, out, batch_size, null_count, validity_reader,
+ validity_run, dec);
+ },
+ decoder);
}
+} // namespace internal
+
+template <typename T>
+template <typename Converter>
+auto RleBitPackedDecoder<T>::GetSpaced(
+ Converter converter, typename Converter::out_type* out, values_count_type batch_size,
+ const byte* validity_bits, int64_t validity_bits_offset, values_count_type null_count)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
+
+ ARROW_DCHECK_GT(batch_size, 0);
+
+ auto batch = internal::BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ if (ARROW_PREDICT_FALSE(batch.IsFullyNull())) {
+ converter.WriteZero(out, out + batch.NullRemaining());
+ return batch.NullRemaining();
+ }
+
+ arrow::internal::BitRunReader validity_reader(validity_bits, validity_bits_offset,
+ /*length=*/batch.TotalRemaining());
+ arrow::internal::BitRun validity_run = validity_reader.NextRun();
+
+ auto const check_and_handle_fully_null_remaining = [&]() {
+ if (batch.IsFullyNull()) {
+ ARROW_DCHECK(validity_run.length == 0 || !validity_run.set);
+ ARROW_DCHECK_GE(validity_run.length, batch.NullRemaining());
+
+ converter.WriteZero(out, out + batch.NullRemaining());
+ out += batch.NullRemaining();
+ batch.AccrueReadNulls(batch.NullRemaining());
+ }
+ };
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const [values_read, null_read] =
+ RunGetSpaced(converter, out, batch.TotalRemaining(), batch.NullRemaining(),
+ validity_reader, validity_run, decoder_);
+
+ batch.AccrueReadNulls(null_read);
+ batch.AccrueReadValues(values_read);
+ out += values_read + null_read;
+
+ // Either we fulfilled all the batch values to be read
+ if (ARROW_PREDICT_FALSE(batch.ValuesRemaining() == 0)) {
+ // There may be remaining null if they are not greedily filled
+ check_and_handle_fully_null_remaining();
+ return batch.TotalRead();
+ }
+
+ /// We finished the remaining run
+ ARROW_DCHECK(RunRemaining() == 0);
+ }
+
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+
+ auto const [values_read, null_read] = internal::RunGetSpaced(
+ converter, out, batch.TotalRemaining(), batch.NullRemaining(),
+ validity_reader, validity_run, decoder);
+
+ batch.AccrueReadNulls(null_read);
+ batch.AccrueReadValues(values_read);
+ out += values_read + null_read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == 0 || batch.ValuesRemaining() == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
+ // There may be remaining null if they are not greedily filled by either decoder calls
+ check_and_handle_fully_null_remaining();
+
+ ARROW_DCHECK(batch.IsDone() || Exhausted());
+ // batch.Done() => batch.NullRemaining() == 0
+ ARROW_DCHECK(!batch.IsDone() || (batch.NullRemaining() == 0));
+ return batch.TotalRead();
+}
+
+namespace internal {
+
// Converter for GetSpaced that handles runs that get returned
// directly as output.
template <typename T>
-struct PlainRleConverter {
- T kZero = {};
- inline bool IsValid(const T& values) const { return true; }
- inline bool IsValid(const T* values, int32_t length) const { return true; }
- inline void Fill(T* begin, T* end, const T& run_value) const {
+struct NoOpConverter {
+ using in_type = T;
+ using out_type = T;
+ using size_type = int32_t;
+
+ static constexpr bool kIsIdentity = true;
+
+ [[nodiscard]] static constexpr bool InputIsValid(const in_type& values) { return true; }
+
+ [[nodiscard]] static constexpr bool InputIsValid(const in_type* values,
+ size_type length) {
+ return true;
+ }
+
+ static void WriteRepeated(out_type* begin, out_type* end, in_type run_value) {
std::fill(begin, end, run_value);
}
- inline void FillZero(T* begin, T* end) { std::fill(begin, end, kZero); }
- inline void Copy(T* out, const T* values, int length) const {
- std::memcpy(out, values, length * sizeof(T));
+
+ static void WriteZero(out_type* begin, out_type* end) {
+ std::fill(begin, end, out_type{});
+ }
+
+ static void WriteRange(out_type* out, const in_type* values, size_type length) {
+ std::memcpy(out, values, length * sizeof(out_type));
}
};
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatchSpaced(int batch_size, int null_count,
- const uint8_t* valid_bits,
- int64_t valid_bits_offset, T* out) {
+auto RleBitPackedDecoder<T>::GetBatchSpaced(values_count_type batch_size,
+ values_count_type null_count,
+ const byte* valid_bits,
+ int64_t valid_bits_offset, value_type* out)
+ -> values_count_type {
if (null_count == 0) {
- return GetBatch<T>(out, batch_size);
+ return GetBatch(out, batch_size);
}
- PlainRleConverter<T> converter;
- arrow::internal::BitBlockCounter block_counter(valid_bits, valid_bits_offset,
- batch_size);
-
- int total_processed = 0;
- int processed = 0;
- arrow::internal::BitBlockCount block;
+ internal::NoOpConverter<value_type> converter;
- do {
- block = block_counter.NextFourWords();
- if (block.length == 0) {
- break;
- }
- if (block.AllSet()) {
- processed = GetBatch<T>(out, block.length);
- } else if (block.NoneSet()) {
- converter.FillZero(out, out + block.length);
- processed = block.length;
- } else {
- processed = GetSpaced<T, /*RunType=*/T, PlainRleConverter<T>>(
- converter, block.length, block.length - block.popcount, valid_bits,
- valid_bits_offset, out);
- }
- total_processed += processed;
- out += block.length;
- valid_bits_offset += block.length;
- } while (processed == block.length);
- return total_processed;
+ return GetSpaced(converter, out, batch_size, valid_bits, valid_bits_offset, null_count);
}
-static inline bool IndexInRange(int32_t idx, int32_t dictionary_length) {
- return idx >= 0 && idx < dictionary_length;
+namespace internal {
+
+template <typename I>
+bool IndexInRange(I idx, int32_t dictionary_length) {
+ ARROW_DCHECK_GT(dictionary_length, 0);
+ using T = std::common_type_t<decltype(idx), decltype(dictionary_length)>;
+ return idx >= 0 && static_cast<T>(idx) < static_cast<T>(dictionary_length);
}
// Converter for GetSpaced that handles runs of returned dictionary
// indices.
-template <typename T>
+template <typename V, typename I>
struct DictionaryConverter {
- T kZero = {};
- const T* dictionary;
- int32_t dictionary_length;
-
- inline bool IsValid(int32_t value) { return IndexInRange(value, dictionary_length); }
-
- inline bool IsValid(const int32_t* values, int32_t length) const {
- using IndexType = int32_t;
- IndexType min_index = std::numeric_limits<IndexType>::max();
- IndexType max_index = std::numeric_limits<IndexType>::min();
- for (int x = 0; x < length; x++) {
- min_index = std::min(values[x], min_index);
- max_index = std::max(values[x], max_index);
+ using out_type = V;
+ using in_type = I;
+ using size_type = int32_t;
+
+ static constexpr bool kIsIdentity = false;
+
+ const out_type* dictionary;
+ size_type dictionary_length;
+
+ [[nodiscard]] bool InputIsValid(in_type idx) const {
+ return IndexInRange(idx, dictionary_length);
+ }
+
+ [[nodiscard]] bool InputIsValid(const in_type* indices, size_type length) const {
+ in_type min_index = std::numeric_limits<in_type>::max();
+ in_type max_index = std::numeric_limits<in_type>::min();
+ for (size_type x = 0; x < length; x++) {
+ min_index = std::min(indices[x], min_index);
+ max_index = std::max(indices[x], max_index);
}
return IndexInRange(min_index, dictionary_length) &&
IndexInRange(max_index, dictionary_length);
}
- inline void Fill(T* begin, T* end, const int32_t& run_value) const {
+
+ void WriteRepeated(out_type* begin, out_type* end, in_type run_value) const {
std::fill(begin, end, dictionary[run_value]);
}
- inline void FillZero(T* begin, T* end) { std::fill(begin, end, kZero); }
- inline void Copy(T* out, const int32_t* values, int length) const {
- for (int x = 0; x < length; x++) {
+ static void WriteZero(out_type* begin, out_type* end) {
+ std::fill(begin, end, out_type{});
+ }
+
+ void WriteRange(out_type* out, const in_type* values, size_type length) const {
+ for (size_type x = 0; x < length; x++) {
out[x] = dictionary[values[x]];
}
}
};
+/// Dummy imitation of BitRun that is all set.
+struct AllSetBitRun {
+ static constexpr bool set = true;
+ int64_t length = 0;
+};
+
+/// Dummy imitation of BitRunReader that should never be called.
+struct UnreachableBitRunReader {
+ constexpr static AllSetBitRun NextRun() { return {}; }
+};
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatchWithDict(const T* dictionary, int32_t dictionary_length,
- T* values, int batch_size) {
- // Per https://github.com/apache/parquet-format/blob/master/Encodings.md,
- // the maximum dictionary index width in Parquet is 32 bits.
- using IndexType = int32_t;
- DictionaryConverter<T> converter;
- converter.dictionary = dictionary;
- converter.dictionary_length = dictionary_length;
-
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) {
- auto idx = static_cast<IndexType>(current_value_);
- if (ARROW_PREDICT_FALSE(!IndexInRange(idx, dictionary_length))) {
- return values_read;
- }
- T val = dictionary[idx];
+template <typename V>
+auto RleBitPackedDecoder<T>::GetBatchWithDict(const V* dictionary,
+ int32_t dictionary_length, V* out,
+ values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
+
+ if (ARROW_PREDICT_FALSE(batch_size <= 0)) {
+ return 0;
+ }
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, val);
+ internal::DictionaryConverter<V, value_type> converter{dictionary, dictionary_length};
- /* Upkeep counters */
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- constexpr int kBufferSize = 1024;
- IndexType indices[kBufferSize];
+ // Make lightweight BitRun class to reuse previous methods.
+ constexpr internal::UnreachableBitRunReader validity_reader{};
+ internal::AllSetBitRun validity_run = {batch_size};
- int literal_batch = std::min(remaining, literal_count_);
- literal_batch = std::min(literal_batch, kBufferSize);
+ values_count_type values_read = 0;
+ auto batch_values_remaining = [&]() {
+ ARROW_DCHECK_LE(values_read, batch_size);
+ return batch_size - values_read;
+ };
- int actual_read = bit_reader_.GetBatch(bit_width_, indices, literal_batch);
- if (ARROW_PREDICT_FALSE(actual_read != literal_batch)) {
- return values_read;
- }
- if (ARROW_PREDICT_FALSE(!converter.IsValid(indices, /*length=*/literal_batch))) {
- return values_read;
- }
- converter.Copy(out, indices, literal_batch);
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const [run_values_read, run_null_read] =
+ RunGetSpaced(converter, out, batch_size, /* null_count= */ 0, validity_reader,
+ validity_run, decoder_);
- /* Upkeep counters */
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<IndexType>()) return values_read;
+ ARROW_DCHECK_EQ(run_null_read, 0);
+ values_read += run_values_read;
+ out += run_values_read;
+
+ // Either we fulfilled all the batch values to be read
+ if (ARROW_PREDICT_FALSE(values_read >= batch_size)) {
+ // There may be remaining null if they are not greedily filled
+ return values_read;
}
+
+ /// We finished the remaining run
Review Comment:
```suggestion
// We finished the remaining run
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2336634847
##########
cpp/src/arrow/util/bit_stream_utils_internal.h:
##########
@@ -450,22 +450,35 @@ inline bool BitWriter::PutVlqInt(uint32_t v) {
}
inline bool BitReader::GetVlqInt(uint32_t* v) {
Review Comment:
An oversight, it's added now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2336232863
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -84,32 +85,278 @@ namespace util {
/// (total 26 bytes, 1 byte overhead)
//
+class RleRun {
+ public:
+ using byte = uint8_t;
+ /// Enough space to store a 64bit value
+ using raw_data_storage = std::array<byte, 8>;
+ using raw_data_const_pointer = const byte*;
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr RleRun() noexcept = default;
+ constexpr RleRun(RleRun const&) noexcept = default;
+ constexpr RleRun(RleRun&&) noexcept = default;
+
+ explicit RleRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr RleRun& operator=(RleRun const&) noexcept = default;
+ constexpr RleRun& operator=(RleRun&&) noexcept = default;
+
+ /// The number of repeated values in this run.
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ /// A pointer to the repeated value raw bytes.
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ /// The number of bytes used for the raw repeated value.
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The repeated value raw bytes stored inside the class
+ raw_data_storage data_ = {};
+ /// The number of time the value is repeated
+ values_count_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+class BitPackedRun {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// According to the Parquet thrift definition the page size can be written into an
+ /// int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr BitPackedRun() noexcept = default;
+ constexpr BitPackedRun(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun(BitPackedRun&&) noexcept = default;
+
+ constexpr BitPackedRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr BitPackedRun& operator=(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun& operator=(BitPackedRun&&) noexcept = default;
+
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Number of values in this run.
+ raw_data_size_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+/// A parser that emits either a ``BitPackedRun`` or a ``RleRun``.
+class RleBitPackedParser {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// By Parquet thrift definition the page size can be written into an int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+ /// The different types of runs emitted by the parser
+ using dynamic_run_type = std::variant<RleRun, BitPackedRun>;
+
+ constexpr RleBitPackedParser() noexcept = default;
+
+ constexpr RleBitPackedParser(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr void Reset(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width_) noexcept;
+
+ /// Get the current run with a small parsing cost without advancing the iteration.
+ [[nodiscard]] std::optional<dynamic_run_type> Peek() const;
Review Comment:
This function is now removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2316432675
##########
cpp/src/arrow/util/bit_util.h:
##########
@@ -365,5 +365,118 @@ void PackBits(const uint32_t* values, uint8_t* out) {
}
}
+constexpr int64_t MaxLEB128ByteLen(int64_t n_bits) { return CeilDiv(n_bits, 7); }
+
+template <typename Int>
+constexpr int64_t MaxLEB128ByteLenFor = MaxLEB128ByteLen(sizeof(Int) * 8);
+
+/// Write a integer as LEB128
+///
+/// Write the input value as LEB128 into the outptu buffer and return the number of bytes
+/// written.
+/// If the output buffer size is insufficient, return 0 but the output may have been
+/// written to.
+///
+/// \see https://en.wikipedia.org/wiki/LEB128
+/// \see MaxLEB128ByteLenFor
+template <typename Int>
+constexpr int32_t WriteLEB128(Int value, uint8_t* out, int32_t max_out_size) {
+ constexpr Int kLow7Mask = Int(0x7F);
+ constexpr Int kHigh7Mask = ~kLow7Mask;
+ constexpr uint8_t kContinuationBit = 0x80;
+
+ auto const out_first = out;
+
+ // Write as many bytes as the could be for the given input
+ while ((value & kHigh7Mask) != Int(0)) {
+ // We do not have enough room to write the LEB128
+ if (out - out_first >= max_out_size) {
+ return 0;
+ }
+
+ // Write the encoded byte with continuation bit
+ *out = static_cast<uint8_t>(value & kLow7Mask) | kContinuationBit;
+ ++out;
+ // Shift remaining data
+ value >>= 7;
+ }
+
+ // We do not have enough room to write the LEB128
+ if (out - out_first >= max_out_size) {
+ return 0;
+ }
+
+ // Write last non-continuing byte
+ *out = static_cast<uint8_t>(value & kLow7Mask);
+ ++out;
+
+ return static_cast<int32_t>(out - out_first);
+}
+
+/// Parse a leading LEB128
+///
+/// Take as input a data pointer and the maximum number of bytes that can be read from it
+/// (typically the array size).
+/// When a valid LEB128 is found at the start of the data, the function writes it to the
+/// out pointer and return the number of bytes read.
+/// Otherwise, the out pointer is unmodified and zero is returned.
+///
+/// \see https://en.wikipedia.org/wiki/LEB128
+/// \see MaxLEB128ByteLenFor
+template <typename Int>
+constexpr int32_t ParseLeadingLEB128(uint8_t const* data, int32_t max_data_size,
+ Int* out) {
+ constexpr auto kMaxBytes = static_cast<int32_t>(MaxLEB128ByteLenFor<Int>);
+ static_assert(kMaxBytes >= 1);
+ constexpr uint8_t kLow7Mask = 0x7F;
+ constexpr uint8_t kContinuationBit = 0x80;
+ constexpr int32_t kSignBitCount = std::is_signed_v<Int> ? 1 : 0;
Review Comment:
Do we allow signed integers here? I think they're supposed to be encoded to unsigned using zig-zag encoding?
##########
cpp/src/arrow/util/bit_stream_utils_internal.h:
##########
@@ -450,22 +450,35 @@ inline bool BitWriter::PutVlqInt(uint32_t v) {
}
inline bool BitReader::GetVlqInt(uint32_t* v) {
- uint32_t tmp = 0;
-
- for (int i = 0; i < kMaxVlqByteLength; i++) {
- uint8_t byte = 0;
- if (ARROW_PREDICT_FALSE(!GetAligned<uint8_t>(1, &byte))) {
- return false;
- }
- tmp |= static_cast<uint32_t>(byte & 0x7F) << (7 * i);
+ // The data that we will pass to the LEB128 parser
+ // In all case, we read an byte-aligned value, skipping remaining bits
+ uint8_t const* data = NULLPTR;
Review Comment:
Nit, but we generally put `const` before the type (if at all possible).
##########
cpp/src/arrow/util/bit_util.h:
##########
@@ -365,5 +365,118 @@ void PackBits(const uint32_t* values, uint8_t* out) {
}
}
+constexpr int64_t MaxLEB128ByteLen(int64_t n_bits) { return CeilDiv(n_bits, 7); }
+
+template <typename Int>
+constexpr int64_t MaxLEB128ByteLenFor = MaxLEB128ByteLen(sizeof(Int) * 8);
+
+/// Write a integer as LEB128
+///
+/// Write the input value as LEB128 into the outptu buffer and return the number of bytes
+/// written.
+/// If the output buffer size is insufficient, return 0 but the output may have been
+/// written to.
+///
+/// \see https://en.wikipedia.org/wiki/LEB128
+/// \see MaxLEB128ByteLenFor
+template <typename Int>
+constexpr int32_t WriteLEB128(Int value, uint8_t* out, int32_t max_out_size) {
+ constexpr Int kLow7Mask = Int(0x7F);
+ constexpr Int kHigh7Mask = ~kLow7Mask;
+ constexpr uint8_t kContinuationBit = 0x80;
+
+ auto const out_first = out;
+
+ // Write as many bytes as the could be for the given input
Review Comment:
```suggestion
// Write as many bytes as we could be for the given input
```
##########
cpp/src/arrow/util/bit_util.h:
##########
@@ -365,5 +365,118 @@ void PackBits(const uint32_t* values, uint8_t* out) {
}
}
+constexpr int64_t MaxLEB128ByteLen(int64_t n_bits) { return CeilDiv(n_bits, 7); }
+
+template <typename Int>
+constexpr int64_t MaxLEB128ByteLenFor = MaxLEB128ByteLen(sizeof(Int) * 8);
+
+/// Write a integer as LEB128
+///
+/// Write the input value as LEB128 into the outptu buffer and return the number of bytes
+/// written.
+/// If the output buffer size is insufficient, return 0 but the output may have been
+/// written to.
+///
+/// \see https://en.wikipedia.org/wiki/LEB128
+/// \see MaxLEB128ByteLenFor
+template <typename Int>
+constexpr int32_t WriteLEB128(Int value, uint8_t* out, int32_t max_out_size) {
+ constexpr Int kLow7Mask = Int(0x7F);
+ constexpr Int kHigh7Mask = ~kLow7Mask;
+ constexpr uint8_t kContinuationBit = 0x80;
+
+ auto const out_first = out;
+
+ // Write as many bytes as the could be for the given input
+ while ((value & kHigh7Mask) != Int(0)) {
+ // We do not have enough room to write the LEB128
+ if (out - out_first >= max_out_size) {
+ return 0;
+ }
+
+ // Write the encoded byte with continuation bit
+ *out = static_cast<uint8_t>(value & kLow7Mask) | kContinuationBit;
+ ++out;
+ // Shift remaining data
+ value >>= 7;
+ }
+
+ // We do not have enough room to write the LEB128
+ if (out - out_first >= max_out_size) {
Review Comment:
`ARROW_PREDICT_FALSE` as well?
##########
cpp/src/arrow/util/bit_util.h:
##########
@@ -365,5 +365,118 @@ void PackBits(const uint32_t* values, uint8_t* out) {
}
}
+constexpr int64_t MaxLEB128ByteLen(int64_t n_bits) { return CeilDiv(n_bits, 7); }
+
+template <typename Int>
+constexpr int64_t MaxLEB128ByteLenFor = MaxLEB128ByteLen(sizeof(Int) * 8);
+
+/// Write a integer as LEB128
+///
+/// Write the input value as LEB128 into the outptu buffer and return the number of bytes
+/// written.
+/// If the output buffer size is insufficient, return 0 but the output may have been
+/// written to.
+///
+/// \see https://en.wikipedia.org/wiki/LEB128
+/// \see MaxLEB128ByteLenFor
+template <typename Int>
+constexpr int32_t WriteLEB128(Int value, uint8_t* out, int32_t max_out_size) {
+ constexpr Int kLow7Mask = Int(0x7F);
+ constexpr Int kHigh7Mask = ~kLow7Mask;
+ constexpr uint8_t kContinuationBit = 0x80;
+
+ auto const out_first = out;
+
+ // Write as many bytes as the could be for the given input
+ while ((value & kHigh7Mask) != Int(0)) {
+ // We do not have enough room to write the LEB128
+ if (out - out_first >= max_out_size) {
+ return 0;
+ }
+
+ // Write the encoded byte with continuation bit
+ *out = static_cast<uint8_t>(value & kLow7Mask) | kContinuationBit;
+ ++out;
+ // Shift remaining data
+ value >>= 7;
+ }
+
+ // We do not have enough room to write the LEB128
+ if (out - out_first >= max_out_size) {
+ return 0;
+ }
+
+ // Write last non-continuing byte
+ *out = static_cast<uint8_t>(value & kLow7Mask);
+ ++out;
+
+ return static_cast<int32_t>(out - out_first);
+}
+
+/// Parse a leading LEB128
+///
+/// Take as input a data pointer and the maximum number of bytes that can be read from it
+/// (typically the array size).
+/// When a valid LEB128 is found at the start of the data, the function writes it to the
+/// out pointer and return the number of bytes read.
+/// Otherwise, the out pointer is unmodified and zero is returned.
+///
+/// \see https://en.wikipedia.org/wiki/LEB128
+/// \see MaxLEB128ByteLenFor
+template <typename Int>
+constexpr int32_t ParseLeadingLEB128(uint8_t const* data, int32_t max_data_size,
+ Int* out) {
+ constexpr auto kMaxBytes = static_cast<int32_t>(MaxLEB128ByteLenFor<Int>);
+ static_assert(kMaxBytes >= 1);
+ constexpr uint8_t kLow7Mask = 0x7F;
+ constexpr uint8_t kContinuationBit = 0x80;
+ constexpr int32_t kSignBitCount = std::is_signed_v<Int> ? 1 : 0;
+ // Number of bits allowed for encoding data on the last byte to avoid overflow
+ constexpr uint8_t kHighBitCount = (8 * sizeof(Int) - kSignBitCount) % 7;
+ // kHighBitCount least significant `0` bits and the rest with `1`
+ constexpr uint8_t kHighForbiddenMask = ~((1 << kHighBitCount) - 1);
+
+ // Iteratively building the value
+ std::make_unsigned_t<Int> value = 0;
+
+ // Read as many bytes as the could be for the give output
Review Comment:
```suggestion
// Read as many bytes as we could be for the given output.
// `kMaxBytes - 1` bytes of LEB128 data can always fit in the output type,
// while `kMaxBytes` bytes may lead to overflow (special-cased later on).
```
##########
cpp/src/arrow/util/bit_util_test.cc:
##########
@@ -1997,11 +1997,119 @@ TEST(BitUtil, RoundUpToPowerOf2) {
#undef U64
#undef S64
+/// Test the maximum number of bytes needed to write a LEB128 of a give size.
+TEST(BitStreamUtil, MaxLEB128ByteLenFor) {
+ EXPECT_EQ(bit_util::MaxLEB128ByteLenFor<int16_t>, 3);
+ EXPECT_EQ(bit_util::MaxLEB128ByteLenFor<int32_t>, 5);
+ EXPECT_EQ(bit_util::MaxLEB128ByteLenFor<int64_t>, 10);
+}
+
+/// Utility function to test LEB128 encoding with known input value and expected byte
+/// array
+template <typename Int, std::size_t N>
+void TestLEB128Encode(Int input_value, std::array<uint8_t, N> const& expected_data,
+ int32_t expected_bytes_written) {
+ std::array<uint8_t, N> buffer{};
+ auto bytes_written = bit_util::WriteLEB128(input_value, buffer.data(),
+ static_cast<int32_t>(buffer.size()));
+ EXPECT_EQ(bytes_written, expected_bytes_written);
+ if (bytes_written > 0) {
+ EXPECT_EQ(buffer, expected_data);
+ }
+}
+
+/// Test encoding to known LEB128 byte sequences
+TEST(WriteLEB128Test, KnownArrayValues) {
+ // Single byte value 0
+ TestLEB128Encode(0U, std::array<uint8_t, 1>{0x00}, 1);
+ // Single byte value 127
+ TestLEB128Encode(127U, std::array<uint8_t, 1>{0x7F}, 1);
+ // Two byte value 128
+ TestLEB128Encode(128U, std::array<uint8_t, 2>{0x80, 0x01}, 2);
+ // Two byte value 300
+ TestLEB128Encode(300U, std::array<uint8_t, 2>{0xAC, 0x02}, 2);
+ // Three byte value 16384
+ TestLEB128Encode(16384U, std::array<uint8_t, 3>{0x80, 0x80, 0x01}, 3);
+ // Four byte value 268435455
+ TestLEB128Encode(268435455U, std::array<uint8_t, 4>{0xFF, 0xFF, 0xFF, 0x7F}, 4);
+ // Five byte uint32_t max value
+ TestLEB128Encode(4294967295U, std::array<uint8_t, 5>{0xFF, 0xFF, 0xFF, 0xFF, 0x0F}, 5);
+ // uint64_t value requiring 10 bytes
+ TestLEB128Encode(
+ 18446744073709551615ULL,
+ std::array<uint8_t, 10>{0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x01},
+ 10);
+ // Edge case: Exact buffer size match
+ TestLEB128Encode(16384U, std::array<uint8_t, 3>{0x80, 0x80, 0x01}, 3);
+ // Various single byte values
+ TestLEB128Encode(1U, std::array<uint8_t, 1>{0x01}, 1);
+ TestLEB128Encode(63U, std::array<uint8_t, 1>{0x3F}, 1);
+ TestLEB128Encode(64U, std::array<uint8_t, 1>{0x40}, 1);
+ // Two byte boundary values
+ TestLEB128Encode(129U, std::array<uint8_t, 2>{0x81, 0x01}, 2);
+ TestLEB128Encode(16383U, std::array<uint8_t, 2>{0xFF, 0x7F}, 2);
+ // Error case: Buffer too small for value 128 (needs 2 bytes but only 1 provided)
+ TestLEB128Encode(128U, std::array<uint8_t, 1>{}, 0);
+ // Error case: Buffer too small for uint32_t max (needs 5 bytes but only 4 provided)
+ TestLEB128Encode(4294967295U, std::array<uint8_t, 4>{}, 0);
+ // Error case: Zero buffer size
+ TestLEB128Encode(52U, std::array<uint8_t, 0>{}, 0);
+}
+
+/// Utility function to test LEB128 decoding with known byte array and expected result
+template <typename Int, std::size_t N>
+void TestLEB128Decode(std::array<uint8_t, N> const& data, Int expected_value,
+ int32_t expected_bytes_read) {
+ Int result = 0;
+ auto bytes_read = bit_util::ParseLeadingLEB128(
+ data.data(), static_cast<int32_t>(data.size()), &result);
+ EXPECT_EQ(bytes_read, expected_bytes_read);
+ if (expected_bytes_read > 0) {
+ EXPECT_EQ(result, expected_value);
+ }
+}
+
+/// Test decoding from known LEB128 byte sequences
+TEST(BitStreamUtil, LEB128) {
+ // Single byte value 0
+ TestLEB128Decode(std::array<uint8_t, 1>{0x00}, 0U, 1);
+ // Single byte value 127
+ TestLEB128Decode(std::array<uint8_t, 1>{0x7F}, 127U, 1);
+ // Two byte value 128
+ TestLEB128Decode(std::array<uint8_t, 2>{0x80, 0x01}, 128U, 2);
+ // Two byte value 300
+ TestLEB128Decode(std::array<uint8_t, 2>{0xAC, 0x02}, 300U, 2);
+ // Three byte value 16384
+ TestLEB128Decode(std::array<uint8_t, 3>{0x80, 0x80, 0x01}, 16384U, 3);
+ // Three byte value 16384, with remaining data
+ TestLEB128Decode(std::array<uint8_t, 5>{0x80, 0x80, 0x01, 0x80, 0x00}, 16384U, 3);
+ // Four byte value 268435455
+ TestLEB128Decode(std::array<uint8_t, 4>{0xFF, 0xFF, 0xFF, 0x7F}, 268435455U, 4);
+ // Five byte uint32_t max value
+ TestLEB128Decode(std::array<uint8_t, 5>{0xFF, 0xFF, 0xFF, 0xFF, 0x0F}, 4294967295U, 5);
+ // uint64_t value requiring 10 bytes
+ TestLEB128Decode(
+ std::array<uint8_t, 10>{0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x01},
+ 18446744073709551615ULL, 10);
+ // int32_t with maximum size (31 bits of 1)
+ TestLEB128Decode(std::array<uint8_t, 5>{0xFF, 0xFF, 0xFF, 0xFF, 0x7},
+ std::numeric_limits<int32_t>::max(), 5);
+
+ // Error case: Truncated sequence (continuation bit set but no more data)
+ TestLEB128Decode(std::array<uint8_t, 1>{0x80}, 0U, 0);
+ // Error case: Input over the maximum number of bytes for a int32_t (5), but the
+ // overflow none the less (7 * 5 = 35 bits of data).
Review Comment:
This comment isn't clear to me.
```suggestion
// Error case: Input has exactly the maximum number of bytes for a int32_t (5),
// but the decoded value overflows nonetheless (7 * 5 = 35 bits of data).
```
##########
cpp/src/arrow/util/bit_stream_utils_internal.h:
##########
@@ -450,22 +450,35 @@ inline bool BitWriter::PutVlqInt(uint32_t v) {
}
inline bool BitReader::GetVlqInt(uint32_t* v) {
- uint32_t tmp = 0;
-
- for (int i = 0; i < kMaxVlqByteLength; i++) {
- uint8_t byte = 0;
- if (ARROW_PREDICT_FALSE(!GetAligned<uint8_t>(1, &byte))) {
- return false;
- }
- tmp |= static_cast<uint32_t>(byte & 0x7F) << (7 * i);
+ // The data that we will pass to the LEB128 parser
+ // In all case, we read an byte-aligned value, skipping remaining bits
+ uint8_t const* data = NULLPTR;
+ int max_size = 0;
+
+ // Number of bytes left in the buffered values, not including the current
+ // byte (i.e., there may be an additional fraction of a byte).
+ int const bytes_left_in_cache =
+ sizeof(buffered_values_) - static_cast<int>(bit_util::BytesForBits(bit_offset_));
+
+ // If there are clearly enough bytes left we can try to parse from the cache
+ if (bytes_left_in_cache >= kMaxVlqByteLengthForInt32) {
+ max_size = bytes_left_in_cache;
+ data = reinterpret_cast<uint8_t const*>(&buffered_values_) +
+ bit_util::BytesForBits(bit_offset_);
+ // Otherwise, we try straight from buffer (ignoring few bytes that may be cached)
+ } else {
+ max_size = bytes_left();
+ data = buffer_ + (max_bytes_ - max_size);
+ }
- if ((byte & 0x80) == 0) {
- *v = tmp;
- return true;
- }
+ auto const read = bit_util::ParseLeadingLEB128(data, max_size, v);
+ if (ARROW_PREDICT_FALSE(read == 0)) {
+ // Corrupt LEB128
+ return false;
}
- return false;
+ // Advance for the bytes we have read + the bit we skipped
Review Comment:
```suggestion
// Advance for the bytes we have read + the bits we skipped
```
##########
cpp/src/arrow/util/bit_stream_utils_internal.h:
##########
@@ -450,22 +450,35 @@ inline bool BitWriter::PutVlqInt(uint32_t v) {
}
inline bool BitReader::GetVlqInt(uint32_t* v) {
Review Comment:
I'm curious, why are you changing 32-bit `GetVlqInt` but not the 64-bit one?
##########
cpp/src/arrow/util/bit_util_test.cc:
##########
@@ -1997,11 +1997,119 @@ TEST(BitUtil, RoundUpToPowerOf2) {
#undef U64
#undef S64
+/// Test the maximum number of bytes needed to write a LEB128 of a give size.
+TEST(BitStreamUtil, MaxLEB128ByteLenFor) {
+ EXPECT_EQ(bit_util::MaxLEB128ByteLenFor<int16_t>, 3);
+ EXPECT_EQ(bit_util::MaxLEB128ByteLenFor<int32_t>, 5);
+ EXPECT_EQ(bit_util::MaxLEB128ByteLenFor<int64_t>, 10);
+}
+
+/// Utility function to test LEB128 encoding with known input value and expected byte
+/// array
+template <typename Int, std::size_t N>
+void TestLEB128Encode(Int input_value, std::array<uint8_t, N> const& expected_data,
+ int32_t expected_bytes_written) {
+ std::array<uint8_t, N> buffer{};
+ auto bytes_written = bit_util::WriteLEB128(input_value, buffer.data(),
+ static_cast<int32_t>(buffer.size()));
+ EXPECT_EQ(bytes_written, expected_bytes_written);
+ if (bytes_written > 0) {
+ EXPECT_EQ(buffer, expected_data);
+ }
+}
+
+/// Test encoding to known LEB128 byte sequences
+TEST(WriteLEB128Test, KnownArrayValues) {
Review Comment:
The only Int type ever tested here is `unsigned int`, do we want to add a couple more tests for other supported types? They don't need to be as comprehensive as these ones, of course. Just the basics.
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -84,32 +85,278 @@ namespace util {
/// (total 26 bytes, 1 byte overhead)
//
+class RleRun {
+ public:
+ using byte = uint8_t;
Review Comment:
I think this alias is a bit useless, `uint8_t` is the convention for all raw byte data across Arrow.
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -84,32 +85,278 @@ namespace util {
/// (total 26 bytes, 1 byte overhead)
//
+class RleRun {
+ public:
+ using byte = uint8_t;
+ /// Enough space to store a 64bit value
+ using raw_data_storage = std::array<byte, 8>;
+ using raw_data_const_pointer = const byte*;
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr RleRun() noexcept = default;
+ constexpr RleRun(RleRun const&) noexcept = default;
+ constexpr RleRun(RleRun&&) noexcept = default;
+
+ explicit RleRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr RleRun& operator=(RleRun const&) noexcept = default;
+ constexpr RleRun& operator=(RleRun&&) noexcept = default;
+
+ /// The number of repeated values in this run.
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ /// A pointer to the repeated value raw bytes.
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ /// The number of bytes used for the raw repeated value.
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The repeated value raw bytes stored inside the class
+ raw_data_storage data_ = {};
+ /// The number of time the value is repeated
+ values_count_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+class BitPackedRun {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// According to the Parquet thrift definition the page size can be written into an
+ /// int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr BitPackedRun() noexcept = default;
+ constexpr BitPackedRun(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun(BitPackedRun&&) noexcept = default;
+
+ constexpr BitPackedRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr BitPackedRun& operator=(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun& operator=(BitPackedRun&&) noexcept = default;
+
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Number of values in this run.
+ raw_data_size_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+/// A parser that emits either a ``BitPackedRun`` or a ``RleRun``.
+class RleBitPackedParser {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// By Parquet thrift definition the page size can be written into an int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+ /// The different types of runs emitted by the parser
+ using dynamic_run_type = std::variant<RleRun, BitPackedRun>;
+
+ constexpr RleBitPackedParser() noexcept = default;
+
+ constexpr RleBitPackedParser(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr void Reset(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width_) noexcept;
+
+ /// Get the current run with a small parsing cost without advancing the iteration.
+ [[nodiscard]] std::optional<dynamic_run_type> Peek() const;
+
+ /// Move to the next run.
+ [[nodiscard]] bool Advance();
+
+ /// Advance and return the current run.
+ [[nodiscard]] std::optional<dynamic_run_type> Next();
+
+ /// Whether there is still runs to iterate over.
+ ///
+ /// WARN: Due to lack of proper error handling, iteration with Next and Peek could
+ /// return not data while the parser is not exhausted.
+ /// This is how one can check for errors.
+ [[nodiscard]] bool Exhausted() const;
+
+ /// Enum to return from an ``Parse`` handler.
+ ///
+ /// Since a callback has no way to know when to stop, the handler must return
+ /// a value indicating to the ``Parse`` function whether to stop or continue.
+ enum class ControlFlow {
+ Continue,
+ Break,
+ };
+
+ /// A callback approach to parsing.
+ ///
+ /// This approach is used to reduce the number of dynamic lookups involved with using a
+ /// variant.
+ ///
+ /// The handler must be of the form
+ /// ```cpp`
+ /// struct Handler {
+ /// ControlFlow OnBitPackedRun(BitPackedRun run);
+ ///
+ /// ControlFlow OnRleRun(RleRun run);
+ /// };
+ /// ```
+ template <typename Handler>
+ void Parse(Handler&& handler);
Review Comment:
I'm curious: why do we have both a callback-based parser and an iteration API? Are both needed? Otherwise, we should keep only the most convenient and performant, IMHO.
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -84,32 +85,278 @@ namespace util {
/// (total 26 bytes, 1 byte overhead)
//
+class RleRun {
+ public:
+ using byte = uint8_t;
+ /// Enough space to store a 64bit value
+ using raw_data_storage = std::array<byte, 8>;
+ using raw_data_const_pointer = const byte*;
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr RleRun() noexcept = default;
+ constexpr RleRun(RleRun const&) noexcept = default;
+ constexpr RleRun(RleRun&&) noexcept = default;
+
+ explicit RleRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr RleRun& operator=(RleRun const&) noexcept = default;
+ constexpr RleRun& operator=(RleRun&&) noexcept = default;
+
+ /// The number of repeated values in this run.
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ /// A pointer to the repeated value raw bytes.
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ /// The number of bytes used for the raw repeated value.
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The repeated value raw bytes stored inside the class
+ raw_data_storage data_ = {};
+ /// The number of time the value is repeated
+ values_count_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+class BitPackedRun {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// According to the Parquet thrift definition the page size can be written into an
+ /// int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr BitPackedRun() noexcept = default;
+ constexpr BitPackedRun(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun(BitPackedRun&&) noexcept = default;
+
+ constexpr BitPackedRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr BitPackedRun& operator=(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun& operator=(BitPackedRun&&) noexcept = default;
+
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Number of values in this run.
+ raw_data_size_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+/// A parser that emits either a ``BitPackedRun`` or a ``RleRun``.
+class RleBitPackedParser {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// By Parquet thrift definition the page size can be written into an int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+ /// The different types of runs emitted by the parser
+ using dynamic_run_type = std::variant<RleRun, BitPackedRun>;
+
+ constexpr RleBitPackedParser() noexcept = default;
+
+ constexpr RleBitPackedParser(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr void Reset(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width_) noexcept;
+
+ /// Get the current run with a small parsing cost without advancing the iteration.
+ [[nodiscard]] std::optional<dynamic_run_type> Peek() const;
+
+ /// Move to the next run.
+ [[nodiscard]] bool Advance();
+
+ /// Advance and return the current run.
+ [[nodiscard]] std::optional<dynamic_run_type> Next();
+
+ /// Whether there is still runs to iterate over.
+ ///
+ /// WARN: Due to lack of proper error handling, iteration with Next and Peek could
+ /// return not data while the parser is not exhausted.
+ /// This is how one can check for errors.
+ [[nodiscard]] bool Exhausted() const;
+
+ /// Enum to return from an ``Parse`` handler.
+ ///
+ /// Since a callback has no way to know when to stop, the handler must return
+ /// a value indicating to the ``Parse`` function whether to stop or continue.
+ enum class ControlFlow {
+ Continue,
+ Break,
+ };
+
+ /// A callback approach to parsing.
+ ///
+ /// This approach is used to reduce the number of dynamic lookups involved with using a
+ /// variant.
+ ///
+ /// The handler must be of the form
+ /// ```cpp`
+ /// struct Handler {
+ /// ControlFlow OnBitPackedRun(BitPackedRun run);
+ ///
+ /// ControlFlow OnRleRun(RleRun run);
+ /// };
+ /// ```
+ template <typename Handler>
+ void Parse(Handler&& handler);
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Size in bytes of the run.
+ raw_data_size_type data_size_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+
+ /// Run the handler on the run read and return the number of values read.
+ /// Does not advance the parser.
+ template <typename Handler>
+ std::pair<raw_data_size_type, ControlFlow> PeekImpl(Handler&&) const;
+};
+
/// Decoder class for RLE encoded data.
+template <typename T>
class RleDecoder {
public:
- /// Create a decoder object. buffer/buffer_len is the decoded data.
- /// bit_width is the width of each value (before encoding).
- RleDecoder(const uint8_t* buffer, int buffer_len, int bit_width)
- : bit_reader_(buffer, buffer_len),
- bit_width_(bit_width),
- current_value_(0),
- repeat_count_(0),
- literal_count_(0) {
- ARROW_DCHECK_GE(bit_width_, 0);
- ARROW_DCHECK_LE(bit_width_, 64);
- }
+ /// The type in which the data should be decoded.
+ using value_type = T;
+ /// The type of run that can be decoded.
+ using run_type = RleRun;
+ using values_count_type = run_type::values_count_type;
- RleDecoder() : bit_width_(-1) {}
+ constexpr RleDecoder() noexcept = default;
- void Reset(const uint8_t* buffer, int buffer_len, int bit_width) {
- ARROW_DCHECK_GE(bit_width, 0);
- ARROW_DCHECK_LE(bit_width, 64);
- bit_reader_.Reset(buffer, buffer_len);
- bit_width_ = bit_width;
- current_value_ = 0;
- repeat_count_ = 0;
- literal_count_ = 0;
- }
+ explicit RleDecoder(run_type const& run) noexcept;
+
+ void Reset(run_type const& run) noexcept;
+
+ /// Return the number of values that can be advanced.
+ [[nodiscard]] values_count_type Remaining() const;
+
+ /// Return the repeated value of this decoder.
+ [[nodiscard]] constexpr value_type Value() const;
+
+ /// Try to advance by as many values as provided.
+ /// Return the number of values skipped.
+ [[nodiscard]] values_count_type Advance(values_count_type batch_size);
+
+ /// Get the next value and return false if there are no more.
+ [[nodiscard]] constexpr bool Get(value_type* out_value);
+
+ /// Get a batch of values return the number of decoded elements.
+ [[nodiscard]] values_count_type GetBatch(value_type* out, values_count_type batch_size);
+
+ private:
+ value_type value_ = {};
+ values_count_type remaining_count_ = 0;
+
+ static_assert(std::is_integral_v<value_type>,
+ "This class makes assumptions about integer endianness and padding");
+};
+
+/// Decoder class for Bit packing encoded data.
+template <typename T>
+class BitPackedDecoder {
Review Comment:
Similar comments for this class as for `RleDecoder` above :)
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
Review Comment:
I suppose this is actually initializing a `RleDecoder`?
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
Review Comment:
`ARROW_PREDICT_FALSE` here?
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -118,50 +365,56 @@ class RleDecoder {
/// input with zeros. Since the encoding does not differentiate between
/// input values and padding, Get() returns true even for these padding
/// values.
- template <typename T>
- bool Get(T* val);
+ [[nodiscard]] bool Get(value_type* val);
- /// Gets a batch of values. Returns the number of decoded elements.
- template <typename T>
- int GetBatch(T* values, int batch_size);
+ /// Get a batch of values return the number of decoded elements.
+ [[nodiscard]] values_count_type GetBatch(value_type* out, values_count_type batch_size);
/// Like GetBatch but add spacing for null entries
- template <typename T>
- int GetBatchSpaced(int batch_size, int null_count, const uint8_t* valid_bits,
- int64_t valid_bits_offset, T* out);
+ [[nodiscard]] values_count_type GetBatchSpaced(values_count_type batch_size,
+ values_count_type null_count,
+ const byte* valid_bits,
+ int64_t valid_bits_offset,
+ value_type* out);
/// Like GetBatch but the values are then decoded using the provided dictionary
- template <typename T>
- int GetBatchWithDict(const T* dictionary, int32_t dictionary_length, T* values,
- int batch_size);
+ template <typename V>
+ [[nodiscard]] values_count_type GetBatchWithDict(const V* dictionary,
+ int32_t dictionary_length, V* out,
+ values_count_type batch_size);
/// Like GetBatchWithDict but add spacing for null entries
///
/// Null entries will be zero-initialized in `values` to avoid leaking
/// private data.
- template <typename T>
- int GetBatchWithDictSpaced(const T* dictionary, int32_t dictionary_length, T* values,
- int batch_size, int null_count, const uint8_t* valid_bits,
- int64_t valid_bits_offset);
-
- protected:
- ::arrow::bit_util::BitReader bit_reader_;
- /// Number of bits needed to encode the value. Must be between 0 and 64.
- int bit_width_;
- uint64_t current_value_;
- int32_t repeat_count_;
- int32_t literal_count_;
+ template <typename V>
+ [[nodiscard]] values_count_type GetBatchWithDictSpaced(
+ const V* dictionary, int32_t dictionary_length, V* out,
+ values_count_type batch_size, values_count_type null_count, const byte* valid_bits,
+ int64_t valid_bits_offset);
private:
- /// Fills literal_count_ and repeat_count_ with next values. Returns false if there
- /// are no more.
- template <typename T>
- bool NextCounts();
+ RleBitPackedParser parser_ = {};
+ std::variant<RleDecoder<value_type>, BitPackedDecoder<value_type>> decoder_ = {};
+
+ /// Return the number of values that are remaining in the current run.
+ [[nodiscard]] values_count_type RunRemaining() const;
+
+ /// Get a batch of values from the current run and return the number elements read.
+ [[nodiscard]] values_count_type RunGetBatch(value_type* out,
+ values_count_type batch_size);
+
+ /// Return the number of values that are remaining in the current run.
Review Comment:
The method returns a bool, not a number of values.
##########
cpp/src/arrow/util/bit_util.h:
##########
@@ -365,5 +365,118 @@ void PackBits(const uint32_t* values, uint8_t* out) {
}
}
+constexpr int64_t MaxLEB128ByteLen(int64_t n_bits) { return CeilDiv(n_bits, 7); }
+
+template <typename Int>
+constexpr int64_t MaxLEB128ByteLenFor = MaxLEB128ByteLen(sizeof(Int) * 8);
+
+/// Write a integer as LEB128
+///
+/// Write the input value as LEB128 into the outptu buffer and return the number of bytes
+/// written.
+/// If the output buffer size is insufficient, return 0 but the output may have been
+/// written to.
+///
+/// \see https://en.wikipedia.org/wiki/LEB128
+/// \see MaxLEB128ByteLenFor
+template <typename Int>
+constexpr int32_t WriteLEB128(Int value, uint8_t* out, int32_t max_out_size) {
+ constexpr Int kLow7Mask = Int(0x7F);
+ constexpr Int kHigh7Mask = ~kLow7Mask;
+ constexpr uint8_t kContinuationBit = 0x80;
+
+ auto const out_first = out;
+
+ // Write as many bytes as the could be for the given input
+ while ((value & kHigh7Mask) != Int(0)) {
+ // We do not have enough room to write the LEB128
+ if (out - out_first >= max_out_size) {
Review Comment:
Perhaps adorn this with `ARROW_PREDICT_FALSE` as we generally should reserve enough space?
##########
cpp/src/arrow/util/bit_util_test.cc:
##########
@@ -1997,11 +1997,119 @@ TEST(BitUtil, RoundUpToPowerOf2) {
#undef U64
#undef S64
+/// Test the maximum number of bytes needed to write a LEB128 of a give size.
+TEST(BitStreamUtil, MaxLEB128ByteLenFor) {
Review Comment:
It's nice to have unit tests for this IMHO.
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
Review Comment:
Again, `ARROW_PREDICT_FALSE` does not sound right to me.
##########
cpp/src/arrow/util/bit_util.h:
##########
@@ -365,5 +365,118 @@ void PackBits(const uint32_t* values, uint8_t* out) {
}
}
+constexpr int64_t MaxLEB128ByteLen(int64_t n_bits) { return CeilDiv(n_bits, 7); }
+
+template <typename Int>
+constexpr int64_t MaxLEB128ByteLenFor = MaxLEB128ByteLen(sizeof(Int) * 8);
+
+/// Write a integer as LEB128
+///
+/// Write the input value as LEB128 into the outptu buffer and return the number of bytes
+/// written.
+/// If the output buffer size is insufficient, return 0 but the output may have been
+/// written to.
+///
+/// \see https://en.wikipedia.org/wiki/LEB128
+/// \see MaxLEB128ByteLenFor
+template <typename Int>
+constexpr int32_t WriteLEB128(Int value, uint8_t* out, int32_t max_out_size) {
+ constexpr Int kLow7Mask = Int(0x7F);
+ constexpr Int kHigh7Mask = ~kLow7Mask;
+ constexpr uint8_t kContinuationBit = 0x80;
+
+ auto const out_first = out;
+
+ // Write as many bytes as the could be for the given input
+ while ((value & kHigh7Mask) != Int(0)) {
+ // We do not have enough room to write the LEB128
+ if (out - out_first >= max_out_size) {
+ return 0;
+ }
+
+ // Write the encoded byte with continuation bit
+ *out = static_cast<uint8_t>(value & kLow7Mask) | kContinuationBit;
+ ++out;
+ // Shift remaining data
+ value >>= 7;
+ }
+
+ // We do not have enough room to write the LEB128
+ if (out - out_first >= max_out_size) {
+ return 0;
+ }
+
+ // Write last non-continuing byte
+ *out = static_cast<uint8_t>(value & kLow7Mask);
+ ++out;
+
+ return static_cast<int32_t>(out - out_first);
+}
+
+/// Parse a leading LEB128
+///
+/// Take as input a data pointer and the maximum number of bytes that can be read from it
+/// (typically the array size).
+/// When a valid LEB128 is found at the start of the data, the function writes it to the
+/// out pointer and return the number of bytes read.
+/// Otherwise, the out pointer is unmodified and zero is returned.
+///
+/// \see https://en.wikipedia.org/wiki/LEB128
+/// \see MaxLEB128ByteLenFor
+template <typename Int>
+constexpr int32_t ParseLeadingLEB128(uint8_t const* data, int32_t max_data_size,
+ Int* out) {
+ constexpr auto kMaxBytes = static_cast<int32_t>(MaxLEB128ByteLenFor<Int>);
+ static_assert(kMaxBytes >= 1);
+ constexpr uint8_t kLow7Mask = 0x7F;
+ constexpr uint8_t kContinuationBit = 0x80;
+ constexpr int32_t kSignBitCount = std::is_signed_v<Int> ? 1 : 0;
+ // Number of bits allowed for encoding data on the last byte to avoid overflow
+ constexpr uint8_t kHighBitCount = (8 * sizeof(Int) - kSignBitCount) % 7;
+ // kHighBitCount least significant `0` bits and the rest with `1`
+ constexpr uint8_t kHighForbiddenMask = ~((1 << kHighBitCount) - 1);
+
+ // Iteratively building the value
+ std::make_unsigned_t<Int> value = 0;
+
+ // Read as many bytes as the could be for the give output
+ for (int32_t i = 0; i < kMaxBytes - 1; i++) {
+ // We have not finished reading a valid LEB128, yet we run out of data
Review Comment:
(if this function is performance-critical, there are probably ways to improve this, for example have a fast path for the common case where `max_data_size >= kMaxBytes`)
##########
cpp/src/arrow/util/bit_util.h:
##########
@@ -365,5 +365,118 @@ void PackBits(const uint32_t* values, uint8_t* out) {
}
}
+constexpr int64_t MaxLEB128ByteLen(int64_t n_bits) { return CeilDiv(n_bits, 7); }
+
+template <typename Int>
+constexpr int64_t MaxLEB128ByteLenFor = MaxLEB128ByteLen(sizeof(Int) * 8);
Review Comment:
We generally prefix constants with `k`
```suggestion
constexpr int64_t kMaxLEB128ByteLenFor = MaxLEB128ByteLen(sizeof(Int) * 8);
```
##########
cpp/src/arrow/util/bit_util.h:
##########
@@ -365,5 +365,118 @@ void PackBits(const uint32_t* values, uint8_t* out) {
}
}
+constexpr int64_t MaxLEB128ByteLen(int64_t n_bits) { return CeilDiv(n_bits, 7); }
+
+template <typename Int>
+constexpr int64_t MaxLEB128ByteLenFor = MaxLEB128ByteLen(sizeof(Int) * 8);
+
+/// Write a integer as LEB128
+///
+/// Write the input value as LEB128 into the outptu buffer and return the number of bytes
+/// written.
+/// If the output buffer size is insufficient, return 0 but the output may have been
+/// written to.
+///
+/// \see https://en.wikipedia.org/wiki/LEB128
+/// \see MaxLEB128ByteLenFor
+template <typename Int>
+constexpr int32_t WriteLEB128(Int value, uint8_t* out, int32_t max_out_size) {
+ constexpr Int kLow7Mask = Int(0x7F);
+ constexpr Int kHigh7Mask = ~kLow7Mask;
+ constexpr uint8_t kContinuationBit = 0x80;
+
+ auto const out_first = out;
Review Comment:
`const auto` again :)
##########
cpp/src/arrow/util/bit_util_test.cc:
##########
@@ -1997,11 +1997,119 @@ TEST(BitUtil, RoundUpToPowerOf2) {
#undef U64
#undef S64
+/// Test the maximum number of bytes needed to write a LEB128 of a give size.
+TEST(BitStreamUtil, MaxLEB128ByteLenFor) {
+ EXPECT_EQ(bit_util::MaxLEB128ByteLenFor<int16_t>, 3);
+ EXPECT_EQ(bit_util::MaxLEB128ByteLenFor<int32_t>, 5);
+ EXPECT_EQ(bit_util::MaxLEB128ByteLenFor<int64_t>, 10);
+}
+
+/// Utility function to test LEB128 encoding with known input value and expected byte
+/// array
+template <typename Int, std::size_t N>
+void TestLEB128Encode(Int input_value, std::array<uint8_t, N> const& expected_data,
+ int32_t expected_bytes_written) {
+ std::array<uint8_t, N> buffer{};
+ auto bytes_written = bit_util::WriteLEB128(input_value, buffer.data(),
+ static_cast<int32_t>(buffer.size()));
Review Comment:
Perhaps also test passing a larger output buffer?
##########
cpp/src/arrow/util/bit_util_test.cc:
##########
@@ -1997,11 +1997,119 @@ TEST(BitUtil, RoundUpToPowerOf2) {
#undef U64
#undef S64
+/// Test the maximum number of bytes needed to write a LEB128 of a give size.
+TEST(BitStreamUtil, MaxLEB128ByteLenFor) {
+ EXPECT_EQ(bit_util::MaxLEB128ByteLenFor<int16_t>, 3);
+ EXPECT_EQ(bit_util::MaxLEB128ByteLenFor<int32_t>, 5);
+ EXPECT_EQ(bit_util::MaxLEB128ByteLenFor<int64_t>, 10);
+}
+
+/// Utility function to test LEB128 encoding with known input value and expected byte
+/// array
+template <typename Int, std::size_t N>
+void TestLEB128Encode(Int input_value, std::array<uint8_t, N> const& expected_data,
+ int32_t expected_bytes_written) {
+ std::array<uint8_t, N> buffer{};
+ auto bytes_written = bit_util::WriteLEB128(input_value, buffer.data(),
+ static_cast<int32_t>(buffer.size()));
+ EXPECT_EQ(bytes_written, expected_bytes_written);
+ if (bytes_written > 0) {
+ EXPECT_EQ(buffer, expected_data);
+ }
+}
+
+/// Test encoding to known LEB128 byte sequences
+TEST(WriteLEB128Test, KnownArrayValues) {
+ // Single byte value 0
+ TestLEB128Encode(0U, std::array<uint8_t, 1>{0x00}, 1);
+ // Single byte value 127
+ TestLEB128Encode(127U, std::array<uint8_t, 1>{0x7F}, 1);
+ // Two byte value 128
+ TestLEB128Encode(128U, std::array<uint8_t, 2>{0x80, 0x01}, 2);
+ // Two byte value 300
+ TestLEB128Encode(300U, std::array<uint8_t, 2>{0xAC, 0x02}, 2);
+ // Three byte value 16384
+ TestLEB128Encode(16384U, std::array<uint8_t, 3>{0x80, 0x80, 0x01}, 3);
+ // Four byte value 268435455
+ TestLEB128Encode(268435455U, std::array<uint8_t, 4>{0xFF, 0xFF, 0xFF, 0x7F}, 4);
+ // Five byte uint32_t max value
+ TestLEB128Encode(4294967295U, std::array<uint8_t, 5>{0xFF, 0xFF, 0xFF, 0xFF, 0x0F}, 5);
+ // uint64_t value requiring 10 bytes
+ TestLEB128Encode(
+ 18446744073709551615ULL,
+ std::array<uint8_t, 10>{0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x01},
+ 10);
+ // Edge case: Exact buffer size match
+ TestLEB128Encode(16384U, std::array<uint8_t, 3>{0x80, 0x80, 0x01}, 3);
+ // Various single byte values
+ TestLEB128Encode(1U, std::array<uint8_t, 1>{0x01}, 1);
+ TestLEB128Encode(63U, std::array<uint8_t, 1>{0x3F}, 1);
+ TestLEB128Encode(64U, std::array<uint8_t, 1>{0x40}, 1);
+ // Two byte boundary values
+ TestLEB128Encode(129U, std::array<uint8_t, 2>{0x81, 0x01}, 2);
+ TestLEB128Encode(16383U, std::array<uint8_t, 2>{0xFF, 0x7F}, 2);
+ // Error case: Buffer too small for value 128 (needs 2 bytes but only 1 provided)
+ TestLEB128Encode(128U, std::array<uint8_t, 1>{}, 0);
+ // Error case: Buffer too small for uint32_t max (needs 5 bytes but only 4 provided)
+ TestLEB128Encode(4294967295U, std::array<uint8_t, 4>{}, 0);
+ // Error case: Zero buffer size
+ TestLEB128Encode(52U, std::array<uint8_t, 0>{}, 0);
+}
+
+/// Utility function to test LEB128 decoding with known byte array and expected result
+template <typename Int, std::size_t N>
+void TestLEB128Decode(std::array<uint8_t, N> const& data, Int expected_value,
+ int32_t expected_bytes_read) {
+ Int result = 0;
+ auto bytes_read = bit_util::ParseLeadingLEB128(
+ data.data(), static_cast<int32_t>(data.size()), &result);
+ EXPECT_EQ(bytes_read, expected_bytes_read);
+ if (expected_bytes_read > 0) {
+ EXPECT_EQ(result, expected_value);
+ }
+}
+
+/// Test decoding from known LEB128 byte sequences
+TEST(BitStreamUtil, LEB128) {
+ // Single byte value 0
+ TestLEB128Decode(std::array<uint8_t, 1>{0x00}, 0U, 1);
+ // Single byte value 127
+ TestLEB128Decode(std::array<uint8_t, 1>{0x7F}, 127U, 1);
+ // Two byte value 128
+ TestLEB128Decode(std::array<uint8_t, 2>{0x80, 0x01}, 128U, 2);
+ // Two byte value 300
+ TestLEB128Decode(std::array<uint8_t, 2>{0xAC, 0x02}, 300U, 2);
+ // Three byte value 16384
+ TestLEB128Decode(std::array<uint8_t, 3>{0x80, 0x80, 0x01}, 16384U, 3);
+ // Three byte value 16384, with remaining data
+ TestLEB128Decode(std::array<uint8_t, 5>{0x80, 0x80, 0x01, 0x80, 0x00}, 16384U, 3);
+ // Four byte value 268435455
+ TestLEB128Decode(std::array<uint8_t, 4>{0xFF, 0xFF, 0xFF, 0x7F}, 268435455U, 4);
+ // Five byte uint32_t max value
+ TestLEB128Decode(std::array<uint8_t, 5>{0xFF, 0xFF, 0xFF, 0xFF, 0x0F}, 4294967295U, 5);
+ // uint64_t value requiring 10 bytes
+ TestLEB128Decode(
+ std::array<uint8_t, 10>{0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x01},
+ 18446744073709551615ULL, 10);
+ // int32_t with maximum size (31 bits of 1)
+ TestLEB128Decode(std::array<uint8_t, 5>{0xFF, 0xFF, 0xFF, 0xFF, 0x7},
+ std::numeric_limits<int32_t>::max(), 5);
+
+ // Error case: Truncated sequence (continuation bit set but no more data)
+ TestLEB128Decode(std::array<uint8_t, 1>{0x80}, 0U, 0);
+ // Error case: Input over the maximum number of bytes for a int32_t (5), but the
+ // overflow none the less (7 * 5 = 35 bits of data).
+ TestLEB128Decode(std::array<uint8_t, 5>{0xFF, 0xFF, 0xFF, 0xFF, 0x7F}, int32_t{}, 0);
Review Comment:
Perhaps also test a value that is just overflowing, and also test that the value `0xffffffff` can be decoded correctly?
##########
cpp/src/arrow/util/bit_util_test.cc:
##########
@@ -1997,11 +1997,119 @@ TEST(BitUtil, RoundUpToPowerOf2) {
#undef U64
#undef S64
+/// Test the maximum number of bytes needed to write a LEB128 of a give size.
+TEST(BitStreamUtil, MaxLEB128ByteLenFor) {
+ EXPECT_EQ(bit_util::MaxLEB128ByteLenFor<int16_t>, 3);
+ EXPECT_EQ(bit_util::MaxLEB128ByteLenFor<int32_t>, 5);
+ EXPECT_EQ(bit_util::MaxLEB128ByteLenFor<int64_t>, 10);
+}
+
+/// Utility function to test LEB128 encoding with known input value and expected byte
+/// array
+template <typename Int, std::size_t N>
Review Comment:
Templating on `N` doesn't seem useful, why not use a `std::vector`? For tests, compilation time is more important than runtime micro-optimization.
##########
cpp/src/arrow/util/bit_util.h:
##########
@@ -365,5 +365,118 @@ void PackBits(const uint32_t* values, uint8_t* out) {
}
}
+constexpr int64_t MaxLEB128ByteLen(int64_t n_bits) { return CeilDiv(n_bits, 7); }
+
+template <typename Int>
+constexpr int64_t MaxLEB128ByteLenFor = MaxLEB128ByteLen(sizeof(Int) * 8);
+
+/// Write a integer as LEB128
+///
+/// Write the input value as LEB128 into the outptu buffer and return the number of bytes
+/// written.
+/// If the output buffer size is insufficient, return 0 but the output may have been
+/// written to.
+///
+/// \see https://en.wikipedia.org/wiki/LEB128
+/// \see MaxLEB128ByteLenFor
+template <typename Int>
+constexpr int32_t WriteLEB128(Int value, uint8_t* out, int32_t max_out_size) {
+ constexpr Int kLow7Mask = Int(0x7F);
+ constexpr Int kHigh7Mask = ~kLow7Mask;
+ constexpr uint8_t kContinuationBit = 0x80;
+
+ auto const out_first = out;
+
+ // Write as many bytes as the could be for the given input
+ while ((value & kHigh7Mask) != Int(0)) {
+ // We do not have enough room to write the LEB128
+ if (out - out_first >= max_out_size) {
+ return 0;
+ }
+
+ // Write the encoded byte with continuation bit
+ *out = static_cast<uint8_t>(value & kLow7Mask) | kContinuationBit;
+ ++out;
+ // Shift remaining data
+ value >>= 7;
Review Comment:
Does `Int` have to be unsigned? Otherwise, a negative number could always remain non-zero when shifted right.
##########
cpp/src/arrow/util/bit_util_test.cc:
##########
@@ -1997,11 +1997,119 @@ TEST(BitUtil, RoundUpToPowerOf2) {
#undef U64
#undef S64
+/// Test the maximum number of bytes needed to write a LEB128 of a give size.
+TEST(BitStreamUtil, MaxLEB128ByteLenFor) {
+ EXPECT_EQ(bit_util::MaxLEB128ByteLenFor<int16_t>, 3);
+ EXPECT_EQ(bit_util::MaxLEB128ByteLenFor<int32_t>, 5);
+ EXPECT_EQ(bit_util::MaxLEB128ByteLenFor<int64_t>, 10);
+}
+
+/// Utility function to test LEB128 encoding with known input value and expected byte
+/// array
+template <typename Int, std::size_t N>
+void TestLEB128Encode(Int input_value, std::array<uint8_t, N> const& expected_data,
+ int32_t expected_bytes_written) {
+ std::array<uint8_t, N> buffer{};
+ auto bytes_written = bit_util::WriteLEB128(input_value, buffer.data(),
+ static_cast<int32_t>(buffer.size()));
+ EXPECT_EQ(bytes_written, expected_bytes_written);
+ if (bytes_written > 0) {
+ EXPECT_EQ(buffer, expected_data);
+ }
+}
+
+/// Test encoding to known LEB128 byte sequences
+TEST(WriteLEB128Test, KnownArrayValues) {
+ // Single byte value 0
+ TestLEB128Encode(0U, std::array<uint8_t, 1>{0x00}, 1);
+ // Single byte value 127
+ TestLEB128Encode(127U, std::array<uint8_t, 1>{0x7F}, 1);
+ // Two byte value 128
+ TestLEB128Encode(128U, std::array<uint8_t, 2>{0x80, 0x01}, 2);
+ // Two byte value 300
+ TestLEB128Encode(300U, std::array<uint8_t, 2>{0xAC, 0x02}, 2);
+ // Three byte value 16384
+ TestLEB128Encode(16384U, std::array<uint8_t, 3>{0x80, 0x80, 0x01}, 3);
+ // Four byte value 268435455
+ TestLEB128Encode(268435455U, std::array<uint8_t, 4>{0xFF, 0xFF, 0xFF, 0x7F}, 4);
+ // Five byte uint32_t max value
+ TestLEB128Encode(4294967295U, std::array<uint8_t, 5>{0xFF, 0xFF, 0xFF, 0xFF, 0x0F}, 5);
+ // uint64_t value requiring 10 bytes
+ TestLEB128Encode(
+ 18446744073709551615ULL,
+ std::array<uint8_t, 10>{0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x01},
+ 10);
+ // Edge case: Exact buffer size match
Review Comment:
Why is this particular one an "exact buffer size match"?
Also, the same test is already above.
##########
cpp/src/arrow/util/bit_stream_utils_internal.h:
##########
@@ -450,22 +450,35 @@ inline bool BitWriter::PutVlqInt(uint32_t v) {
}
inline bool BitReader::GetVlqInt(uint32_t* v) {
- uint32_t tmp = 0;
-
- for (int i = 0; i < kMaxVlqByteLength; i++) {
- uint8_t byte = 0;
- if (ARROW_PREDICT_FALSE(!GetAligned<uint8_t>(1, &byte))) {
- return false;
- }
- tmp |= static_cast<uint32_t>(byte & 0x7F) << (7 * i);
+ // The data that we will pass to the LEB128 parser
+ // In all case, we read an byte-aligned value, skipping remaining bits
+ uint8_t const* data = NULLPTR;
+ int max_size = 0;
+
+ // Number of bytes left in the buffered values, not including the current
+ // byte (i.e., there may be an additional fraction of a byte).
+ int const bytes_left_in_cache =
+ sizeof(buffered_values_) - static_cast<int>(bit_util::BytesForBits(bit_offset_));
+
+ // If there are clearly enough bytes left we can try to parse from the cache
+ if (bytes_left_in_cache >= kMaxVlqByteLengthForInt32) {
+ max_size = bytes_left_in_cache;
+ data = reinterpret_cast<uint8_t const*>(&buffered_values_) +
+ bit_util::BytesForBits(bit_offset_);
+ // Otherwise, we try straight from buffer (ignoring few bytes that may be cached)
+ } else {
+ max_size = bytes_left();
+ data = buffer_ + (max_bytes_ - max_size);
+ }
- if ((byte & 0x80) == 0) {
- *v = tmp;
- return true;
- }
+ auto const read = bit_util::ParseLeadingLEB128(data, max_size, v);
+ if (ARROW_PREDICT_FALSE(read == 0)) {
Review Comment:
```suggestion
const auto bytes_read = bit_util::ParseLeadingLEB128(data, max_size, v);
if (ARROW_PREDICT_FALSE(bytes_read == 0)) {
```
##########
cpp/src/arrow/util/bit_util_test.cc:
##########
@@ -1997,11 +1997,119 @@ TEST(BitUtil, RoundUpToPowerOf2) {
#undef U64
#undef S64
+/// Test the maximum number of bytes needed to write a LEB128 of a give size.
+TEST(BitStreamUtil, MaxLEB128ByteLenFor) {
+ EXPECT_EQ(bit_util::MaxLEB128ByteLenFor<int16_t>, 3);
+ EXPECT_EQ(bit_util::MaxLEB128ByteLenFor<int32_t>, 5);
+ EXPECT_EQ(bit_util::MaxLEB128ByteLenFor<int64_t>, 10);
+}
+
+/// Utility function to test LEB128 encoding with known input value and expected byte
+/// array
+template <typename Int, std::size_t N>
+void TestLEB128Encode(Int input_value, std::array<uint8_t, N> const& expected_data,
+ int32_t expected_bytes_written) {
+ std::array<uint8_t, N> buffer{};
+ auto bytes_written = bit_util::WriteLEB128(input_value, buffer.data(),
+ static_cast<int32_t>(buffer.size()));
+ EXPECT_EQ(bytes_written, expected_bytes_written);
+ if (bytes_written > 0) {
+ EXPECT_EQ(buffer, expected_data);
+ }
+}
+
+/// Test encoding to known LEB128 byte sequences
+TEST(WriteLEB128Test, KnownArrayValues) {
+ // Single byte value 0
+ TestLEB128Encode(0U, std::array<uint8_t, 1>{0x00}, 1);
+ // Single byte value 127
+ TestLEB128Encode(127U, std::array<uint8_t, 1>{0x7F}, 1);
+ // Two byte value 128
+ TestLEB128Encode(128U, std::array<uint8_t, 2>{0x80, 0x01}, 2);
+ // Two byte value 300
+ TestLEB128Encode(300U, std::array<uint8_t, 2>{0xAC, 0x02}, 2);
+ // Three byte value 16384
+ TestLEB128Encode(16384U, std::array<uint8_t, 3>{0x80, 0x80, 0x01}, 3);
+ // Four byte value 268435455
+ TestLEB128Encode(268435455U, std::array<uint8_t, 4>{0xFF, 0xFF, 0xFF, 0x7F}, 4);
+ // Five byte uint32_t max value
+ TestLEB128Encode(4294967295U, std::array<uint8_t, 5>{0xFF, 0xFF, 0xFF, 0xFF, 0x0F}, 5);
+ // uint64_t value requiring 10 bytes
+ TestLEB128Encode(
+ 18446744073709551615ULL,
+ std::array<uint8_t, 10>{0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x01},
+ 10);
+ // Edge case: Exact buffer size match
+ TestLEB128Encode(16384U, std::array<uint8_t, 3>{0x80, 0x80, 0x01}, 3);
+ // Various single byte values
+ TestLEB128Encode(1U, std::array<uint8_t, 1>{0x01}, 1);
+ TestLEB128Encode(63U, std::array<uint8_t, 1>{0x3F}, 1);
+ TestLEB128Encode(64U, std::array<uint8_t, 1>{0x40}, 1);
+ // Two byte boundary values
+ TestLEB128Encode(129U, std::array<uint8_t, 2>{0x81, 0x01}, 2);
+ TestLEB128Encode(16383U, std::array<uint8_t, 2>{0xFF, 0x7F}, 2);
+ // Error case: Buffer too small for value 128 (needs 2 bytes but only 1 provided)
+ TestLEB128Encode(128U, std::array<uint8_t, 1>{}, 0);
+ // Error case: Buffer too small for uint32_t max (needs 5 bytes but only 4 provided)
+ TestLEB128Encode(4294967295U, std::array<uint8_t, 4>{}, 0);
+ // Error case: Zero buffer size
+ TestLEB128Encode(52U, std::array<uint8_t, 0>{}, 0);
+}
+
+/// Utility function to test LEB128 decoding with known byte array and expected result
+template <typename Int, std::size_t N>
+void TestLEB128Decode(std::array<uint8_t, N> const& data, Int expected_value,
+ int32_t expected_bytes_read) {
+ Int result = 0;
+ auto bytes_read = bit_util::ParseLeadingLEB128(
+ data.data(), static_cast<int32_t>(data.size()), &result);
+ EXPECT_EQ(bytes_read, expected_bytes_read);
+ if (expected_bytes_read > 0) {
+ EXPECT_EQ(result, expected_value);
+ }
+}
+
+/// Test decoding from known LEB128 byte sequences
+TEST(BitStreamUtil, LEB128) {
+ // Single byte value 0
+ TestLEB128Decode(std::array<uint8_t, 1>{0x00}, 0U, 1);
+ // Single byte value 127
+ TestLEB128Decode(std::array<uint8_t, 1>{0x7F}, 127U, 1);
+ // Two byte value 128
+ TestLEB128Decode(std::array<uint8_t, 2>{0x80, 0x01}, 128U, 2);
+ // Two byte value 300
+ TestLEB128Decode(std::array<uint8_t, 2>{0xAC, 0x02}, 300U, 2);
+ // Three byte value 16384
+ TestLEB128Decode(std::array<uint8_t, 3>{0x80, 0x80, 0x01}, 16384U, 3);
+ // Three byte value 16384, with remaining data
+ TestLEB128Decode(std::array<uint8_t, 5>{0x80, 0x80, 0x01, 0x80, 0x00}, 16384U, 3);
+ // Four byte value 268435455
+ TestLEB128Decode(std::array<uint8_t, 4>{0xFF, 0xFF, 0xFF, 0x7F}, 268435455U, 4);
+ // Five byte uint32_t max value
+ TestLEB128Decode(std::array<uint8_t, 5>{0xFF, 0xFF, 0xFF, 0xFF, 0x0F}, 4294967295U, 5);
+ // uint64_t value requiring 10 bytes
+ TestLEB128Decode(
+ std::array<uint8_t, 10>{0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x01},
+ 18446744073709551615ULL, 10);
+ // int32_t with maximum size (31 bits of 1)
+ TestLEB128Decode(std::array<uint8_t, 5>{0xFF, 0xFF, 0xFF, 0xFF, 0x7},
+ std::numeric_limits<int32_t>::max(), 5);
+
+ // Error case: Truncated sequence (continuation bit set but no more data)
+ TestLEB128Decode(std::array<uint8_t, 1>{0x80}, 0U, 0);
+ // Error case: Input over the maximum number of bytes for a int32_t (5), but the
+ // overflow none the less (7 * 5 = 35 bits of data).
+ TestLEB128Decode(std::array<uint8_t, 5>{0xFF, 0xFF, 0xFF, 0xFF, 0x7F}, int32_t{}, 0);
+ // Error case: Oversized sequence for uint32_t (too many bytes)
+ TestLEB128Decode(std::array<uint8_t, 6>{0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x01}, 0U, 0);
+}
+
static void TestZigZag(int32_t v, std::array<uint8_t, 5> buffer_expect) {
- uint8_t buffer[bit_util::BitReader::kMaxVlqByteLength] = {};
+ uint8_t buffer[bit_util::BitReader::kMaxVlqByteLengthForInt32] = {};
bit_util::BitWriter writer(buffer, sizeof(buffer));
- bit_util::BitReader reader(buffer, sizeof(buffer));
writer.PutZigZagVlqInt(v);
+ // WARN reader buffer input on creation so it must be created after the data is written
Review Comment:
I'm not sure what this comment means, do you want to elaborate?
##########
cpp/src/arrow/util/bit_stream_utils_internal.h:
##########
@@ -483,13 +496,17 @@ inline bool BitReader::GetZigZagVlqInt(int32_t* v) {
}
inline bool BitWriter::PutVlqInt(uint64_t v) {
Review Comment:
Conversely why change the 64-bit `PutVlqInt` but not the 64-bit one?
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -84,32 +85,278 @@ namespace util {
/// (total 26 bytes, 1 byte overhead)
//
+class RleRun {
+ public:
+ using byte = uint8_t;
+ /// Enough space to store a 64bit value
+ using raw_data_storage = std::array<byte, 8>;
+ using raw_data_const_pointer = const byte*;
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr RleRun() noexcept = default;
Review Comment:
All this feels a bit pointlessly complicated. This could be a simple struct and let C++ handle construction/zero-init/etc. automatically:
```c++
struct RleRun {
/// The repeated value, as raw bytes (max 64 bits)
std::array<uint8_t, 8> repeated_value;
/// The number of time the value is repeated
int32_t repeat_count;
/// The size in bits of a packed value in the run
int32_t value_bit_width;
};
```
It has the added benefit of making this header file easier to read :)
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -84,32 +85,278 @@ namespace util {
/// (total 26 bytes, 1 byte overhead)
//
+class RleRun {
+ public:
+ using byte = uint8_t;
+ /// Enough space to store a 64bit value
+ using raw_data_storage = std::array<byte, 8>;
+ using raw_data_const_pointer = const byte*;
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr RleRun() noexcept = default;
+ constexpr RleRun(RleRun const&) noexcept = default;
+ constexpr RleRun(RleRun&&) noexcept = default;
+
+ explicit RleRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr RleRun& operator=(RleRun const&) noexcept = default;
+ constexpr RleRun& operator=(RleRun&&) noexcept = default;
+
+ /// The number of repeated values in this run.
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ /// A pointer to the repeated value raw bytes.
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ /// The number of bytes used for the raw repeated value.
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The repeated value raw bytes stored inside the class
+ raw_data_storage data_ = {};
+ /// The number of time the value is repeated
+ values_count_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+class BitPackedRun {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// According to the Parquet thrift definition the page size can be written into an
+ /// int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr BitPackedRun() noexcept = default;
+ constexpr BitPackedRun(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun(BitPackedRun&&) noexcept = default;
+
+ constexpr BitPackedRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr BitPackedRun& operator=(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun& operator=(BitPackedRun&&) noexcept = default;
+
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Number of values in this run.
+ raw_data_size_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+/// A parser that emits either a ``BitPackedRun`` or a ``RleRun``.
+class RleBitPackedParser {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// By Parquet thrift definition the page size can be written into an int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+ /// The different types of runs emitted by the parser
+ using dynamic_run_type = std::variant<RleRun, BitPackedRun>;
+
+ constexpr RleBitPackedParser() noexcept = default;
+
+ constexpr RleBitPackedParser(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr void Reset(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width_) noexcept;
+
+ /// Get the current run with a small parsing cost without advancing the iteration.
+ [[nodiscard]] std::optional<dynamic_run_type> Peek() const;
Review Comment:
Having a third variant alternative `struct RleEmpty {}` might be more efficient than an optional variant? Though this should be benchmarked :)
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -84,32 +85,278 @@ namespace util {
/// (total 26 bytes, 1 byte overhead)
//
+class RleRun {
+ public:
+ using byte = uint8_t;
+ /// Enough space to store a 64bit value
+ using raw_data_storage = std::array<byte, 8>;
+ using raw_data_const_pointer = const byte*;
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr RleRun() noexcept = default;
+ constexpr RleRun(RleRun const&) noexcept = default;
+ constexpr RleRun(RleRun&&) noexcept = default;
+
+ explicit RleRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr RleRun& operator=(RleRun const&) noexcept = default;
+ constexpr RleRun& operator=(RleRun&&) noexcept = default;
+
+ /// The number of repeated values in this run.
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ /// A pointer to the repeated value raw bytes.
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ /// The number of bytes used for the raw repeated value.
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The repeated value raw bytes stored inside the class
+ raw_data_storage data_ = {};
+ /// The number of time the value is repeated
+ values_count_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+class BitPackedRun {
Review Comment:
Same comment on the principle as on `RleRun` above. This can also be a simple struct with 3 member variables...
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -84,32 +85,278 @@ namespace util {
/// (total 26 bytes, 1 byte overhead)
//
+class RleRun {
+ public:
+ using byte = uint8_t;
+ /// Enough space to store a 64bit value
+ using raw_data_storage = std::array<byte, 8>;
+ using raw_data_const_pointer = const byte*;
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr RleRun() noexcept = default;
+ constexpr RleRun(RleRun const&) noexcept = default;
+ constexpr RleRun(RleRun&&) noexcept = default;
+
+ explicit RleRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr RleRun& operator=(RleRun const&) noexcept = default;
+ constexpr RleRun& operator=(RleRun&&) noexcept = default;
+
+ /// The number of repeated values in this run.
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ /// A pointer to the repeated value raw bytes.
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ /// The number of bytes used for the raw repeated value.
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The repeated value raw bytes stored inside the class
+ raw_data_storage data_ = {};
+ /// The number of time the value is repeated
+ values_count_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+class BitPackedRun {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// According to the Parquet thrift definition the page size can be written into an
+ /// int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr BitPackedRun() noexcept = default;
+ constexpr BitPackedRun(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun(BitPackedRun&&) noexcept = default;
+
+ constexpr BitPackedRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr BitPackedRun& operator=(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun& operator=(BitPackedRun&&) noexcept = default;
+
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Number of values in this run.
+ raw_data_size_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+/// A parser that emits either a ``BitPackedRun`` or a ``RleRun``.
+class RleBitPackedParser {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// By Parquet thrift definition the page size can be written into an int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+ /// The different types of runs emitted by the parser
+ using dynamic_run_type = std::variant<RleRun, BitPackedRun>;
+
+ constexpr RleBitPackedParser() noexcept = default;
+
+ constexpr RleBitPackedParser(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr void Reset(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width_) noexcept;
+
+ /// Get the current run with a small parsing cost without advancing the iteration.
+ [[nodiscard]] std::optional<dynamic_run_type> Peek() const;
+
+ /// Move to the next run.
+ [[nodiscard]] bool Advance();
+
+ /// Advance and return the current run.
+ [[nodiscard]] std::optional<dynamic_run_type> Next();
+
+ /// Whether there is still runs to iterate over.
+ ///
+ /// WARN: Due to lack of proper error handling, iteration with Next and Peek could
+ /// return not data while the parser is not exhausted.
Review Comment:
```suggestion
/// WARN: Due to simplistic error handling, iteration with Next and Peek could
/// fail to return not data while the parser is not exhausted.
```
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -84,32 +85,278 @@ namespace util {
/// (total 26 bytes, 1 byte overhead)
//
+class RleRun {
+ public:
+ using byte = uint8_t;
+ /// Enough space to store a 64bit value
+ using raw_data_storage = std::array<byte, 8>;
+ using raw_data_const_pointer = const byte*;
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr RleRun() noexcept = default;
+ constexpr RleRun(RleRun const&) noexcept = default;
+ constexpr RleRun(RleRun&&) noexcept = default;
+
+ explicit RleRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr RleRun& operator=(RleRun const&) noexcept = default;
+ constexpr RleRun& operator=(RleRun&&) noexcept = default;
+
+ /// The number of repeated values in this run.
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ /// A pointer to the repeated value raw bytes.
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ /// The number of bytes used for the raw repeated value.
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The repeated value raw bytes stored inside the class
+ raw_data_storage data_ = {};
+ /// The number of time the value is repeated
+ values_count_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+class BitPackedRun {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// According to the Parquet thrift definition the page size can be written into an
+ /// int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr BitPackedRun() noexcept = default;
+ constexpr BitPackedRun(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun(BitPackedRun&&) noexcept = default;
+
+ constexpr BitPackedRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr BitPackedRun& operator=(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun& operator=(BitPackedRun&&) noexcept = default;
+
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Number of values in this run.
+ raw_data_size_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+/// A parser that emits either a ``BitPackedRun`` or a ``RleRun``.
+class RleBitPackedParser {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// By Parquet thrift definition the page size can be written into an int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+ /// The different types of runs emitted by the parser
+ using dynamic_run_type = std::variant<RleRun, BitPackedRun>;
+
+ constexpr RleBitPackedParser() noexcept = default;
+
+ constexpr RleBitPackedParser(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr void Reset(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width_) noexcept;
+
+ /// Get the current run with a small parsing cost without advancing the iteration.
+ [[nodiscard]] std::optional<dynamic_run_type> Peek() const;
+
+ /// Move to the next run.
+ [[nodiscard]] bool Advance();
+
+ /// Advance and return the current run.
+ [[nodiscard]] std::optional<dynamic_run_type> Next();
Review Comment:
This is optional to signal EOF, right?
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -84,32 +85,278 @@ namespace util {
/// (total 26 bytes, 1 byte overhead)
//
+class RleRun {
+ public:
+ using byte = uint8_t;
+ /// Enough space to store a 64bit value
+ using raw_data_storage = std::array<byte, 8>;
+ using raw_data_const_pointer = const byte*;
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr RleRun() noexcept = default;
+ constexpr RleRun(RleRun const&) noexcept = default;
+ constexpr RleRun(RleRun&&) noexcept = default;
+
+ explicit RleRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr RleRun& operator=(RleRun const&) noexcept = default;
+ constexpr RleRun& operator=(RleRun&&) noexcept = default;
+
+ /// The number of repeated values in this run.
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ /// A pointer to the repeated value raw bytes.
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ /// The number of bytes used for the raw repeated value.
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The repeated value raw bytes stored inside the class
+ raw_data_storage data_ = {};
+ /// The number of time the value is repeated
+ values_count_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+class BitPackedRun {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// According to the Parquet thrift definition the page size can be written into an
+ /// int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr BitPackedRun() noexcept = default;
+ constexpr BitPackedRun(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun(BitPackedRun&&) noexcept = default;
+
+ constexpr BitPackedRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr BitPackedRun& operator=(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun& operator=(BitPackedRun&&) noexcept = default;
+
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Number of values in this run.
+ raw_data_size_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+/// A parser that emits either a ``BitPackedRun`` or a ``RleRun``.
+class RleBitPackedParser {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// By Parquet thrift definition the page size can be written into an int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+ /// The different types of runs emitted by the parser
+ using dynamic_run_type = std::variant<RleRun, BitPackedRun>;
+
+ constexpr RleBitPackedParser() noexcept = default;
+
+ constexpr RleBitPackedParser(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr void Reset(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width_) noexcept;
+
+ /// Get the current run with a small parsing cost without advancing the iteration.
+ [[nodiscard]] std::optional<dynamic_run_type> Peek() const;
+
+ /// Move to the next run.
+ [[nodiscard]] bool Advance();
+
+ /// Advance and return the current run.
+ [[nodiscard]] std::optional<dynamic_run_type> Next();
+
+ /// Whether there is still runs to iterate over.
+ ///
+ /// WARN: Due to lack of proper error handling, iteration with Next and Peek could
+ /// return not data while the parser is not exhausted.
+ /// This is how one can check for errors.
+ [[nodiscard]] bool Exhausted() const;
+
+ /// Enum to return from an ``Parse`` handler.
+ ///
+ /// Since a callback has no way to know when to stop, the handler must return
+ /// a value indicating to the ``Parse`` function whether to stop or continue.
+ enum class ControlFlow {
+ Continue,
+ Break,
+ };
+
+ /// A callback approach to parsing.
+ ///
+ /// This approach is used to reduce the number of dynamic lookups involved with using a
+ /// variant.
+ ///
+ /// The handler must be of the form
+ /// ```cpp`
+ /// struct Handler {
+ /// ControlFlow OnBitPackedRun(BitPackedRun run);
+ ///
+ /// ControlFlow OnRleRun(RleRun run);
+ /// };
+ /// ```
+ template <typename Handler>
+ void Parse(Handler&& handler);
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Size in bytes of the run.
+ raw_data_size_type data_size_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+
+ /// Run the handler on the run read and return the number of values read.
+ /// Does not advance the parser.
+ template <typename Handler>
+ std::pair<raw_data_size_type, ControlFlow> PeekImpl(Handler&&) const;
+};
+
/// Decoder class for RLE encoded data.
+template <typename T>
class RleDecoder {
public:
- /// Create a decoder object. buffer/buffer_len is the decoded data.
- /// bit_width is the width of each value (before encoding).
- RleDecoder(const uint8_t* buffer, int buffer_len, int bit_width)
- : bit_reader_(buffer, buffer_len),
- bit_width_(bit_width),
- current_value_(0),
- repeat_count_(0),
- literal_count_(0) {
- ARROW_DCHECK_GE(bit_width_, 0);
- ARROW_DCHECK_LE(bit_width_, 64);
- }
+ /// The type in which the data should be decoded.
+ using value_type = T;
+ /// The type of run that can be decoded.
+ using run_type = RleRun;
+ using values_count_type = run_type::values_count_type;
- RleDecoder() : bit_width_(-1) {}
+ constexpr RleDecoder() noexcept = default;
- void Reset(const uint8_t* buffer, int buffer_len, int bit_width) {
- ARROW_DCHECK_GE(bit_width, 0);
- ARROW_DCHECK_LE(bit_width, 64);
- bit_reader_.Reset(buffer, buffer_len);
- bit_width_ = bit_width;
- current_value_ = 0;
- repeat_count_ = 0;
- literal_count_ = 0;
- }
+ explicit RleDecoder(run_type const& run) noexcept;
+
+ void Reset(run_type const& run) noexcept;
+
+ /// Return the number of values that can be advanced.
+ [[nodiscard]] values_count_type Remaining() const;
Review Comment:
The `[[nodiscard]]` annotations are bit distracting and pedantic, IMHO.
For now, we only use `[[nodiscard]]` in cases where it is dangerous to ignore a return value (such as an error).
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -84,32 +85,278 @@ namespace util {
/// (total 26 bytes, 1 byte overhead)
//
+class RleRun {
+ public:
+ using byte = uint8_t;
+ /// Enough space to store a 64bit value
+ using raw_data_storage = std::array<byte, 8>;
+ using raw_data_const_pointer = const byte*;
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr RleRun() noexcept = default;
+ constexpr RleRun(RleRun const&) noexcept = default;
+ constexpr RleRun(RleRun&&) noexcept = default;
+
+ explicit RleRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr RleRun& operator=(RleRun const&) noexcept = default;
+ constexpr RleRun& operator=(RleRun&&) noexcept = default;
+
+ /// The number of repeated values in this run.
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ /// A pointer to the repeated value raw bytes.
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ /// The number of bytes used for the raw repeated value.
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The repeated value raw bytes stored inside the class
+ raw_data_storage data_ = {};
+ /// The number of time the value is repeated
+ values_count_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+class BitPackedRun {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// According to the Parquet thrift definition the page size can be written into an
+ /// int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr BitPackedRun() noexcept = default;
+ constexpr BitPackedRun(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun(BitPackedRun&&) noexcept = default;
+
+ constexpr BitPackedRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr BitPackedRun& operator=(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun& operator=(BitPackedRun&&) noexcept = default;
+
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Number of values in this run.
+ raw_data_size_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+/// A parser that emits either a ``BitPackedRun`` or a ``RleRun``.
+class RleBitPackedParser {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// By Parquet thrift definition the page size can be written into an int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+ /// The different types of runs emitted by the parser
+ using dynamic_run_type = std::variant<RleRun, BitPackedRun>;
+
+ constexpr RleBitPackedParser() noexcept = default;
+
+ constexpr RleBitPackedParser(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr void Reset(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width_) noexcept;
+
+ /// Get the current run with a small parsing cost without advancing the iteration.
+ [[nodiscard]] std::optional<dynamic_run_type> Peek() const;
+
+ /// Move to the next run.
+ [[nodiscard]] bool Advance();
+
+ /// Advance and return the current run.
+ [[nodiscard]] std::optional<dynamic_run_type> Next();
+
+ /// Whether there is still runs to iterate over.
+ ///
+ /// WARN: Due to lack of proper error handling, iteration with Next and Peek could
+ /// return not data while the parser is not exhausted.
+ /// This is how one can check for errors.
+ [[nodiscard]] bool Exhausted() const;
+
+ /// Enum to return from an ``Parse`` handler.
+ ///
+ /// Since a callback has no way to know when to stop, the handler must return
+ /// a value indicating to the ``Parse`` function whether to stop or continue.
+ enum class ControlFlow {
+ Continue,
+ Break,
+ };
+
+ /// A callback approach to parsing.
+ ///
+ /// This approach is used to reduce the number of dynamic lookups involved with using a
+ /// variant.
+ ///
+ /// The handler must be of the form
+ /// ```cpp`
+ /// struct Handler {
+ /// ControlFlow OnBitPackedRun(BitPackedRun run);
+ ///
+ /// ControlFlow OnRleRun(RleRun run);
+ /// };
+ /// ```
+ template <typename Handler>
+ void Parse(Handler&& handler);
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Size in bytes of the run.
+ raw_data_size_type data_size_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+
+ /// Run the handler on the run read and return the number of values read.
+ /// Does not advance the parser.
+ template <typename Handler>
+ std::pair<raw_data_size_type, ControlFlow> PeekImpl(Handler&&) const;
+};
+
/// Decoder class for RLE encoded data.
+template <typename T>
class RleDecoder {
public:
- /// Create a decoder object. buffer/buffer_len is the decoded data.
- /// bit_width is the width of each value (before encoding).
- RleDecoder(const uint8_t* buffer, int buffer_len, int bit_width)
- : bit_reader_(buffer, buffer_len),
- bit_width_(bit_width),
- current_value_(0),
- repeat_count_(0),
- literal_count_(0) {
- ARROW_DCHECK_GE(bit_width_, 0);
- ARROW_DCHECK_LE(bit_width_, 64);
- }
+ /// The type in which the data should be decoded.
+ using value_type = T;
+ /// The type of run that can be decoded.
+ using run_type = RleRun;
+ using values_count_type = run_type::values_count_type;
Review Comment:
Again, these aliases only make the code more difficult to follow, IMHO. `RleRun` is perfectly clear.
##########
cpp/src/parquet/decoder.cc:
##########
@@ -1898,7 +1900,7 @@ class RleBooleanDecoder : public TypedDecoderImpl<BooleanType>, public BooleanDe
}
private:
- std::shared_ptr<::arrow::util::RleDecoder> decoder_;
+ std::shared_ptr<::arrow::util::RleBitPackedDecoder<bool>> decoder_;
Review Comment:
Minor improvement suggestion: I think `std::shared_ptr` is only used here for delayed initialization, so it can be replaced with `std::optional`.
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -84,32 +85,278 @@ namespace util {
/// (total 26 bytes, 1 byte overhead)
//
+class RleRun {
+ public:
+ using byte = uint8_t;
+ /// Enough space to store a 64bit value
+ using raw_data_storage = std::array<byte, 8>;
+ using raw_data_const_pointer = const byte*;
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr RleRun() noexcept = default;
+ constexpr RleRun(RleRun const&) noexcept = default;
+ constexpr RleRun(RleRun&&) noexcept = default;
+
+ explicit RleRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr RleRun& operator=(RleRun const&) noexcept = default;
+ constexpr RleRun& operator=(RleRun&&) noexcept = default;
+
+ /// The number of repeated values in this run.
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ /// A pointer to the repeated value raw bytes.
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ /// The number of bytes used for the raw repeated value.
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The repeated value raw bytes stored inside the class
+ raw_data_storage data_ = {};
+ /// The number of time the value is repeated
+ values_count_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+class BitPackedRun {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// According to the Parquet thrift definition the page size can be written into an
+ /// int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr BitPackedRun() noexcept = default;
+ constexpr BitPackedRun(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun(BitPackedRun&&) noexcept = default;
+
+ constexpr BitPackedRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr BitPackedRun& operator=(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun& operator=(BitPackedRun&&) noexcept = default;
+
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Number of values in this run.
+ raw_data_size_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+/// A parser that emits either a ``BitPackedRun`` or a ``RleRun``.
+class RleBitPackedParser {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// By Parquet thrift definition the page size can be written into an int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+ /// The different types of runs emitted by the parser
+ using dynamic_run_type = std::variant<RleRun, BitPackedRun>;
+
+ constexpr RleBitPackedParser() noexcept = default;
+
+ constexpr RleBitPackedParser(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr void Reset(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width_) noexcept;
+
+ /// Get the current run with a small parsing cost without advancing the iteration.
+ [[nodiscard]] std::optional<dynamic_run_type> Peek() const;
+
+ /// Move to the next run.
+ [[nodiscard]] bool Advance();
+
+ /// Advance and return the current run.
+ [[nodiscard]] std::optional<dynamic_run_type> Next();
+
+ /// Whether there is still runs to iterate over.
+ ///
+ /// WARN: Due to lack of proper error handling, iteration with Next and Peek could
+ /// return not data while the parser is not exhausted.
+ /// This is how one can check for errors.
+ [[nodiscard]] bool Exhausted() const;
+
+ /// Enum to return from an ``Parse`` handler.
+ ///
+ /// Since a callback has no way to know when to stop, the handler must return
+ /// a value indicating to the ``Parse`` function whether to stop or continue.
+ enum class ControlFlow {
+ Continue,
+ Break,
+ };
+
+ /// A callback approach to parsing.
+ ///
+ /// This approach is used to reduce the number of dynamic lookups involved with using a
+ /// variant.
+ ///
+ /// The handler must be of the form
+ /// ```cpp`
+ /// struct Handler {
+ /// ControlFlow OnBitPackedRun(BitPackedRun run);
+ ///
+ /// ControlFlow OnRleRun(RleRun run);
+ /// };
+ /// ```
+ template <typename Handler>
+ void Parse(Handler&& handler);
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Size in bytes of the run.
+ raw_data_size_type data_size_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+
+ /// Run the handler on the run read and return the number of values read.
+ /// Does not advance the parser.
+ template <typename Handler>
+ std::pair<raw_data_size_type, ControlFlow> PeekImpl(Handler&&) const;
+};
+
/// Decoder class for RLE encoded data.
+template <typename T>
class RleDecoder {
public:
- /// Create a decoder object. buffer/buffer_len is the decoded data.
- /// bit_width is the width of each value (before encoding).
- RleDecoder(const uint8_t* buffer, int buffer_len, int bit_width)
- : bit_reader_(buffer, buffer_len),
- bit_width_(bit_width),
- current_value_(0),
- repeat_count_(0),
- literal_count_(0) {
- ARROW_DCHECK_GE(bit_width_, 0);
- ARROW_DCHECK_LE(bit_width_, 64);
- }
+ /// The type in which the data should be decoded.
+ using value_type = T;
+ /// The type of run that can be decoded.
+ using run_type = RleRun;
+ using values_count_type = run_type::values_count_type;
- RleDecoder() : bit_width_(-1) {}
+ constexpr RleDecoder() noexcept = default;
- void Reset(const uint8_t* buffer, int buffer_len, int bit_width) {
- ARROW_DCHECK_GE(bit_width, 0);
- ARROW_DCHECK_LE(bit_width, 64);
- bit_reader_.Reset(buffer, buffer_len);
- bit_width_ = bit_width;
- current_value_ = 0;
- repeat_count_ = 0;
- literal_count_ = 0;
- }
+ explicit RleDecoder(run_type const& run) noexcept;
+
+ void Reset(run_type const& run) noexcept;
+
+ /// Return the number of values that can be advanced.
+ [[nodiscard]] values_count_type Remaining() const;
+
+ /// Return the repeated value of this decoder.
+ [[nodiscard]] constexpr value_type Value() const;
+
+ /// Try to advance by as many values as provided.
+ /// Return the number of values skipped.
+ [[nodiscard]] values_count_type Advance(values_count_type batch_size);
+
+ /// Get the next value and return false if there are no more.
+ [[nodiscard]] constexpr bool Get(value_type* out_value);
+
+ /// Get a batch of values return the number of decoded elements.
+ [[nodiscard]] values_count_type GetBatch(value_type* out, values_count_type batch_size);
+
+ private:
+ value_type value_ = {};
+ values_count_type remaining_count_ = 0;
+
+ static_assert(std::is_integral_v<value_type>,
+ "This class makes assumptions about integer endianness and padding");
Review Comment:
The assertion message (endianness and padding) does not match the assertion itself.
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
Review Comment:
Why call `operator->()` ??? You should simply be able to write `std::get_if<BitPackedRun>(&dyn_run)`...
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -84,32 +85,278 @@ namespace util {
/// (total 26 bytes, 1 byte overhead)
//
+class RleRun {
+ public:
+ using byte = uint8_t;
+ /// Enough space to store a 64bit value
+ using raw_data_storage = std::array<byte, 8>;
+ using raw_data_const_pointer = const byte*;
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr RleRun() noexcept = default;
+ constexpr RleRun(RleRun const&) noexcept = default;
+ constexpr RleRun(RleRun&&) noexcept = default;
+
+ explicit RleRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr RleRun& operator=(RleRun const&) noexcept = default;
+ constexpr RleRun& operator=(RleRun&&) noexcept = default;
+
+ /// The number of repeated values in this run.
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ /// A pointer to the repeated value raw bytes.
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ /// The number of bytes used for the raw repeated value.
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The repeated value raw bytes stored inside the class
+ raw_data_storage data_ = {};
+ /// The number of time the value is repeated
+ values_count_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+class BitPackedRun {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// According to the Parquet thrift definition the page size can be written into an
+ /// int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr BitPackedRun() noexcept = default;
+ constexpr BitPackedRun(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun(BitPackedRun&&) noexcept = default;
+
+ constexpr BitPackedRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr BitPackedRun& operator=(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun& operator=(BitPackedRun&&) noexcept = default;
+
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Number of values in this run.
+ raw_data_size_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+/// A parser that emits either a ``BitPackedRun`` or a ``RleRun``.
+class RleBitPackedParser {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// By Parquet thrift definition the page size can be written into an int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+ /// The different types of runs emitted by the parser
+ using dynamic_run_type = std::variant<RleRun, BitPackedRun>;
+
+ constexpr RleBitPackedParser() noexcept = default;
+
+ constexpr RleBitPackedParser(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr void Reset(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width_) noexcept;
+
+ /// Get the current run with a small parsing cost without advancing the iteration.
+ [[nodiscard]] std::optional<dynamic_run_type> Peek() const;
+
+ /// Move to the next run.
+ [[nodiscard]] bool Advance();
+
+ /// Advance and return the current run.
+ [[nodiscard]] std::optional<dynamic_run_type> Next();
+
+ /// Whether there is still runs to iterate over.
+ ///
+ /// WARN: Due to lack of proper error handling, iteration with Next and Peek could
+ /// return not data while the parser is not exhausted.
+ /// This is how one can check for errors.
+ [[nodiscard]] bool Exhausted() const;
+
+ /// Enum to return from an ``Parse`` handler.
+ ///
+ /// Since a callback has no way to know when to stop, the handler must return
+ /// a value indicating to the ``Parse`` function whether to stop or continue.
+ enum class ControlFlow {
+ Continue,
+ Break,
+ };
+
+ /// A callback approach to parsing.
+ ///
+ /// This approach is used to reduce the number of dynamic lookups involved with using a
+ /// variant.
+ ///
+ /// The handler must be of the form
+ /// ```cpp`
+ /// struct Handler {
+ /// ControlFlow OnBitPackedRun(BitPackedRun run);
+ ///
+ /// ControlFlow OnRleRun(RleRun run);
+ /// };
+ /// ```
+ template <typename Handler>
+ void Parse(Handler&& handler);
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Size in bytes of the run.
+ raw_data_size_type data_size_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+
+ /// Run the handler on the run read and return the number of values read.
+ /// Does not advance the parser.
+ template <typename Handler>
+ std::pair<raw_data_size_type, ControlFlow> PeekImpl(Handler&&) const;
+};
+
/// Decoder class for RLE encoded data.
+template <typename T>
class RleDecoder {
public:
- /// Create a decoder object. buffer/buffer_len is the decoded data.
- /// bit_width is the width of each value (before encoding).
- RleDecoder(const uint8_t* buffer, int buffer_len, int bit_width)
- : bit_reader_(buffer, buffer_len),
- bit_width_(bit_width),
- current_value_(0),
- repeat_count_(0),
- literal_count_(0) {
- ARROW_DCHECK_GE(bit_width_, 0);
- ARROW_DCHECK_LE(bit_width_, 64);
- }
+ /// The type in which the data should be decoded.
+ using value_type = T;
+ /// The type of run that can be decoded.
+ using run_type = RleRun;
+ using values_count_type = run_type::values_count_type;
- RleDecoder() : bit_width_(-1) {}
+ constexpr RleDecoder() noexcept = default;
- void Reset(const uint8_t* buffer, int buffer_len, int bit_width) {
- ARROW_DCHECK_GE(bit_width, 0);
- ARROW_DCHECK_LE(bit_width, 64);
- bit_reader_.Reset(buffer, buffer_len);
- bit_width_ = bit_width;
- current_value_ = 0;
- repeat_count_ = 0;
- literal_count_ = 0;
- }
+ explicit RleDecoder(run_type const& run) noexcept;
+
+ void Reset(run_type const& run) noexcept;
+
+ /// Return the number of values that can be advanced.
+ [[nodiscard]] values_count_type Remaining() const;
+
+ /// Return the repeated value of this decoder.
+ [[nodiscard]] constexpr value_type Value() const;
+
+ /// Try to advance by as many values as provided.
+ /// Return the number of values skipped.
+ [[nodiscard]] values_count_type Advance(values_count_type batch_size);
+
+ /// Get the next value and return false if there are no more.
+ [[nodiscard]] constexpr bool Get(value_type* out_value);
+
+ /// Get a batch of values return the number of decoded elements.
+ [[nodiscard]] values_count_type GetBatch(value_type* out, values_count_type batch_size);
+
+ private:
+ value_type value_ = {};
+ values_count_type remaining_count_ = 0;
+
+ static_assert(std::is_integral_v<value_type>,
+ "This class makes assumptions about integer endianness and padding");
+};
+
+/// Decoder class for Bit packing encoded data.
Review Comment:
```suggestion
/// Decoder class for Bit-packed data.
```
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -84,32 +85,278 @@ namespace util {
/// (total 26 bytes, 1 byte overhead)
//
+class RleRun {
+ public:
+ using byte = uint8_t;
+ /// Enough space to store a 64bit value
+ using raw_data_storage = std::array<byte, 8>;
+ using raw_data_const_pointer = const byte*;
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr RleRun() noexcept = default;
+ constexpr RleRun(RleRun const&) noexcept = default;
+ constexpr RleRun(RleRun&&) noexcept = default;
+
+ explicit RleRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr RleRun& operator=(RleRun const&) noexcept = default;
+ constexpr RleRun& operator=(RleRun&&) noexcept = default;
+
+ /// The number of repeated values in this run.
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ /// A pointer to the repeated value raw bytes.
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ /// The number of bytes used for the raw repeated value.
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The repeated value raw bytes stored inside the class
+ raw_data_storage data_ = {};
+ /// The number of time the value is repeated
+ values_count_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+class BitPackedRun {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// According to the Parquet thrift definition the page size can be written into an
+ /// int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr BitPackedRun() noexcept = default;
+ constexpr BitPackedRun(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun(BitPackedRun&&) noexcept = default;
+
+ constexpr BitPackedRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr BitPackedRun& operator=(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun& operator=(BitPackedRun&&) noexcept = default;
+
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Number of values in this run.
+ raw_data_size_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+/// A parser that emits either a ``BitPackedRun`` or a ``RleRun``.
+class RleBitPackedParser {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// By Parquet thrift definition the page size can be written into an int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+ /// The different types of runs emitted by the parser
+ using dynamic_run_type = std::variant<RleRun, BitPackedRun>;
+
+ constexpr RleBitPackedParser() noexcept = default;
+
+ constexpr RleBitPackedParser(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr void Reset(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width_) noexcept;
+
+ /// Get the current run with a small parsing cost without advancing the iteration.
+ [[nodiscard]] std::optional<dynamic_run_type> Peek() const;
+
+ /// Move to the next run.
+ [[nodiscard]] bool Advance();
+
+ /// Advance and return the current run.
+ [[nodiscard]] std::optional<dynamic_run_type> Next();
+
+ /// Whether there is still runs to iterate over.
+ ///
+ /// WARN: Due to lack of proper error handling, iteration with Next and Peek could
+ /// return not data while the parser is not exhausted.
+ /// This is how one can check for errors.
+ [[nodiscard]] bool Exhausted() const;
+
+ /// Enum to return from an ``Parse`` handler.
+ ///
+ /// Since a callback has no way to know when to stop, the handler must return
+ /// a value indicating to the ``Parse`` function whether to stop or continue.
+ enum class ControlFlow {
+ Continue,
+ Break,
+ };
+
+ /// A callback approach to parsing.
+ ///
+ /// This approach is used to reduce the number of dynamic lookups involved with using a
+ /// variant.
+ ///
+ /// The handler must be of the form
+ /// ```cpp`
+ /// struct Handler {
+ /// ControlFlow OnBitPackedRun(BitPackedRun run);
+ ///
+ /// ControlFlow OnRleRun(RleRun run);
+ /// };
+ /// ```
+ template <typename Handler>
+ void Parse(Handler&& handler);
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Size in bytes of the run.
+ raw_data_size_type data_size_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+
+ /// Run the handler on the run read and return the number of values read.
+ /// Does not advance the parser.
+ template <typename Handler>
+ std::pair<raw_data_size_type, ControlFlow> PeekImpl(Handler&&) const;
+};
+
/// Decoder class for RLE encoded data.
+template <typename T>
class RleDecoder {
public:
- /// Create a decoder object. buffer/buffer_len is the decoded data.
- /// bit_width is the width of each value (before encoding).
- RleDecoder(const uint8_t* buffer, int buffer_len, int bit_width)
- : bit_reader_(buffer, buffer_len),
- bit_width_(bit_width),
- current_value_(0),
- repeat_count_(0),
- literal_count_(0) {
- ARROW_DCHECK_GE(bit_width_, 0);
- ARROW_DCHECK_LE(bit_width_, 64);
- }
+ /// The type in which the data should be decoded.
+ using value_type = T;
+ /// The type of run that can be decoded.
+ using run_type = RleRun;
+ using values_count_type = run_type::values_count_type;
- RleDecoder() : bit_width_(-1) {}
+ constexpr RleDecoder() noexcept = default;
- void Reset(const uint8_t* buffer, int buffer_len, int bit_width) {
- ARROW_DCHECK_GE(bit_width, 0);
- ARROW_DCHECK_LE(bit_width, 64);
- bit_reader_.Reset(buffer, buffer_len);
- bit_width_ = bit_width;
- current_value_ = 0;
- repeat_count_ = 0;
- literal_count_ = 0;
- }
+ explicit RleDecoder(run_type const& run) noexcept;
Review Comment:
Ok, does a `RleDecoder` decode a single RLE run? This should be explicit in the docstrings somewhere.
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -84,32 +85,278 @@ namespace util {
/// (total 26 bytes, 1 byte overhead)
//
+class RleRun {
+ public:
+ using byte = uint8_t;
+ /// Enough space to store a 64bit value
+ using raw_data_storage = std::array<byte, 8>;
+ using raw_data_const_pointer = const byte*;
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr RleRun() noexcept = default;
+ constexpr RleRun(RleRun const&) noexcept = default;
+ constexpr RleRun(RleRun&&) noexcept = default;
+
+ explicit RleRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr RleRun& operator=(RleRun const&) noexcept = default;
+ constexpr RleRun& operator=(RleRun&&) noexcept = default;
+
+ /// The number of repeated values in this run.
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ /// A pointer to the repeated value raw bytes.
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ /// The number of bytes used for the raw repeated value.
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The repeated value raw bytes stored inside the class
+ raw_data_storage data_ = {};
+ /// The number of time the value is repeated
+ values_count_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+class BitPackedRun {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// According to the Parquet thrift definition the page size can be written into an
+ /// int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr BitPackedRun() noexcept = default;
+ constexpr BitPackedRun(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun(BitPackedRun&&) noexcept = default;
+
+ constexpr BitPackedRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr BitPackedRun& operator=(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun& operator=(BitPackedRun&&) noexcept = default;
+
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Number of values in this run.
+ raw_data_size_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+/// A parser that emits either a ``BitPackedRun`` or a ``RleRun``.
+class RleBitPackedParser {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// By Parquet thrift definition the page size can be written into an int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+ /// The different types of runs emitted by the parser
+ using dynamic_run_type = std::variant<RleRun, BitPackedRun>;
+
+ constexpr RleBitPackedParser() noexcept = default;
+
+ constexpr RleBitPackedParser(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr void Reset(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width_) noexcept;
+
+ /// Get the current run with a small parsing cost without advancing the iteration.
+ [[nodiscard]] std::optional<dynamic_run_type> Peek() const;
+
+ /// Move to the next run.
+ [[nodiscard]] bool Advance();
+
+ /// Advance and return the current run.
+ [[nodiscard]] std::optional<dynamic_run_type> Next();
+
+ /// Whether there is still runs to iterate over.
+ ///
+ /// WARN: Due to lack of proper error handling, iteration with Next and Peek could
+ /// return not data while the parser is not exhausted.
+ /// This is how one can check for errors.
+ [[nodiscard]] bool Exhausted() const;
+
+ /// Enum to return from an ``Parse`` handler.
+ ///
+ /// Since a callback has no way to know when to stop, the handler must return
+ /// a value indicating to the ``Parse`` function whether to stop or continue.
+ enum class ControlFlow {
+ Continue,
+ Break,
+ };
+
+ /// A callback approach to parsing.
+ ///
+ /// This approach is used to reduce the number of dynamic lookups involved with using a
+ /// variant.
+ ///
+ /// The handler must be of the form
+ /// ```cpp`
+ /// struct Handler {
+ /// ControlFlow OnBitPackedRun(BitPackedRun run);
+ ///
+ /// ControlFlow OnRleRun(RleRun run);
+ /// };
+ /// ```
+ template <typename Handler>
+ void Parse(Handler&& handler);
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Size in bytes of the run.
+ raw_data_size_type data_size_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+
+ /// Run the handler on the run read and return the number of values read.
+ /// Does not advance the parser.
+ template <typename Handler>
+ std::pair<raw_data_size_type, ControlFlow> PeekImpl(Handler&&) const;
+};
+
/// Decoder class for RLE encoded data.
+template <typename T>
class RleDecoder {
public:
- /// Create a decoder object. buffer/buffer_len is the decoded data.
- /// bit_width is the width of each value (before encoding).
- RleDecoder(const uint8_t* buffer, int buffer_len, int bit_width)
- : bit_reader_(buffer, buffer_len),
- bit_width_(bit_width),
- current_value_(0),
- repeat_count_(0),
- literal_count_(0) {
- ARROW_DCHECK_GE(bit_width_, 0);
- ARROW_DCHECK_LE(bit_width_, 64);
- }
+ /// The type in which the data should be decoded.
+ using value_type = T;
+ /// The type of run that can be decoded.
+ using run_type = RleRun;
+ using values_count_type = run_type::values_count_type;
- RleDecoder() : bit_width_(-1) {}
+ constexpr RleDecoder() noexcept = default;
- void Reset(const uint8_t* buffer, int buffer_len, int bit_width) {
- ARROW_DCHECK_GE(bit_width, 0);
- ARROW_DCHECK_LE(bit_width, 64);
- bit_reader_.Reset(buffer, buffer_len);
- bit_width_ = bit_width;
- current_value_ = 0;
- repeat_count_ = 0;
- literal_count_ = 0;
- }
+ explicit RleDecoder(run_type const& run) noexcept;
+
+ void Reset(run_type const& run) noexcept;
+
+ /// Return the number of values that can be advanced.
+ [[nodiscard]] values_count_type Remaining() const;
+
+ /// Return the repeated value of this decoder.
+ [[nodiscard]] constexpr value_type Value() const;
+
+ /// Try to advance by as many values as provided.
+ /// Return the number of values skipped.
+ [[nodiscard]] values_count_type Advance(values_count_type batch_size);
+
+ /// Get the next value and return false if there are no more.
+ [[nodiscard]] constexpr bool Get(value_type* out_value);
+
+ /// Get a batch of values return the number of decoded elements.
+ [[nodiscard]] values_count_type GetBatch(value_type* out, values_count_type batch_size);
+
+ private:
+ value_type value_ = {};
+ values_count_type remaining_count_ = 0;
+
+ static_assert(std::is_integral_v<value_type>,
+ "This class makes assumptions about integer endianness and padding");
+};
+
+/// Decoder class for Bit packing encoded data.
+template <typename T>
+class BitPackedDecoder {
+ public:
+ /// The type in which the data should be decoded.
+ using value_type = T;
+ /// The type of run that can be decoded.
+ using run_type = BitPackedRun;
+ using values_count_type = run_type::values_count_type;
+ using bit_size_type = run_type::bit_size_type;
+
+ BitPackedDecoder() noexcept = default;
+
+ explicit BitPackedDecoder(run_type const& run) noexcept;
+
+ void Reset(run_type const& run) noexcept;
+
+ /// Return the number of values that can be advanced.
+ [[nodiscard]] constexpr values_count_type Remaining() const;
+
+ /// Return the size in bit in which each encoded value is written.
+ [[nodiscard]] constexpr bit_size_type ValueBitWidth() const;
+
+ /// Try to advance by as many values as provided.
+ /// Return the number of values skipped.
+ [[nodiscard]] values_count_type Advance(values_count_type batch_size);
+
+ /// Get the next value and return false if there are no more.
+ [[nodiscard]] bool Get(value_type* out_value);
+
+ /// Get a batch of values return the number of decoded elements.
+ [[nodiscard]] values_count_type GetBatch(value_type* out, values_count_type batch_size);
+
+ private:
+ ::arrow::bit_util::BitReader bit_reader_ = {};
+ bit_size_type value_bit_width_ = 0;
+ values_count_type remaining_count_ = 0;
+
+ static_assert(std::is_integral_v<value_type>,
+ "This class makes assumptions about integer endianness and padding");
+};
+
+/// Decoder class for RLE encoded data.
Review Comment:
```suggestion
/// Decoder class for Parquet RLE bit-packed data.
```
##########
cpp/src/arrow/util/bit_util.h:
##########
@@ -365,5 +365,118 @@ void PackBits(const uint32_t* values, uint8_t* out) {
}
}
+constexpr int64_t MaxLEB128ByteLen(int64_t n_bits) { return CeilDiv(n_bits, 7); }
+
+template <typename Int>
+constexpr int64_t MaxLEB128ByteLenFor = MaxLEB128ByteLen(sizeof(Int) * 8);
+
+/// Write a integer as LEB128
+///
+/// Write the input value as LEB128 into the outptu buffer and return the number of bytes
+/// written.
+/// If the output buffer size is insufficient, return 0 but the output may have been
+/// written to.
+///
+/// \see https://en.wikipedia.org/wiki/LEB128
+/// \see MaxLEB128ByteLenFor
+template <typename Int>
+constexpr int32_t WriteLEB128(Int value, uint8_t* out, int32_t max_out_size) {
+ constexpr Int kLow7Mask = Int(0x7F);
+ constexpr Int kHigh7Mask = ~kLow7Mask;
+ constexpr uint8_t kContinuationBit = 0x80;
+
+ auto const out_first = out;
+
+ // Write as many bytes as the could be for the given input
+ while ((value & kHigh7Mask) != Int(0)) {
+ // We do not have enough room to write the LEB128
+ if (out - out_first >= max_out_size) {
Review Comment:
Also, if this function is ever performance-critical, perhaps computing the required output size can be improved using `__builtin_clz` or similar? (this should be a followup issue at best)
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -84,32 +85,278 @@ namespace util {
/// (total 26 bytes, 1 byte overhead)
//
+class RleRun {
+ public:
+ using byte = uint8_t;
+ /// Enough space to store a 64bit value
+ using raw_data_storage = std::array<byte, 8>;
+ using raw_data_const_pointer = const byte*;
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr RleRun() noexcept = default;
+ constexpr RleRun(RleRun const&) noexcept = default;
+ constexpr RleRun(RleRun&&) noexcept = default;
+
+ explicit RleRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr RleRun& operator=(RleRun const&) noexcept = default;
+ constexpr RleRun& operator=(RleRun&&) noexcept = default;
+
+ /// The number of repeated values in this run.
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ /// A pointer to the repeated value raw bytes.
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ /// The number of bytes used for the raw repeated value.
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The repeated value raw bytes stored inside the class
+ raw_data_storage data_ = {};
+ /// The number of time the value is repeated
+ values_count_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+class BitPackedRun {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// According to the Parquet thrift definition the page size can be written into an
+ /// int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr BitPackedRun() noexcept = default;
+ constexpr BitPackedRun(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun(BitPackedRun&&) noexcept = default;
+
+ constexpr BitPackedRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr BitPackedRun& operator=(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun& operator=(BitPackedRun&&) noexcept = default;
+
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Number of values in this run.
+ raw_data_size_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+/// A parser that emits either a ``BitPackedRun`` or a ``RleRun``.
+class RleBitPackedParser {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// By Parquet thrift definition the page size can be written into an int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+ /// The different types of runs emitted by the parser
+ using dynamic_run_type = std::variant<RleRun, BitPackedRun>;
+
+ constexpr RleBitPackedParser() noexcept = default;
+
+ constexpr RleBitPackedParser(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr void Reset(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width_) noexcept;
+
+ /// Get the current run with a small parsing cost without advancing the iteration.
+ [[nodiscard]] std::optional<dynamic_run_type> Peek() const;
+
+ /// Move to the next run.
+ [[nodiscard]] bool Advance();
+
+ /// Advance and return the current run.
+ [[nodiscard]] std::optional<dynamic_run_type> Next();
+
+ /// Whether there is still runs to iterate over.
+ ///
+ /// WARN: Due to lack of proper error handling, iteration with Next and Peek could
+ /// return not data while the parser is not exhausted.
+ /// This is how one can check for errors.
+ [[nodiscard]] bool Exhausted() const;
+
+ /// Enum to return from an ``Parse`` handler.
+ ///
+ /// Since a callback has no way to know when to stop, the handler must return
+ /// a value indicating to the ``Parse`` function whether to stop or continue.
+ enum class ControlFlow {
+ Continue,
+ Break,
+ };
+
+ /// A callback approach to parsing.
+ ///
+ /// This approach is used to reduce the number of dynamic lookups involved with using a
+ /// variant.
+ ///
+ /// The handler must be of the form
+ /// ```cpp`
+ /// struct Handler {
+ /// ControlFlow OnBitPackedRun(BitPackedRun run);
+ ///
+ /// ControlFlow OnRleRun(RleRun run);
+ /// };
+ /// ```
+ template <typename Handler>
+ void Parse(Handler&& handler);
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Size in bytes of the run.
+ raw_data_size_type data_size_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+
+ /// Run the handler on the run read and return the number of values read.
+ /// Does not advance the parser.
+ template <typename Handler>
+ std::pair<raw_data_size_type, ControlFlow> PeekImpl(Handler&&) const;
+};
+
/// Decoder class for RLE encoded data.
+template <typename T>
class RleDecoder {
public:
- /// Create a decoder object. buffer/buffer_len is the decoded data.
- /// bit_width is the width of each value (before encoding).
- RleDecoder(const uint8_t* buffer, int buffer_len, int bit_width)
- : bit_reader_(buffer, buffer_len),
- bit_width_(bit_width),
- current_value_(0),
- repeat_count_(0),
- literal_count_(0) {
- ARROW_DCHECK_GE(bit_width_, 0);
- ARROW_DCHECK_LE(bit_width_, 64);
- }
+ /// The type in which the data should be decoded.
+ using value_type = T;
+ /// The type of run that can be decoded.
+ using run_type = RleRun;
+ using values_count_type = run_type::values_count_type;
- RleDecoder() : bit_width_(-1) {}
+ constexpr RleDecoder() noexcept = default;
- void Reset(const uint8_t* buffer, int buffer_len, int bit_width) {
- ARROW_DCHECK_GE(bit_width, 0);
- ARROW_DCHECK_LE(bit_width, 64);
- bit_reader_.Reset(buffer, buffer_len);
- bit_width_ = bit_width;
- current_value_ = 0;
- repeat_count_ = 0;
- literal_count_ = 0;
- }
+ explicit RleDecoder(run_type const& run) noexcept;
+
+ void Reset(run_type const& run) noexcept;
+
+ /// Return the number of values that can be advanced.
+ [[nodiscard]] values_count_type Remaining() const;
+
+ /// Return the repeated value of this decoder.
+ [[nodiscard]] constexpr value_type Value() const;
+
+ /// Try to advance by as many values as provided.
+ /// Return the number of values skipped.
+ [[nodiscard]] values_count_type Advance(values_count_type batch_size);
+
+ /// Get the next value and return false if there are no more.
+ [[nodiscard]] constexpr bool Get(value_type* out_value);
+
+ /// Get a batch of values return the number of decoded elements.
+ [[nodiscard]] values_count_type GetBatch(value_type* out, values_count_type batch_size);
+
+ private:
+ value_type value_ = {};
+ values_count_type remaining_count_ = 0;
+
+ static_assert(std::is_integral_v<value_type>,
+ "This class makes assumptions about integer endianness and padding");
+};
+
+/// Decoder class for Bit packing encoded data.
+template <typename T>
+class BitPackedDecoder {
+ public:
+ /// The type in which the data should be decoded.
+ using value_type = T;
+ /// The type of run that can be decoded.
+ using run_type = BitPackedRun;
+ using values_count_type = run_type::values_count_type;
+ using bit_size_type = run_type::bit_size_type;
+
+ BitPackedDecoder() noexcept = default;
+
+ explicit BitPackedDecoder(run_type const& run) noexcept;
+
+ void Reset(run_type const& run) noexcept;
+
+ /// Return the number of values that can be advanced.
+ [[nodiscard]] constexpr values_count_type Remaining() const;
+
+ /// Return the size in bit in which each encoded value is written.
+ [[nodiscard]] constexpr bit_size_type ValueBitWidth() const;
+
+ /// Try to advance by as many values as provided.
+ /// Return the number of values skipped.
+ [[nodiscard]] values_count_type Advance(values_count_type batch_size);
+
+ /// Get the next value and return false if there are no more.
+ [[nodiscard]] bool Get(value_type* out_value);
+
+ /// Get a batch of values return the number of decoded elements.
+ [[nodiscard]] values_count_type GetBatch(value_type* out, values_count_type batch_size);
+
+ private:
+ ::arrow::bit_util::BitReader bit_reader_ = {};
+ bit_size_type value_bit_width_ = 0;
+ values_count_type remaining_count_ = 0;
+
+ static_assert(std::is_integral_v<value_type>,
+ "This class makes assumptions about integer endianness and padding");
+};
+
+/// Decoder class for RLE encoded data.
+template <typename T>
+class RleBitPackedDecoder {
+ public:
+ /// The type in which the data should be decoded.
+ using value_type = T;
+ using byte = RleBitPackedParser::byte;
Review Comment:
Same comment about type aliases...
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
Review Comment:
Can this be put inside `RleParser`? I don't think it deserves being at the top-level.
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
Review Comment:
Again, is it useful to expose this at the top-level?
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -84,32 +85,278 @@ namespace util {
/// (total 26 bytes, 1 byte overhead)
//
+class RleRun {
+ public:
+ using byte = uint8_t;
+ /// Enough space to store a 64bit value
+ using raw_data_storage = std::array<byte, 8>;
+ using raw_data_const_pointer = const byte*;
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr RleRun() noexcept = default;
+ constexpr RleRun(RleRun const&) noexcept = default;
+ constexpr RleRun(RleRun&&) noexcept = default;
+
+ explicit RleRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr RleRun& operator=(RleRun const&) noexcept = default;
+ constexpr RleRun& operator=(RleRun&&) noexcept = default;
+
+ /// The number of repeated values in this run.
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ /// A pointer to the repeated value raw bytes.
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ /// The number of bytes used for the raw repeated value.
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The repeated value raw bytes stored inside the class
+ raw_data_storage data_ = {};
+ /// The number of time the value is repeated
+ values_count_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+class BitPackedRun {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// According to the Parquet thrift definition the page size can be written into an
+ /// int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr BitPackedRun() noexcept = default;
+ constexpr BitPackedRun(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun(BitPackedRun&&) noexcept = default;
+
+ constexpr BitPackedRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr BitPackedRun& operator=(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun& operator=(BitPackedRun&&) noexcept = default;
+
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Number of values in this run.
+ raw_data_size_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+/// A parser that emits either a ``BitPackedRun`` or a ``RleRun``.
+class RleBitPackedParser {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// By Parquet thrift definition the page size can be written into an int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+ /// The different types of runs emitted by the parser
+ using dynamic_run_type = std::variant<RleRun, BitPackedRun>;
+
+ constexpr RleBitPackedParser() noexcept = default;
+
+ constexpr RleBitPackedParser(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr void Reset(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width_) noexcept;
+
+ /// Get the current run with a small parsing cost without advancing the iteration.
+ [[nodiscard]] std::optional<dynamic_run_type> Peek() const;
+
+ /// Move to the next run.
+ [[nodiscard]] bool Advance();
+
+ /// Advance and return the current run.
+ [[nodiscard]] std::optional<dynamic_run_type> Next();
+
+ /// Whether there is still runs to iterate over.
+ ///
+ /// WARN: Due to lack of proper error handling, iteration with Next and Peek could
+ /// return not data while the parser is not exhausted.
+ /// This is how one can check for errors.
+ [[nodiscard]] bool Exhausted() const;
+
+ /// Enum to return from an ``Parse`` handler.
+ ///
+ /// Since a callback has no way to know when to stop, the handler must return
+ /// a value indicating to the ``Parse`` function whether to stop or continue.
+ enum class ControlFlow {
+ Continue,
+ Break,
+ };
+
+ /// A callback approach to parsing.
+ ///
+ /// This approach is used to reduce the number of dynamic lookups involved with using a
+ /// variant.
+ ///
+ /// The handler must be of the form
+ /// ```cpp`
+ /// struct Handler {
+ /// ControlFlow OnBitPackedRun(BitPackedRun run);
+ ///
+ /// ControlFlow OnRleRun(RleRun run);
+ /// };
+ /// ```
+ template <typename Handler>
+ void Parse(Handler&& handler);
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Size in bytes of the run.
+ raw_data_size_type data_size_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+
+ /// Run the handler on the run read and return the number of values read.
+ /// Does not advance the parser.
+ template <typename Handler>
+ std::pair<raw_data_size_type, ControlFlow> PeekImpl(Handler&&) const;
+};
+
/// Decoder class for RLE encoded data.
+template <typename T>
class RleDecoder {
public:
- /// Create a decoder object. buffer/buffer_len is the decoded data.
- /// bit_width is the width of each value (before encoding).
- RleDecoder(const uint8_t* buffer, int buffer_len, int bit_width)
- : bit_reader_(buffer, buffer_len),
- bit_width_(bit_width),
- current_value_(0),
- repeat_count_(0),
- literal_count_(0) {
- ARROW_DCHECK_GE(bit_width_, 0);
- ARROW_DCHECK_LE(bit_width_, 64);
- }
+ /// The type in which the data should be decoded.
+ using value_type = T;
+ /// The type of run that can be decoded.
+ using run_type = RleRun;
+ using values_count_type = run_type::values_count_type;
- RleDecoder() : bit_width_(-1) {}
+ constexpr RleDecoder() noexcept = default;
- void Reset(const uint8_t* buffer, int buffer_len, int bit_width) {
- ARROW_DCHECK_GE(bit_width, 0);
- ARROW_DCHECK_LE(bit_width, 64);
- bit_reader_.Reset(buffer, buffer_len);
- bit_width_ = bit_width;
- current_value_ = 0;
- repeat_count_ = 0;
- literal_count_ = 0;
- }
+ explicit RleDecoder(run_type const& run) noexcept;
Review Comment:
Also, I would expect all methods to be marked `inline` if that is really the case.
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
Review Comment:
Or this could simply be hidden in a convenience method in `RleParser`, such as:
```c++
template <typename Callable>
void ParseWithCallable(Callable&& func) {
struct {
Callable func;
auto OnBitPackedRun(BitPackedRun run) { return func(std::move(run)); }
auto OnRleRun(RleRun run) { return func(std::move(run)); }
} handler{std::move(func)};
Parse(std::move(handler));
}
```
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
Review Comment:
I'm not sure the `ARROW_PREDICT_FALSE` is desirable here? That depends on how the caller decides to batch data.
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -84,32 +85,278 @@ namespace util {
/// (total 26 bytes, 1 byte overhead)
//
+class RleRun {
+ public:
+ using byte = uint8_t;
+ /// Enough space to store a 64bit value
+ using raw_data_storage = std::array<byte, 8>;
+ using raw_data_const_pointer = const byte*;
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr RleRun() noexcept = default;
+ constexpr RleRun(RleRun const&) noexcept = default;
+ constexpr RleRun(RleRun&&) noexcept = default;
+
+ explicit RleRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr RleRun& operator=(RleRun const&) noexcept = default;
+ constexpr RleRun& operator=(RleRun&&) noexcept = default;
+
+ /// The number of repeated values in this run.
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ /// A pointer to the repeated value raw bytes.
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ /// The number of bytes used for the raw repeated value.
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The repeated value raw bytes stored inside the class
+ raw_data_storage data_ = {};
+ /// The number of time the value is repeated
+ values_count_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+class BitPackedRun {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// According to the Parquet thrift definition the page size can be written into an
+ /// int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr BitPackedRun() noexcept = default;
+ constexpr BitPackedRun(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun(BitPackedRun&&) noexcept = default;
+
+ constexpr BitPackedRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr BitPackedRun& operator=(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun& operator=(BitPackedRun&&) noexcept = default;
+
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Number of values in this run.
+ raw_data_size_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+/// A parser that emits either a ``BitPackedRun`` or a ``RleRun``.
+class RleBitPackedParser {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// By Parquet thrift definition the page size can be written into an int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+ /// The different types of runs emitted by the parser
+ using dynamic_run_type = std::variant<RleRun, BitPackedRun>;
+
+ constexpr RleBitPackedParser() noexcept = default;
+
+ constexpr RleBitPackedParser(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr void Reset(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width_) noexcept;
+
+ /// Get the current run with a small parsing cost without advancing the iteration.
+ [[nodiscard]] std::optional<dynamic_run_type> Peek() const;
+
+ /// Move to the next run.
+ [[nodiscard]] bool Advance();
+
+ /// Advance and return the current run.
+ [[nodiscard]] std::optional<dynamic_run_type> Next();
+
+ /// Whether there is still runs to iterate over.
+ ///
+ /// WARN: Due to lack of proper error handling, iteration with Next and Peek could
+ /// return not data while the parser is not exhausted.
+ /// This is how one can check for errors.
+ [[nodiscard]] bool Exhausted() const;
+
+ /// Enum to return from an ``Parse`` handler.
+ ///
+ /// Since a callback has no way to know when to stop, the handler must return
+ /// a value indicating to the ``Parse`` function whether to stop or continue.
+ enum class ControlFlow {
+ Continue,
+ Break,
+ };
+
+ /// A callback approach to parsing.
+ ///
+ /// This approach is used to reduce the number of dynamic lookups involved with using a
+ /// variant.
+ ///
+ /// The handler must be of the form
+ /// ```cpp`
+ /// struct Handler {
+ /// ControlFlow OnBitPackedRun(BitPackedRun run);
+ ///
+ /// ControlFlow OnRleRun(RleRun run);
+ /// };
+ /// ```
+ template <typename Handler>
+ void Parse(Handler&& handler);
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Size in bytes of the run.
+ raw_data_size_type data_size_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+
+ /// Run the handler on the run read and return the number of values read.
+ /// Does not advance the parser.
+ template <typename Handler>
+ std::pair<raw_data_size_type, ControlFlow> PeekImpl(Handler&&) const;
+};
+
/// Decoder class for RLE encoded data.
+template <typename T>
class RleDecoder {
Review Comment:
To avoid confusion, I would suggest renaming this to `RleRunDecoder` (it decodes a single run).
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
Review Comment:
Typo
```suggestion
Lambda handler_;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2322069610
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -207,12 +209,310 @@ TEST(BitUtil, RoundTripIntValues) {
}
}
+/// A Rle run is a simple class owning some data and a repetition count.
+/// It does not know how to read such data.
+TEST(Rle, RleRun) {
+ const std::array<RleRun::byte, 4> value = {21, 2, 0, 0};
+
+ RleRun::values_count_type value_count = 12;
+
+ // 12 times the value 21 fitting over 5 bits
+ auto const run_5 = RleRun(value.data(), value_count, /* value_bit_width= */ 5);
+ EXPECT_EQ(run_5.ValuesCount(), value_count);
+ EXPECT_EQ(run_5.ValuesBitWidth(), 5);
+ EXPECT_EQ(run_5.RawDataSize(), 1); // 5 bits fit in one byte
+ EXPECT_EQ(*run_5.RawDataPtr(), 21);
+
+ // 12 times the value 21 fitting over 16 bits
+ auto const run_8 = RleRun(value.data(), value_count, /* value_bit_width= */ 8);
+ EXPECT_EQ(run_8.ValuesCount(), value_count);
+ EXPECT_EQ(run_8.ValuesBitWidth(), 8);
+ EXPECT_EQ(run_8.RawDataSize(), 1); // 8 bits fit in 1 byte
+ EXPECT_EQ(*run_8.RawDataPtr(), 21);
+
+ // 12 times the value {21, 2} fitting over 10 bits
+ auto const run_10 = RleRun(value.data(), value_count, /* value_bit_width= */ 10);
+
+ EXPECT_EQ(run_10.ValuesCount(), value_count);
+ EXPECT_EQ(run_10.ValuesBitWidth(), 10);
+ EXPECT_EQ(run_10.RawDataSize(), 2); // 10 bits fit in 2 bytes
+ EXPECT_EQ(*(run_10.RawDataPtr() + 0), 21);
+ EXPECT_EQ(*(run_10.RawDataPtr() + 1), 2);
+
+ // 12 times the value {21, 2} fitting over 32 bits
+ auto const run_32 = RleRun(value.data(), value_count, /* value_bit_width= */ 32);
+ EXPECT_EQ(run_32.ValuesCount(), value_count);
+ EXPECT_EQ(run_32.ValuesBitWidth(), 32);
+ EXPECT_EQ(run_32.RawDataSize(), 4); // 32 bits fit in 4 bytes
+ EXPECT_EQ(*(run_32.RawDataPtr() + 0), 21);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 1), 2);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 2), 0);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 3), 0);
+}
+
+/// A BitPacked run is a simple class owning some data and its size.
+/// It does not know how to read such data.
+TEST(BitPacked, BitPackedRun) {
+ const std::array<BitPackedRun::byte, 4> value = {0b10101010, 0, 0, 0b1111111};
+
+ /// 16 values of 1 bit for a total of 16 bits
Review Comment:
Again, let's use `//` for simple comments not `///`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2322126250
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -207,12 +209,310 @@ TEST(BitUtil, RoundTripIntValues) {
}
}
+/// A Rle run is a simple class owning some data and a repetition count.
+/// It does not know how to read such data.
+TEST(Rle, RleRun) {
+ const std::array<RleRun::byte, 4> value = {21, 2, 0, 0};
+
+ RleRun::values_count_type value_count = 12;
+
+ // 12 times the value 21 fitting over 5 bits
+ auto const run_5 = RleRun(value.data(), value_count, /* value_bit_width= */ 5);
+ EXPECT_EQ(run_5.ValuesCount(), value_count);
+ EXPECT_EQ(run_5.ValuesBitWidth(), 5);
+ EXPECT_EQ(run_5.RawDataSize(), 1); // 5 bits fit in one byte
+ EXPECT_EQ(*run_5.RawDataPtr(), 21);
+
+ // 12 times the value 21 fitting over 16 bits
+ auto const run_8 = RleRun(value.data(), value_count, /* value_bit_width= */ 8);
+ EXPECT_EQ(run_8.ValuesCount(), value_count);
+ EXPECT_EQ(run_8.ValuesBitWidth(), 8);
+ EXPECT_EQ(run_8.RawDataSize(), 1); // 8 bits fit in 1 byte
+ EXPECT_EQ(*run_8.RawDataPtr(), 21);
+
+ // 12 times the value {21, 2} fitting over 10 bits
+ auto const run_10 = RleRun(value.data(), value_count, /* value_bit_width= */ 10);
+
+ EXPECT_EQ(run_10.ValuesCount(), value_count);
+ EXPECT_EQ(run_10.ValuesBitWidth(), 10);
+ EXPECT_EQ(run_10.RawDataSize(), 2); // 10 bits fit in 2 bytes
+ EXPECT_EQ(*(run_10.RawDataPtr() + 0), 21);
+ EXPECT_EQ(*(run_10.RawDataPtr() + 1), 2);
+
+ // 12 times the value {21, 2} fitting over 32 bits
+ auto const run_32 = RleRun(value.data(), value_count, /* value_bit_width= */ 32);
+ EXPECT_EQ(run_32.ValuesCount(), value_count);
+ EXPECT_EQ(run_32.ValuesBitWidth(), 32);
+ EXPECT_EQ(run_32.RawDataSize(), 4); // 32 bits fit in 4 bytes
+ EXPECT_EQ(*(run_32.RawDataPtr() + 0), 21);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 1), 2);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 2), 0);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 3), 0);
+}
+
+/// A BitPacked run is a simple class owning some data and its size.
+/// It does not know how to read such data.
+TEST(BitPacked, BitPackedRun) {
+ const std::array<BitPackedRun::byte, 4> value = {0b10101010, 0, 0, 0b1111111};
+
+ /// 16 values of 1 bit for a total of 16 bits
+ BitPackedRun::values_count_type value_count_1 = 16;
+ auto const run_1 = BitPackedRun(value.data(), value_count_1, /* value_bit_width= */ 1);
+ EXPECT_EQ(run_1.ValuesCount(), value_count_1);
+ EXPECT_EQ(run_1.ValuesBitWidth(), 1);
+ EXPECT_EQ(run_1.RawDataSize(), 2); // 16 bits fit in 2 bytes
+ for (BitPackedRun::raw_data_size_type i = 0; i < run_1.RawDataSize(); ++i) {
+ EXPECT_EQ(*(run_1.RawDataPtr() + i), value[i]);
+ }
+
+ /// 8 values of 3 bits for a total of 24 bits
+ BitPackedRun::values_count_type value_count_3 = 8;
+ auto const run_3 = BitPackedRun(value.data(), value_count_3, /* value_bit_width= */ 3);
+ EXPECT_EQ(run_3.ValuesCount(), value_count_3);
+ EXPECT_EQ(run_3.ValuesBitWidth(), 3);
+ EXPECT_EQ(run_3.RawDataSize(), 3); // 24 bits fit in 3 bytes
+ for (BitPackedRun::raw_data_size_type i = 0; i < run_3.RawDataSize(); ++i) {
+ EXPECT_EQ(*(run_3.RawDataPtr() + i), value[i]);
+ }
+}
+
+template <typename T>
+void TestRleDecoder(std::vector<RleRun::byte> bytes,
+ RleRun::values_count_type value_count,
+ RleRun::bit_size_type bit_width) {
+ // Pre-requisite for this test
+ EXPECT_GT(value_count, 6);
+
+ // Compute value associated with bytes encoded as little endian
+ T value = 0;
+ for (std::size_t i = 0; i < bytes.size(); ++i) {
+ value += static_cast<T>(bytes.at(i)) << (8 * i);
+ }
+
+ auto const run = RleRun(bytes.data(), value_count, bit_width);
+
+ auto decoder = RleDecoder<T>(run);
+ std::vector<T> vals = {0, 0};
+
+ EXPECT_EQ(decoder.Remaining(), value_count);
+
+ typename decltype(decoder)::values_count_type read = 0;
+ EXPECT_EQ(decoder.Get(vals.data()), 1);
+ read += 1;
+ EXPECT_EQ(vals.at(0), value);
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ EXPECT_EQ(decoder.Advance(3), 3);
+ read += 3;
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ vals = {0, 0};
+ EXPECT_EQ(decoder.GetBatch(vals.data(), 2), vals.size());
+ EXPECT_EQ(vals.at(0), value);
+ EXPECT_EQ(vals.at(1), value);
+ read += static_cast<decltype(read)>(vals.size());
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ // Exhaust iteration
+ EXPECT_EQ(decoder.Advance(value_count - read), value_count - read);
+ EXPECT_EQ(decoder.Remaining(), 0);
+ EXPECT_EQ(decoder.Advance(1), 0);
+ vals = {0, 0};
+ EXPECT_EQ(decoder.Get(vals.data()), 0);
+ EXPECT_EQ(vals.at(0), 0);
+
+ // Reset the decoder
+ decoder.Reset(run);
+ EXPECT_EQ(decoder.Remaining(), value_count);
+ vals = {0, 0};
+ EXPECT_EQ(decoder.GetBatch(vals.data(), 2), vals.size());
+ EXPECT_EQ(vals.at(0), value);
+ EXPECT_EQ(vals.at(1), value);
+}
+
+TEST(Rle, RleDecoder) {
+ TestRleDecoder<uint32_t>({21, 0, 0}, /* value_count= */ 21, /* bit_width= */ 5);
+ TestRleDecoder<uint16_t>({1, 0}, /* value_count= */ 13, /* bit_width= */ 1);
+ TestRleDecoder<uint64_t>({21, 2, 0, 1}, /* value_count= */ 20, /* bit_width= */ 30);
+}
+
+template <typename T>
+void TestBitPackedDecoder(std::vector<RleRun::byte> bytes,
+ BitPackedRun::values_count_type value_count,
+ BitPackedRun::bit_size_type bit_width,
+ std::vector<T> expected) {
+ // Pre-requisite for this test
+ EXPECT_GT(value_count, 6);
+
+ auto const run = BitPackedRun(bytes.data(), value_count, bit_width);
+
+ auto decoder = BitPackedDecoder<T>(run);
+ std::vector<T> vals = {0, 0};
+
+ EXPECT_EQ(decoder.Remaining(), value_count);
+
+ typename decltype(decoder)::values_count_type read = 0;
+ EXPECT_EQ(decoder.Get(vals.data()), 1);
+ EXPECT_EQ(vals.at(0), expected.at(0 + read));
+ read += 1;
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ EXPECT_EQ(decoder.Advance(3), 3);
+ read += 3;
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ vals = {0, 0};
+ EXPECT_EQ(decoder.GetBatch(vals.data(), 2), vals.size());
+ EXPECT_EQ(vals.at(0), expected.at(0 + read));
+ EXPECT_EQ(vals.at(1), expected.at(1 + read));
+ read += static_cast<decltype(read)>(vals.size());
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ // Exhaust iteration
+ EXPECT_EQ(decoder.Advance(value_count - read), value_count - read);
+ EXPECT_EQ(decoder.Remaining(), 0);
+ EXPECT_EQ(decoder.Advance(1), 0);
+ vals = {0, 0};
+ EXPECT_EQ(decoder.Get(vals.data()), 0);
+ EXPECT_EQ(vals.at(0), 0);
+
+ // Reset the decoder
+ decoder.Reset(run);
+ read = 0;
+ EXPECT_EQ(decoder.Remaining(), value_count);
+ vals = {0, 0};
+ EXPECT_EQ(decoder.GetBatch(vals.data(), 2), vals.size());
+ EXPECT_EQ(vals.at(0), expected.at(0 + read));
+ EXPECT_EQ(vals.at(1), expected.at(1 + read));
+}
+
+TEST(BitPacked, BitPackedDecoder) {
+ /// See parquet encoding for bytes layout
+ TestBitPackedDecoder<uint16_t>(
+ /* bytes= */ {0x88, 0xc6, 0xfa},
+ /* values_count= */ 8,
+ /* bit_width= */ 3,
+ /* expected= */ {0, 1, 2, 3, 4, 5, 6, 7});
+ TestBitPackedDecoder<uint8_t>(
+ /* bytes= */ {0x0, 0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7},
+ /* values_count= */ 8,
+ /* bit_width= */ 8,
+ /* expected= */ {0, 1, 2, 3, 4, 5, 6, 7});
+ TestBitPackedDecoder<uint32_t>(
+ /* bytes= */ {0x47, 0xc, 0x10, 0x35},
+ /* values_count= */ 8,
+ /* bit_width= */ 4,
+ /* expected= */ {7, 4, 12, 0, 0, 1, 5, 3});
+ TestBitPackedDecoder<uint64_t>(
+ /* bytes= */ {0xe8, 0x7, 0x20, 0xc0, 0x0, 0x4, 0x14, 0x60, 0xc0, 0x1},
+ /* values_count= */ 8,
+ /* bit_width= */ 10,
+ /* expected= */ {1000, 1, 2, 3, 4, 5, 6, 7});
+}
+
+template <typename T>
+void TestRleBitPackedParser(std::vector<RleRun::byte> bytes,
+ RleBitPackedParser::bit_size_type bit_width,
+ std::vector<T> expected) {
+ auto parser = RleBitPackedParser(
+ bytes.data(), static_cast<BitPackedRun::raw_data_size_type>(bytes.size()),
+ bit_width);
+ EXPECT_FALSE(parser.Exhausted());
+
+ // Peek return the same data
+ auto run1 = parser.Peek();
+ EXPECT_TRUE(run1.has_value());
+ auto run2 = parser.Peek();
+ EXPECT_TRUE(run2.has_value());
+ auto ptr1 = std::visit([](auto const& r) { return r.RawDataPtr(); }, run1.value());
+ auto size1 = std::visit([](auto const& r) { return r.RawDataSize(); }, run1.value());
+ auto ptr2 = std::visit([](auto const& r) { return r.RawDataPtr(); }, run2.value());
+ auto size2 = std::visit([](auto const& r) { return r.RawDataSize(); }, run2.value());
+ EXPECT_TRUE(std::equal(ptr1, ptr1 + size1, ptr2, ptr2 + size2));
Review Comment:
Why not compare the run structures? You could even define a `operator==` for them (or a custom test helper here).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2319547667
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ ARROW_DCHECK_LT(values_read, batch_size);
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+ auto const read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
return values_read;
}
-template <typename T, typename RunType, typename Converter>
-inline int RleDecoder::GetSpaced(Converter converter, int batch_size, int null_count,
- const uint8_t* valid_bits, int64_t valid_bits_offset,
- T* out) {
- if (ARROW_PREDICT_FALSE(null_count == batch_size)) {
- converter.FillZero(out, out + batch_size);
- return batch_size;
+namespace internal {
+
+/// Utility class to safely handle values and null count without too error-prone
+/// verbosity.
+class BatchCounter {
+ public:
+ using size_type = int32_t;
+
+ [[nodiscard]] static constexpr BatchCounter FromBatchSizeAndNulls(
+ size_type batch_size, size_type null_count) {
+ ARROW_DCHECK_LE(null_count, batch_size);
+ return {batch_size - null_count, null_count};
}
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
- int values_remaining = batch_size - null_count;
+ constexpr BatchCounter(size_type values_count, size_type null_count) noexcept
+ : values_count_(values_count), null_count_(null_count) {}
- // Assume no bits to start.
- arrow::internal::BitRunReader bit_reader(valid_bits, valid_bits_offset,
- /*length=*/batch_size);
- arrow::internal::BitRun valid_run = bit_reader.NextRun();
- while (values_read < batch_size) {
- if (ARROW_PREDICT_FALSE(valid_run.length == 0)) {
- valid_run = bit_reader.NextRun();
+ [[nodiscard]] constexpr size_type ValuesCount() const noexcept { return values_count_; }
+
+ [[nodiscard]] constexpr size_type ValuesRead() const noexcept { return values_read_; }
+
+ [[nodiscard]] constexpr size_type ValuesRemaining() const noexcept {
+ ARROW_DCHECK_LE(values_read_, values_count_);
+ return values_count_ - values_read_;
+ }
+
+ constexpr void AccrueReadValues(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, ValuesRemaining());
+ values_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type NullCount() const noexcept { return null_count_; }
+
+ [[nodiscard]] constexpr size_type NullRead() const noexcept { return null_read_; }
+
+ [[nodiscard]] constexpr size_type NullRemaining() const noexcept {
+ ARROW_DCHECK_LE(null_read_, null_count_);
+ return null_count_ - null_read_;
+ }
+
+ constexpr void AccrueReadNulls(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, NullRemaining());
+ null_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type TotalRemaining() const noexcept {
+ return ValuesRemaining() + NullRemaining();
+ }
+
+ [[nodiscard]] constexpr size_type TotalRead() const noexcept {
+ return values_read_ + null_read_;
+ }
+
+ [[nodiscard]] constexpr bool IsFullyNull() const noexcept {
+ return ValuesRemaining() == 0;
+ }
+
+ [[nodiscard]] constexpr bool IsDone() const noexcept { return TotalRemaining() == 0; }
+
+ private:
+ size_type values_count_ = 0;
+ size_type values_read_ = 0;
+ size_type null_count_ = 0;
+ size_type null_read_ = 0;
+};
+
+// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+/// Overload for GetSpaced for a single run in a RleDecoder
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ RleDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto values_remaining_run = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // Consume as much as possible from the repeated run.
+ // We only need to count the number of nulls and non-nulls because we can fill in the
+ // same value for nulls and non-nulls.
+ // This proves to be a big efficiency win.
+ while (values_remaining_run() > 0 && !batch.IsDone()) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const& validity_run_size = static_cast<values_count_type>(validity_run.length);
+
+ if (validity_run.set) {
+ // We may end the current RLE run in the middle of the validity run
+ auto update_size = std::min(validity_run_size, values_remaining_run());
+ batch.AccrueReadValues(update_size);
+ validity_run.length -= update_size;
+ } else {
+ // We can consume all nulls here because it does not matter if we consume on this
+ // RLE run, or an a next encoded run. The value filled does not matter.
+ auto update_size = std::min(validity_run_size, batch.NullRemaining());
+ batch.AccrueReadNulls(update_size);
+ validity_run.length -= update_size;
+ }
+
+ if (ARROW_PREDICT_TRUE(validity_run.length == 0)) {
+ validity_run = validity_reader.NextRun();
}
+ }
+
+ value_type const value = decoder.Value();
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(value))) {
+ return {0, 0};
+ }
+ converter.WriteRepeated(out, out + batch.TotalRead(), value);
+ auto const actual_values_read = decoder.Advance(batch.ValuesRead());
+ // We always cropped the number of values_read by the remaining values in the run.
+ // What's more the RLE decoder should not encounter any errors.
+ ARROW_DCHECK_EQ(actual_values_read, batch.ValuesRead());
- ARROW_DCHECK_GT(batch_size, 0);
- ARROW_DCHECK_GT(valid_run.length, 0);
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+template <typename T, typename... Ts>
+[[nodiscard]] constexpr T min(T x, Ts... ys) {
+ ((x = std::min(x, ys)), ...);
+ return x;
+}
+
+static_assert(min(5) == 5);
+static_assert(min(5, 4, -1) == -1);
+static_assert(min(5, 41) == 5);
+
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ BitPackedDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto run_values_remaining = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ while (run_values_remaining() > 0 && batch.ValuesRemaining() > 0) {
+ // TODO should this size be tune depending on sizeof(value_size)? cpu cache size?
+ // Pull a batch of values from the bit packed encoded data and store it in a local
+ // buffer to benefit from unpacking intrinsics and data locality.
+ static constexpr values_count_type kBufferCapacity = 1024;
+ std::array<value_type, kBufferCapacity> buffer = {};
+
+ values_count_type buffer_start = 0;
+ values_count_type buffer_end = 0;
+ auto buffer_size = [&]() {
+ auto out = buffer_end - buffer_start;
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // buffer_start is 0 at this point so size is end
+ buffer_end = min(run_values_remaining(), batch.ValuesRemaining(), kBufferCapacity);
+ buffer_end = decoder.GetBatch(buffer.data(), buffer_size());
+ ARROW_DCHECK_LE(buffer_size(), kBufferCapacity);
+
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(buffer.data(), buffer_size()))) {
+ return {batch.ValuesRead(), batch.NullRead()};
+ }
- if (valid_run.set) {
- if ((repeat_count_ == 0) && (literal_count_ == 0)) {
- if (!NextCounts<RunType>()) return values_read;
- ARROW_DCHECK((repeat_count_ > 0) ^ (literal_count_ > 0));
+ // Copy chunks of valid values into the output, while adjusting spacing for null
+ // values.
+ while (buffer_size() > 0) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const validity_run_length =
+ static_cast<values_count_type>(validity_run.length);
+
+ // Copy as much as possible from the buffer into the output while not exceeding
+ // validity run
+ if (validity_run.set) {
+ auto const update_size = std::min(validity_run_length, buffer_size());
+ converter.WriteRange(out, buffer.data() + buffer_start, update_size);
+ buffer_start += update_size;
+ batch.AccrueReadValues(update_size);
+ out += update_size;
+ validity_run.length -= update_size;
+ // Simply write zeros in the output
+ } else {
+ auto const update_size = std::min(validity_run_length, batch.NullRemaining());
+ converter.WriteZero(out, out + update_size);
+ batch.AccrueReadNulls(update_size);
+ out += update_size;
+ validity_run.length -= update_size;
}
- if (repeat_count_ > 0) {
- int repeat_batch = 0;
- // Consume the entire repeat counts incrementing repeat_batch to
- // be the total of nulls + values consumed, we only need to
- // get the total count because we can fill in the same value for
- // nulls and non-nulls. This proves to be a big efficiency win.
- while (repeat_count_ > 0 && (values_read + repeat_batch) < batch_size) {
- ARROW_DCHECK_GT(valid_run.length, 0);
- if (valid_run.set) {
- int update_size = std::min(static_cast<int>(valid_run.length), repeat_count_);
- repeat_count_ -= update_size;
- repeat_batch += update_size;
- valid_run.length -= update_size;
- values_remaining -= update_size;
- } else {
- // We can consume all nulls here because we would do so on
- // the next loop anyways.
- repeat_batch += static_cast<int>(valid_run.length);
- valid_run.length = 0;
- }
- if (valid_run.length == 0) {
- valid_run = bit_reader.NextRun();
- }
- }
- RunType current_value = static_cast<RunType>(current_value_);
- if (ARROW_PREDICT_FALSE(!converter.IsValid(current_value))) {
- return values_read;
- }
- converter.Fill(out, out + repeat_batch, current_value);
- out += repeat_batch;
- values_read += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(values_remaining, literal_count_);
- ARROW_DCHECK_GT(literal_batch, 0);
-
- // Decode the literals
- constexpr int kBufferSize = 1024;
- RunType indices[kBufferSize];
- literal_batch = std::min(literal_batch, kBufferSize);
- int actual_read = bit_reader_.GetBatch(bit_width_, indices, literal_batch);
- if (ARROW_PREDICT_FALSE(actual_read != literal_batch)) {
- return values_read;
- }
- if (!converter.IsValid(indices, /*length=*/actual_read)) {
- return values_read;
- }
- int skipped = 0;
- int literals_read = 0;
- while (literals_read < literal_batch) {
- if (valid_run.set) {
- int update_size = std::min(literal_batch - literals_read,
- static_cast<int>(valid_run.length));
- converter.Copy(out, indices + literals_read, update_size);
- literals_read += update_size;
- out += update_size;
- valid_run.length -= update_size;
- } else {
- converter.FillZero(out, out + valid_run.length);
- out += valid_run.length;
- skipped += static_cast<int>(valid_run.length);
- valid_run.length = 0;
- }
- if (valid_run.length == 0) {
- valid_run = bit_reader.NextRun();
- }
- }
- literal_count_ -= literal_batch;
- values_remaining -= literal_batch;
- values_read += literal_batch + skipped;
+ if (validity_run.length == 0) {
+ validity_run = validity_reader.NextRun();
}
- } else {
- converter.FillZero(out, out + valid_run.length);
- out += valid_run.length;
- values_read += static_cast<int>(valid_run.length);
- valid_run.length = 0;
}
+
+ ARROW_DCHECK_EQ(buffer_size(), 0);
}
- ARROW_DCHECK_EQ(valid_run.length, 0);
- ARROW_DCHECK_EQ(values_remaining, 0);
- return values_read;
+
+ ARROW_DCHECK_EQ(values_available - decoder.Remaining(), batch.ValuesRead());
+ ARROW_DCHECK_LE(batch.TotalRead(), batch_size);
+ ARROW_DCHECK_LE(batch.NullRead(), batch.NullCount());
+
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+/// Overload for GetSpaced for a single run in a decoder variant
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(
+ Converter& converter, typename Converter::out_type* out, values_count_type batch_size,
+ values_count_type null_count, BitRunReader&& validity_reader, BitRun&& validity_run,
+ std::variant<RleDecoder<value_type>, BitPackedDecoder<value_type>>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ return std::visit(
+ [&](auto& dec) {
+ ARROW_DCHECK_GT(dec.Remaining(), 0);
+ return RunGetSpaced(converter, out, batch_size, null_count, validity_reader,
+ validity_run, dec);
+ },
+ decoder);
}
+} // namespace internal
+
+template <typename T>
+template <typename Converter>
+auto RleBitPackedDecoder<T>::GetSpaced(
+ Converter converter, typename Converter::out_type* out, values_count_type batch_size,
+ const byte* validity_bits, int64_t validity_bits_offset, values_count_type null_count)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
+
+ ARROW_DCHECK_GT(batch_size, 0);
+
+ auto batch = internal::BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ if (ARROW_PREDICT_FALSE(batch.IsFullyNull())) {
+ converter.WriteZero(out, out + batch.NullRemaining());
+ return batch.NullRemaining();
+ }
+
+ arrow::internal::BitRunReader validity_reader(validity_bits, validity_bits_offset,
+ /*length=*/batch.TotalRemaining());
+ arrow::internal::BitRun validity_run = validity_reader.NextRun();
+
+ auto const check_and_handle_fully_null_remaining = [&]() {
+ if (batch.IsFullyNull()) {
+ ARROW_DCHECK(validity_run.length == 0 || !validity_run.set);
+ ARROW_DCHECK_GE(validity_run.length, batch.NullRemaining());
+
+ converter.WriteZero(out, out + batch.NullRemaining());
+ out += batch.NullRemaining();
+ batch.AccrueReadNulls(batch.NullRemaining());
+ }
+ };
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const [values_read, null_read] =
+ RunGetSpaced(converter, out, batch.TotalRemaining(), batch.NullRemaining(),
+ validity_reader, validity_run, decoder_);
+
+ batch.AccrueReadNulls(null_read);
+ batch.AccrueReadValues(values_read);
+ out += values_read + null_read;
+
+ // Either we fulfilled all the batch values to be read
+ if (ARROW_PREDICT_FALSE(batch.ValuesRemaining() == 0)) {
+ // There may be remaining null if they are not greedily filled
+ check_and_handle_fully_null_remaining();
+ return batch.TotalRead();
+ }
+
+ /// We finished the remaining run
+ ARROW_DCHECK(RunRemaining() == 0);
+ }
+
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+
+ auto const [values_read, null_read] = internal::RunGetSpaced(
+ converter, out, batch.TotalRemaining(), batch.NullRemaining(),
+ validity_reader, validity_run, decoder);
+
+ batch.AccrueReadNulls(null_read);
+ batch.AccrueReadValues(values_read);
+ out += values_read + null_read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == 0 || batch.ValuesRemaining() == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
+ // There may be remaining null if they are not greedily filled by either decoder calls
+ check_and_handle_fully_null_remaining();
+
+ ARROW_DCHECK(batch.IsDone() || Exhausted());
+ // batch.Done() => batch.NullRemaining() == 0
+ ARROW_DCHECK(!batch.IsDone() || (batch.NullRemaining() == 0));
Review Comment:
If it's by definition of BatchCounter semantics, do we need to check for it here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2319532082
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ ARROW_DCHECK_LT(values_read, batch_size);
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+ auto const read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
return values_read;
}
-template <typename T, typename RunType, typename Converter>
-inline int RleDecoder::GetSpaced(Converter converter, int batch_size, int null_count,
- const uint8_t* valid_bits, int64_t valid_bits_offset,
- T* out) {
- if (ARROW_PREDICT_FALSE(null_count == batch_size)) {
- converter.FillZero(out, out + batch_size);
- return batch_size;
+namespace internal {
+
+/// Utility class to safely handle values and null count without too error-prone
+/// verbosity.
+class BatchCounter {
+ public:
+ using size_type = int32_t;
+
+ [[nodiscard]] static constexpr BatchCounter FromBatchSizeAndNulls(
+ size_type batch_size, size_type null_count) {
+ ARROW_DCHECK_LE(null_count, batch_size);
+ return {batch_size - null_count, null_count};
}
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
- int values_remaining = batch_size - null_count;
+ constexpr BatchCounter(size_type values_count, size_type null_count) noexcept
+ : values_count_(values_count), null_count_(null_count) {}
- // Assume no bits to start.
- arrow::internal::BitRunReader bit_reader(valid_bits, valid_bits_offset,
- /*length=*/batch_size);
- arrow::internal::BitRun valid_run = bit_reader.NextRun();
- while (values_read < batch_size) {
- if (ARROW_PREDICT_FALSE(valid_run.length == 0)) {
- valid_run = bit_reader.NextRun();
+ [[nodiscard]] constexpr size_type ValuesCount() const noexcept { return values_count_; }
+
+ [[nodiscard]] constexpr size_type ValuesRead() const noexcept { return values_read_; }
+
+ [[nodiscard]] constexpr size_type ValuesRemaining() const noexcept {
+ ARROW_DCHECK_LE(values_read_, values_count_);
+ return values_count_ - values_read_;
+ }
+
+ constexpr void AccrueReadValues(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, ValuesRemaining());
+ values_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type NullCount() const noexcept { return null_count_; }
+
+ [[nodiscard]] constexpr size_type NullRead() const noexcept { return null_read_; }
+
+ [[nodiscard]] constexpr size_type NullRemaining() const noexcept {
+ ARROW_DCHECK_LE(null_read_, null_count_);
+ return null_count_ - null_read_;
+ }
+
+ constexpr void AccrueReadNulls(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, NullRemaining());
+ null_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type TotalRemaining() const noexcept {
+ return ValuesRemaining() + NullRemaining();
+ }
+
+ [[nodiscard]] constexpr size_type TotalRead() const noexcept {
+ return values_read_ + null_read_;
+ }
+
+ [[nodiscard]] constexpr bool IsFullyNull() const noexcept {
+ return ValuesRemaining() == 0;
+ }
+
+ [[nodiscard]] constexpr bool IsDone() const noexcept { return TotalRemaining() == 0; }
+
+ private:
+ size_type values_count_ = 0;
+ size_type values_read_ = 0;
+ size_type null_count_ = 0;
+ size_type null_read_ = 0;
+};
+
+// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+/// Overload for GetSpaced for a single run in a RleDecoder
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ RleDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto values_remaining_run = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // Consume as much as possible from the repeated run.
+ // We only need to count the number of nulls and non-nulls because we can fill in the
+ // same value for nulls and non-nulls.
+ // This proves to be a big efficiency win.
+ while (values_remaining_run() > 0 && !batch.IsDone()) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const& validity_run_size = static_cast<values_count_type>(validity_run.length);
+
+ if (validity_run.set) {
+ // We may end the current RLE run in the middle of the validity run
+ auto update_size = std::min(validity_run_size, values_remaining_run());
+ batch.AccrueReadValues(update_size);
+ validity_run.length -= update_size;
+ } else {
+ // We can consume all nulls here because it does not matter if we consume on this
+ // RLE run, or an a next encoded run. The value filled does not matter.
+ auto update_size = std::min(validity_run_size, batch.NullRemaining());
+ batch.AccrueReadNulls(update_size);
+ validity_run.length -= update_size;
+ }
+
+ if (ARROW_PREDICT_TRUE(validity_run.length == 0)) {
+ validity_run = validity_reader.NextRun();
}
+ }
+
+ value_type const value = decoder.Value();
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(value))) {
+ return {0, 0};
+ }
+ converter.WriteRepeated(out, out + batch.TotalRead(), value);
+ auto const actual_values_read = decoder.Advance(batch.ValuesRead());
+ // We always cropped the number of values_read by the remaining values in the run.
+ // What's more the RLE decoder should not encounter any errors.
+ ARROW_DCHECK_EQ(actual_values_read, batch.ValuesRead());
- ARROW_DCHECK_GT(batch_size, 0);
- ARROW_DCHECK_GT(valid_run.length, 0);
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+template <typename T, typename... Ts>
+[[nodiscard]] constexpr T min(T x, Ts... ys) {
+ ((x = std::min(x, ys)), ...);
+ return x;
+}
+
+static_assert(min(5) == 5);
+static_assert(min(5, 4, -1) == -1);
+static_assert(min(5, 41) == 5);
+
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ BitPackedDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto run_values_remaining = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ while (run_values_remaining() > 0 && batch.ValuesRemaining() > 0) {
Review Comment:
For simplification, can we instead instantiate `batch` like this:
```c++
auto batch = BatchCounter::FromBatchSizeAndNulls(
std::min(batch_size, values_available + null_count), null_count);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "HuaHuaY (via GitHub)" <gi...@apache.org>.
HuaHuaY commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2306668207
##########
cpp/src/arrow/util/bit_util.h:
##########
@@ -365,5 +365,118 @@ void PackBits(const uint32_t* values, uint8_t* out) {
}
}
+constexpr int64_t MaxLEB128ByteLen(int64_t n_bits) { return CeilDiv(n_bits, 7); }
+
+template <typename Int>
+constexpr int64_t MaxLEB128ByteLenFor = MaxLEB128ByteLen(sizeof(Int) * 8);
+
+/// Write a integer as LEB128
+///
+/// Write the input value as LEB128 into the outptu buffer and return the number of bytes
+/// written.
+/// If the output buffer size is insufficient, return 0 but the output may have been
+/// written to.
+///
+/// \see https://en.wikipedia.org/wiki/LEB128
+/// \see MaxLEB128ByteLenFor
+template <typename Int>
+constexpr int32_t WriteLEB128(Int value, uint8_t* out, int32_t max_out_size) {
+ constexpr Int kLow7Mask = Int(0x7F);
+ constexpr Int kHigh7Mask = ~kLow7Mask;
+ constexpr uint8_t kContinuationBit = 0x80;
+
+ auto const out_first = out;
+
+ // Write as many bytes as the could be for the given input
+ while ((value & kHigh7Mask) != Int(0)) {
+ // We do not have enough room to write the LEB128
+ if (out - out_first >= max_out_size) {
+ return 0;
+ }
+
+ // Write the encoded byte with continuation bit
+ *out = static_cast<uint8_t>(value & kLow7Mask) | kContinuationBit;
+ ++out;
+ // Shift remaining data
+ value >>= 7;
+ }
+
+ // We do not have enough room to write the LEB128
+ if (out - out_first >= max_out_size) {
+ return 0;
+ }
+
+ // Write last non-continuing byte
+ *out = static_cast<uint8_t>(value & kLow7Mask);
+ ++out;
+
+ return static_cast<int32_t>(out - out_first);
+}
+
+/// Parse a leading LEB128
+///
+/// Take as input a data pointer and the maximum number of bytes that can be read from it
+/// (typically the array size).
+/// When a valid LEB128 is found at the start of the data, the function writes it to the
+/// out pointer and return the number of bytes read.
+/// Otherwise, the out pointer is unmodified and zero is returned.
+///
+/// \see https://en.wikipedia.org/wiki/LEB128
+/// \see MaxLEB128ByteLenFor
+template <typename Int>
+constexpr int32_t ParseLeadingLEB128(uint8_t const* data, int32_t max_data_size,
+ Int* out) {
+ constexpr auto kMaxBytes = static_cast<int32_t>(MaxLEB128ByteLenFor<Int>);
+ static_assert(kMaxBytes >= 1);
+ constexpr uint8_t kLow7Mask = 0x7F;
+ constexpr uint8_t kContinuationBit = 0x80;
+ constexpr int32_t kSignBitCount = std::is_signed_v<Int> ? 1 : 0;
+ // Number of bits allowed for encoding data on the last byte to avoid overflow
+ constexpr uint8_t kHighBitCount = (8 * sizeof(Int) - kSignBitCount) % 7;
+ // kHighBitCount least significant `0` bits and the rest with `1`
+ constexpr uint8_t kHighForbiddenMask = ~((1 << kHighBitCount) - 1);
+
+ // Iteratively building the value
+ std::make_unsigned_t<Int> value = 0;
+
+ // Read as many bytes as the could be for the give output
+ for (int32_t i = 0; i < kMaxBytes - 1; i++) {
+ // We have not finished reading a valid LEB128, yet we run out of data
+ if (ARROW_PREDICT_FALSE(i >= max_data_size)) {
+ return 0;
+ }
+
+ // Read the byte and set its 7 LSB to in the final value
+ uint8_t const byte = data[i];
+ value |= static_cast<Int>(byte & kLow7Mask) << (7 * i);
+
+ // Check for lack of continuation flag in MSB
+ if ((byte & kContinuationBit) == 0) {
+ *out = value;
+ return i + 1;
+ }
+ }
+
+ // Process the last index avoiding overflowing
+ constexpr int32_t last = kMaxBytes - 1;
+
+ // We have not finished reading a valid LEB128, yet we run out of data
+ if (ARROW_PREDICT_FALSE(last >= max_data_size)) {
+ return 0;
+ }
+
+ uint8_t const byte = data[last];
+
+ // Need to check if there are bits that would overflow the output.
+ // Also checks that there is no continuation.
+ if (ARROW_PREDICT_FALSE((byte & kHighForbiddenMask) != 0)) {
Review Comment:
In my opinion, due to right shift of `uint64_t`, high bit remains bit0 when original `BitWriter::PutVlqInt(uint64_t v)` writes the last byte. In which case should we treat the last byte specially in `WriteLEB128` and `ParseLeadingLEB128 `?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "HuaHuaY (via GitHub)" <gi...@apache.org>.
HuaHuaY commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2306668207
##########
cpp/src/arrow/util/bit_util.h:
##########
@@ -365,5 +365,118 @@ void PackBits(const uint32_t* values, uint8_t* out) {
}
}
+constexpr int64_t MaxLEB128ByteLen(int64_t n_bits) { return CeilDiv(n_bits, 7); }
+
+template <typename Int>
+constexpr int64_t MaxLEB128ByteLenFor = MaxLEB128ByteLen(sizeof(Int) * 8);
+
+/// Write a integer as LEB128
+///
+/// Write the input value as LEB128 into the outptu buffer and return the number of bytes
+/// written.
+/// If the output buffer size is insufficient, return 0 but the output may have been
+/// written to.
+///
+/// \see https://en.wikipedia.org/wiki/LEB128
+/// \see MaxLEB128ByteLenFor
+template <typename Int>
+constexpr int32_t WriteLEB128(Int value, uint8_t* out, int32_t max_out_size) {
+ constexpr Int kLow7Mask = Int(0x7F);
+ constexpr Int kHigh7Mask = ~kLow7Mask;
+ constexpr uint8_t kContinuationBit = 0x80;
+
+ auto const out_first = out;
+
+ // Write as many bytes as the could be for the given input
+ while ((value & kHigh7Mask) != Int(0)) {
+ // We do not have enough room to write the LEB128
+ if (out - out_first >= max_out_size) {
+ return 0;
+ }
+
+ // Write the encoded byte with continuation bit
+ *out = static_cast<uint8_t>(value & kLow7Mask) | kContinuationBit;
+ ++out;
+ // Shift remaining data
+ value >>= 7;
+ }
+
+ // We do not have enough room to write the LEB128
+ if (out - out_first >= max_out_size) {
+ return 0;
+ }
+
+ // Write last non-continuing byte
+ *out = static_cast<uint8_t>(value & kLow7Mask);
+ ++out;
+
+ return static_cast<int32_t>(out - out_first);
+}
+
+/// Parse a leading LEB128
+///
+/// Take as input a data pointer and the maximum number of bytes that can be read from it
+/// (typically the array size).
+/// When a valid LEB128 is found at the start of the data, the function writes it to the
+/// out pointer and return the number of bytes read.
+/// Otherwise, the out pointer is unmodified and zero is returned.
+///
+/// \see https://en.wikipedia.org/wiki/LEB128
+/// \see MaxLEB128ByteLenFor
+template <typename Int>
+constexpr int32_t ParseLeadingLEB128(uint8_t const* data, int32_t max_data_size,
+ Int* out) {
+ constexpr auto kMaxBytes = static_cast<int32_t>(MaxLEB128ByteLenFor<Int>);
+ static_assert(kMaxBytes >= 1);
+ constexpr uint8_t kLow7Mask = 0x7F;
+ constexpr uint8_t kContinuationBit = 0x80;
+ constexpr int32_t kSignBitCount = std::is_signed_v<Int> ? 1 : 0;
+ // Number of bits allowed for encoding data on the last byte to avoid overflow
+ constexpr uint8_t kHighBitCount = (8 * sizeof(Int) - kSignBitCount) % 7;
+ // kHighBitCount least significant `0` bits and the rest with `1`
+ constexpr uint8_t kHighForbiddenMask = ~((1 << kHighBitCount) - 1);
+
+ // Iteratively building the value
+ std::make_unsigned_t<Int> value = 0;
+
+ // Read as many bytes as the could be for the give output
+ for (int32_t i = 0; i < kMaxBytes - 1; i++) {
+ // We have not finished reading a valid LEB128, yet we run out of data
+ if (ARROW_PREDICT_FALSE(i >= max_data_size)) {
+ return 0;
+ }
+
+ // Read the byte and set its 7 LSB to in the final value
+ uint8_t const byte = data[i];
+ value |= static_cast<Int>(byte & kLow7Mask) << (7 * i);
+
+ // Check for lack of continuation flag in MSB
+ if ((byte & kContinuationBit) == 0) {
+ *out = value;
+ return i + 1;
+ }
+ }
+
+ // Process the last index avoiding overflowing
+ constexpr int32_t last = kMaxBytes - 1;
+
+ // We have not finished reading a valid LEB128, yet we run out of data
+ if (ARROW_PREDICT_FALSE(last >= max_data_size)) {
+ return 0;
+ }
+
+ uint8_t const byte = data[last];
+
+ // Need to check if there are bits that would overflow the output.
+ // Also checks that there is no continuation.
+ if (ARROW_PREDICT_FALSE((byte & kHighForbiddenMask) != 0)) {
Review Comment:
In my opinion, due to right shift of `uint64_t`, high bit remains bit 0 when original `BitWriter::PutVlqInt(uint64_t v)` writes the last byte. In which case should we treat the last byte specially in `WriteLEB128` and `ParseLeadingLEB128 `?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "conbench-apache-arrow[bot] (via GitHub)" <gi...@apache.org>.
conbench-apache-arrow[bot] commented on PR #47294:
URL: https://github.com/apache/arrow/pull/47294#issuecomment-3324914921
Thanks for your patience. Conbench analyzed the 4 benchmarking runs that have been run so far on PR commit fc4f31108625857d372ec3ff4248226af812e050.
There were 13 benchmark results indicating a performance regression:
- Pull Request Run on `amd64-c6a-4xlarge-linux` at [2025-09-23 11:51:59Z](https://conbench.ursa.dev/compare/runs/7360221cd26d4efba56f1af7836e5096...c7c006fd3e1549f3abf14b5cc7b6a500/)
- [`BenchmarkTemporalRounding` (C++) with params=<CeilTemporal, zoned, round_1_week>/524288/100, source=cpp-micro, suite=arrow-compute-scalar-temporal-benchmark](https://conbench.ursa.dev/compare/benchmarks/068d2360dff878ad8000504c6e49d68d...068d28aaa7c67132800031cbf466aeea)
- [`BM_RleDecoding` (C++) with params=65536/16, source=cpp-micro, suite=parquet-column-io-benchmark](https://conbench.ursa.dev/compare/benchmarks/068d236412e2706e8000cbe80be69c56...068d28acf9667d4b800027bdb42665a8)
- and 11 more (see the report linked below)
The [full Conbench report](https://github.com/apache/arrow/runs/51060237288) has more details.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2355088657
##########
cpp/src/arrow/util/bit_stream_utils_internal.h:
##########
@@ -439,91 +424,84 @@ inline bool BitReader::Advance(int64_t num_bits) {
return true;
}
-inline bool BitWriter::PutVlqInt(uint32_t v) {
- bool result = true;
- while ((v & 0xFFFFFF80UL) != 0UL) {
- result &= PutAligned<uint8_t>(static_cast<uint8_t>((v & 0x7F) | 0x80), 1);
- v >>= 7;
- }
- result &= PutAligned<uint8_t>(static_cast<uint8_t>(v & 0x7F), 1);
- return result;
-}
+template <typename Int>
+inline bool BitWriter::PutVlqInt(Int v) {
+ static_assert(std::is_integral_v<Int>);
-inline bool BitReader::GetVlqInt(uint32_t* v) {
- uint32_t tmp = 0;
+ constexpr auto kBufferSize = kMaxLEB128ByteLenFor<Int>;
- for (int i = 0; i < kMaxVlqByteLength; i++) {
- uint8_t byte = 0;
- if (ARROW_PREDICT_FALSE(!GetAligned<uint8_t>(1, &byte))) {
- return false;
- }
- tmp |= static_cast<uint32_t>(byte & 0x7F) << (7 * i);
+ uint8_t buffer[kBufferSize] = {};
+ const auto bytes_written = WriteLEB128(v, buffer, kBufferSize);
+ ARROW_DCHECK_LE(bytes_written, kBufferSize);
+ ARROW_DCHECK_GT(bytes_written, 0); // Cannot fail since we gave max space
- if ((byte & 0x80) == 0) {
- *v = tmp;
- return true;
+ for (int i = 0; i < bytes_written; ++i) {
+ const bool success = PutAligned(buffer[i], 1);
+ if (ARROW_PREDICT_FALSE(!success)) {
+ return false;
}
}
- return false;
-}
-
-inline bool BitWriter::PutZigZagVlqInt(int32_t v) {
- uint32_t u_v = ::arrow::util::SafeCopy<uint32_t>(v);
- u_v = (u_v << 1) ^ static_cast<uint32_t>(v >> 31);
- return PutVlqInt(u_v);
-}
-
-inline bool BitReader::GetZigZagVlqInt(int32_t* v) {
- uint32_t u;
- if (!GetVlqInt(&u)) return false;
- u = (u >> 1) ^ (~(u & 1) + 1);
- *v = ::arrow::util::SafeCopy<int32_t>(u);
return true;
}
-inline bool BitWriter::PutVlqInt(uint64_t v) {
- bool result = true;
- while ((v & 0xFFFFFFFFFFFFFF80ULL) != 0ULL) {
- result &= PutAligned<uint8_t>(static_cast<uint8_t>((v & 0x7F) | 0x80), 1);
- v >>= 7;
+template <typename Int>
+inline bool BitReader::GetVlqInt(Int* v) {
+ static_assert(std::is_integral_v<Int>);
Review Comment:
We can, but we don't have to. The code safely works for extracting a positive number into a signed integer type (if it fits, but that's true for all types).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2349601891
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ ARROW_DCHECK_LT(values_read, batch_size);
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+ auto const read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
return values_read;
}
-template <typename T, typename RunType, typename Converter>
-inline int RleDecoder::GetSpaced(Converter converter, int batch_size, int null_count,
- const uint8_t* valid_bits, int64_t valid_bits_offset,
- T* out) {
- if (ARROW_PREDICT_FALSE(null_count == batch_size)) {
- converter.FillZero(out, out + batch_size);
- return batch_size;
+namespace internal {
+
+/// Utility class to safely handle values and null count without too error-prone
+/// verbosity.
+class BatchCounter {
+ public:
+ using size_type = int32_t;
+
+ [[nodiscard]] static constexpr BatchCounter FromBatchSizeAndNulls(
+ size_type batch_size, size_type null_count) {
+ ARROW_DCHECK_LE(null_count, batch_size);
+ return {batch_size - null_count, null_count};
}
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
- int values_remaining = batch_size - null_count;
+ constexpr BatchCounter(size_type values_count, size_type null_count) noexcept
+ : values_count_(values_count), null_count_(null_count) {}
- // Assume no bits to start.
- arrow::internal::BitRunReader bit_reader(valid_bits, valid_bits_offset,
- /*length=*/batch_size);
- arrow::internal::BitRun valid_run = bit_reader.NextRun();
- while (values_read < batch_size) {
- if (ARROW_PREDICT_FALSE(valid_run.length == 0)) {
- valid_run = bit_reader.NextRun();
+ [[nodiscard]] constexpr size_type ValuesCount() const noexcept { return values_count_; }
+
+ [[nodiscard]] constexpr size_type ValuesRead() const noexcept { return values_read_; }
+
+ [[nodiscard]] constexpr size_type ValuesRemaining() const noexcept {
+ ARROW_DCHECK_LE(values_read_, values_count_);
+ return values_count_ - values_read_;
+ }
+
+ constexpr void AccrueReadValues(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, ValuesRemaining());
+ values_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type NullCount() const noexcept { return null_count_; }
+
+ [[nodiscard]] constexpr size_type NullRead() const noexcept { return null_read_; }
+
+ [[nodiscard]] constexpr size_type NullRemaining() const noexcept {
+ ARROW_DCHECK_LE(null_read_, null_count_);
+ return null_count_ - null_read_;
+ }
+
+ constexpr void AccrueReadNulls(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, NullRemaining());
+ null_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type TotalRemaining() const noexcept {
+ return ValuesRemaining() + NullRemaining();
+ }
+
+ [[nodiscard]] constexpr size_type TotalRead() const noexcept {
+ return values_read_ + null_read_;
+ }
+
+ [[nodiscard]] constexpr bool IsFullyNull() const noexcept {
+ return ValuesRemaining() == 0;
+ }
+
+ [[nodiscard]] constexpr bool IsDone() const noexcept { return TotalRemaining() == 0; }
+
+ private:
+ size_type values_count_ = 0;
+ size_type values_read_ = 0;
+ size_type null_count_ = 0;
+ size_type null_read_ = 0;
+};
+
+// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+/// Overload for GetSpaced for a single run in a RleDecoder
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ RleDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto values_remaining_run = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // Consume as much as possible from the repeated run.
+ // We only need to count the number of nulls and non-nulls because we can fill in the
+ // same value for nulls and non-nulls.
+ // This proves to be a big efficiency win.
+ while (values_remaining_run() > 0 && !batch.IsDone()) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const& validity_run_size = static_cast<values_count_type>(validity_run.length);
+
+ if (validity_run.set) {
+ // We may end the current RLE run in the middle of the validity run
+ auto update_size = std::min(validity_run_size, values_remaining_run());
+ batch.AccrueReadValues(update_size);
+ validity_run.length -= update_size;
+ } else {
+ // We can consume all nulls here because it does not matter if we consume on this
+ // RLE run, or an a next encoded run. The value filled does not matter.
+ auto update_size = std::min(validity_run_size, batch.NullRemaining());
+ batch.AccrueReadNulls(update_size);
+ validity_run.length -= update_size;
+ }
+
+ if (ARROW_PREDICT_TRUE(validity_run.length == 0)) {
+ validity_run = validity_reader.NextRun();
}
+ }
+
+ value_type const value = decoder.Value();
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(value))) {
+ return {0, 0};
+ }
+ converter.WriteRepeated(out, out + batch.TotalRead(), value);
+ auto const actual_values_read = decoder.Advance(batch.ValuesRead());
+ // We always cropped the number of values_read by the remaining values in the run.
+ // What's more the RLE decoder should not encounter any errors.
+ ARROW_DCHECK_EQ(actual_values_read, batch.ValuesRead());
- ARROW_DCHECK_GT(batch_size, 0);
- ARROW_DCHECK_GT(valid_run.length, 0);
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+template <typename T, typename... Ts>
+[[nodiscard]] constexpr T min(T x, Ts... ys) {
+ ((x = std::min(x, ys)), ...);
+ return x;
+}
+
+static_assert(min(5) == 5);
+static_assert(min(5, 4, -1) == -1);
+static_assert(min(5, 41) == 5);
+
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ BitPackedDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto run_values_remaining = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ while (run_values_remaining() > 0 && batch.ValuesRemaining() > 0) {
+ // TODO should this size be tune depending on sizeof(value_size)? cpu cache size?
+ // Pull a batch of values from the bit packed encoded data and store it in a local
+ // buffer to benefit from unpacking intrinsics and data locality.
+ static constexpr values_count_type kBufferCapacity = 1024;
+ std::array<value_type, kBufferCapacity> buffer = {};
+
+ values_count_type buffer_start = 0;
+ values_count_type buffer_end = 0;
+ auto buffer_size = [&]() {
+ auto out = buffer_end - buffer_start;
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // buffer_start is 0 at this point so size is end
+ buffer_end = min(run_values_remaining(), batch.ValuesRemaining(), kBufferCapacity);
+ buffer_end = decoder.GetBatch(buffer.data(), buffer_size());
+ ARROW_DCHECK_LE(buffer_size(), kBufferCapacity);
+
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(buffer.data(), buffer_size()))) {
+ return {batch.ValuesRead(), batch.NullRead()};
+ }
- if (valid_run.set) {
- if ((repeat_count_ == 0) && (literal_count_ == 0)) {
- if (!NextCounts<RunType>()) return values_read;
- ARROW_DCHECK((repeat_count_ > 0) ^ (literal_count_ > 0));
+ // Copy chunks of valid values into the output, while adjusting spacing for null
+ // values.
+ while (buffer_size() > 0) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const validity_run_length =
+ static_cast<values_count_type>(validity_run.length);
+
+ // Copy as much as possible from the buffer into the output while not exceeding
+ // validity run
+ if (validity_run.set) {
+ auto const update_size = std::min(validity_run_length, buffer_size());
+ converter.WriteRange(out, buffer.data() + buffer_start, update_size);
+ buffer_start += update_size;
+ batch.AccrueReadValues(update_size);
+ out += update_size;
+ validity_run.length -= update_size;
+ // Simply write zeros in the output
+ } else {
+ auto const update_size = std::min(validity_run_length, batch.NullRemaining());
+ converter.WriteZero(out, out + update_size);
+ batch.AccrueReadNulls(update_size);
+ out += update_size;
+ validity_run.length -= update_size;
}
- if (repeat_count_ > 0) {
- int repeat_batch = 0;
- // Consume the entire repeat counts incrementing repeat_batch to
- // be the total of nulls + values consumed, we only need to
- // get the total count because we can fill in the same value for
- // nulls and non-nulls. This proves to be a big efficiency win.
- while (repeat_count_ > 0 && (values_read + repeat_batch) < batch_size) {
- ARROW_DCHECK_GT(valid_run.length, 0);
- if (valid_run.set) {
- int update_size = std::min(static_cast<int>(valid_run.length), repeat_count_);
- repeat_count_ -= update_size;
- repeat_batch += update_size;
- valid_run.length -= update_size;
- values_remaining -= update_size;
- } else {
- // We can consume all nulls here because we would do so on
- // the next loop anyways.
- repeat_batch += static_cast<int>(valid_run.length);
- valid_run.length = 0;
- }
- if (valid_run.length == 0) {
- valid_run = bit_reader.NextRun();
- }
- }
- RunType current_value = static_cast<RunType>(current_value_);
- if (ARROW_PREDICT_FALSE(!converter.IsValid(current_value))) {
- return values_read;
- }
- converter.Fill(out, out + repeat_batch, current_value);
- out += repeat_batch;
- values_read += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(values_remaining, literal_count_);
- ARROW_DCHECK_GT(literal_batch, 0);
-
- // Decode the literals
- constexpr int kBufferSize = 1024;
- RunType indices[kBufferSize];
- literal_batch = std::min(literal_batch, kBufferSize);
- int actual_read = bit_reader_.GetBatch(bit_width_, indices, literal_batch);
- if (ARROW_PREDICT_FALSE(actual_read != literal_batch)) {
- return values_read;
- }
- if (!converter.IsValid(indices, /*length=*/actual_read)) {
- return values_read;
- }
- int skipped = 0;
- int literals_read = 0;
- while (literals_read < literal_batch) {
- if (valid_run.set) {
- int update_size = std::min(literal_batch - literals_read,
- static_cast<int>(valid_run.length));
- converter.Copy(out, indices + literals_read, update_size);
- literals_read += update_size;
- out += update_size;
- valid_run.length -= update_size;
- } else {
- converter.FillZero(out, out + valid_run.length);
- out += valid_run.length;
- skipped += static_cast<int>(valid_run.length);
- valid_run.length = 0;
- }
- if (valid_run.length == 0) {
- valid_run = bit_reader.NextRun();
- }
- }
- literal_count_ -= literal_batch;
- values_remaining -= literal_batch;
- values_read += literal_batch + skipped;
+ if (validity_run.length == 0) {
+ validity_run = validity_reader.NextRun();
}
- } else {
- converter.FillZero(out, out + valid_run.length);
- out += valid_run.length;
- values_read += static_cast<int>(valid_run.length);
- valid_run.length = 0;
}
+
+ ARROW_DCHECK_EQ(buffer_size(), 0);
}
- ARROW_DCHECK_EQ(valid_run.length, 0);
- ARROW_DCHECK_EQ(values_remaining, 0);
- return values_read;
+
+ ARROW_DCHECK_EQ(values_available - decoder.Remaining(), batch.ValuesRead());
+ ARROW_DCHECK_LE(batch.TotalRead(), batch_size);
+ ARROW_DCHECK_LE(batch.NullRead(), batch.NullCount());
+
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+/// Overload for GetSpaced for a single run in a decoder variant
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(
+ Converter& converter, typename Converter::out_type* out, values_count_type batch_size,
+ values_count_type null_count, BitRunReader&& validity_reader, BitRun&& validity_run,
+ std::variant<RleDecoder<value_type>, BitPackedDecoder<value_type>>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ return std::visit(
+ [&](auto& dec) {
+ ARROW_DCHECK_GT(dec.Remaining(), 0);
+ return RunGetSpaced(converter, out, batch_size, null_count, validity_reader,
+ validity_run, dec);
+ },
+ decoder);
}
+} // namespace internal
+
+template <typename T>
+template <typename Converter>
+auto RleBitPackedDecoder<T>::GetSpaced(
+ Converter converter, typename Converter::out_type* out, values_count_type batch_size,
+ const byte* validity_bits, int64_t validity_bits_offset, values_count_type null_count)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
+
+ ARROW_DCHECK_GT(batch_size, 0);
+
+ auto batch = internal::BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ if (ARROW_PREDICT_FALSE(batch.IsFullyNull())) {
+ converter.WriteZero(out, out + batch.NullRemaining());
+ return batch.NullRemaining();
+ }
+
+ arrow::internal::BitRunReader validity_reader(validity_bits, validity_bits_offset,
+ /*length=*/batch.TotalRemaining());
+ arrow::internal::BitRun validity_run = validity_reader.NextRun();
+
+ auto const check_and_handle_fully_null_remaining = [&]() {
+ if (batch.IsFullyNull()) {
+ ARROW_DCHECK(validity_run.length == 0 || !validity_run.set);
+ ARROW_DCHECK_GE(validity_run.length, batch.NullRemaining());
+
+ converter.WriteZero(out, out + batch.NullRemaining());
+ out += batch.NullRemaining();
+ batch.AccrueReadNulls(batch.NullRemaining());
+ }
+ };
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const [values_read, null_read] =
+ RunGetSpaced(converter, out, batch.TotalRemaining(), batch.NullRemaining(),
+ validity_reader, validity_run, decoder_);
+
+ batch.AccrueReadNulls(null_read);
+ batch.AccrueReadValues(values_read);
+ out += values_read + null_read;
+
+ // Either we fulfilled all the batch values to be read
+ if (ARROW_PREDICT_FALSE(batch.ValuesRemaining() == 0)) {
+ // There may be remaining null if they are not greedily filled
+ check_and_handle_fully_null_remaining();
+ return batch.TotalRead();
+ }
+
+ /// We finished the remaining run
+ ARROW_DCHECK(RunRemaining() == 0);
+ }
+
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+
+ auto const [values_read, null_read] = internal::RunGetSpaced(
+ converter, out, batch.TotalRemaining(), batch.NullRemaining(),
+ validity_reader, validity_run, decoder);
+
+ batch.AccrueReadNulls(null_read);
+ batch.AccrueReadValues(values_read);
+ out += values_read + null_read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == 0 || batch.ValuesRemaining() == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
+ // There may be remaining null if they are not greedily filled by either decoder calls
+ check_and_handle_fully_null_remaining();
+
+ ARROW_DCHECK(batch.IsDone() || Exhausted());
+ // batch.Done() => batch.NullRemaining() == 0
+ ARROW_DCHECK(!batch.IsDone() || (batch.NullRemaining() == 0));
+ return batch.TotalRead();
+}
+
+namespace internal {
+
// Converter for GetSpaced that handles runs that get returned
// directly as output.
template <typename T>
-struct PlainRleConverter {
- T kZero = {};
- inline bool IsValid(const T& values) const { return true; }
- inline bool IsValid(const T* values, int32_t length) const { return true; }
- inline void Fill(T* begin, T* end, const T& run_value) const {
+struct NoOpConverter {
+ using in_type = T;
+ using out_type = T;
+ using size_type = int32_t;
+
+ static constexpr bool kIsIdentity = true;
+
+ [[nodiscard]] static constexpr bool InputIsValid(const in_type& values) { return true; }
+
+ [[nodiscard]] static constexpr bool InputIsValid(const in_type* values,
+ size_type length) {
+ return true;
+ }
+
+ static void WriteRepeated(out_type* begin, out_type* end, in_type run_value) {
std::fill(begin, end, run_value);
}
- inline void FillZero(T* begin, T* end) { std::fill(begin, end, kZero); }
- inline void Copy(T* out, const T* values, int length) const {
- std::memcpy(out, values, length * sizeof(T));
+
+ static void WriteZero(out_type* begin, out_type* end) {
+ std::fill(begin, end, out_type{});
+ }
+
+ static void WriteRange(out_type* out, const in_type* values, size_type length) {
+ std::memcpy(out, values, length * sizeof(out_type));
}
};
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatchSpaced(int batch_size, int null_count,
- const uint8_t* valid_bits,
- int64_t valid_bits_offset, T* out) {
+auto RleBitPackedDecoder<T>::GetBatchSpaced(values_count_type batch_size,
+ values_count_type null_count,
+ const byte* valid_bits,
+ int64_t valid_bits_offset, value_type* out)
+ -> values_count_type {
if (null_count == 0) {
- return GetBatch<T>(out, batch_size);
+ return GetBatch(out, batch_size);
}
- PlainRleConverter<T> converter;
- arrow::internal::BitBlockCounter block_counter(valid_bits, valid_bits_offset,
- batch_size);
-
- int total_processed = 0;
- int processed = 0;
- arrow::internal::BitBlockCount block;
+ internal::NoOpConverter<value_type> converter;
- do {
- block = block_counter.NextFourWords();
- if (block.length == 0) {
- break;
- }
- if (block.AllSet()) {
- processed = GetBatch<T>(out, block.length);
- } else if (block.NoneSet()) {
- converter.FillZero(out, out + block.length);
- processed = block.length;
- } else {
- processed = GetSpaced<T, /*RunType=*/T, PlainRleConverter<T>>(
- converter, block.length, block.length - block.popcount, valid_bits,
- valid_bits_offset, out);
- }
- total_processed += processed;
- out += block.length;
- valid_bits_offset += block.length;
- } while (processed == block.length);
- return total_processed;
+ return GetSpaced(converter, out, batch_size, valid_bits, valid_bits_offset, null_count);
}
-static inline bool IndexInRange(int32_t idx, int32_t dictionary_length) {
- return idx >= 0 && idx < dictionary_length;
+namespace internal {
+
+template <typename I>
+bool IndexInRange(I idx, int32_t dictionary_length) {
+ ARROW_DCHECK_GT(dictionary_length, 0);
+ using T = std::common_type_t<decltype(idx), decltype(dictionary_length)>;
+ return idx >= 0 && static_cast<T>(idx) < static_cast<T>(dictionary_length);
}
// Converter for GetSpaced that handles runs of returned dictionary
// indices.
-template <typename T>
+template <typename V, typename I>
struct DictionaryConverter {
- T kZero = {};
- const T* dictionary;
- int32_t dictionary_length;
-
- inline bool IsValid(int32_t value) { return IndexInRange(value, dictionary_length); }
-
- inline bool IsValid(const int32_t* values, int32_t length) const {
- using IndexType = int32_t;
- IndexType min_index = std::numeric_limits<IndexType>::max();
- IndexType max_index = std::numeric_limits<IndexType>::min();
- for (int x = 0; x < length; x++) {
- min_index = std::min(values[x], min_index);
- max_index = std::max(values[x], max_index);
+ using out_type = V;
+ using in_type = I;
+ using size_type = int32_t;
+
+ static constexpr bool kIsIdentity = false;
+
+ const out_type* dictionary;
+ size_type dictionary_length;
+
+ [[nodiscard]] bool InputIsValid(in_type idx) const {
+ return IndexInRange(idx, dictionary_length);
+ }
+
+ [[nodiscard]] bool InputIsValid(const in_type* indices, size_type length) const {
Review Comment:
I'd say it work even for `zero` since `std::numeric_limits<int32_t>::min()` will never be in range.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2349572624
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
Review Comment:
This one probably does not have much impact.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2354994158
##########
cpp/src/arrow/util/bit_util_test.cc:
##########
@@ -1997,11 +1997,191 @@ TEST(BitUtil, RoundUpToPowerOf2) {
#undef U64
#undef S64
+/// Test the maximum number of bytes needed to write a LEB128 of a give size.
+TEST(LEB128, MaxLEB128ByteLenFor) {
+ EXPECT_EQ(bit_util::kMaxLEB128ByteLenFor<int16_t>, 3);
+ EXPECT_EQ(bit_util::kMaxLEB128ByteLenFor<int32_t>, 5);
+ EXPECT_EQ(bit_util::kMaxLEB128ByteLenFor<int64_t>, 10);
+}
+
+/// Utility function to test LEB128 encoding with known input value and expected byte
+/// array
+template <typename Int>
+void TestLEB128Encode(Int input_value, const std::vector<uint8_t>& expected_data,
+ std::size_t buffer_size) {
+ std::vector<uint8_t> buffer(buffer_size);
+ auto bytes_written = bit_util::WriteLEB128(input_value, buffer.data(),
+ static_cast<int32_t>(buffer.size()));
+
+ EXPECT_EQ(bytes_written, expected_data.size());
+ // Encoded data
+ for (std::size_t i = 0; i < expected_data.size(); ++i) {
+ EXPECT_EQ(buffer.at(i), expected_data.at(i));
+ }
+
+ // When the value is successfully encoded, the remaining of the buffer is untouched
+ if (bytes_written > 0) {
+ for (std::size_t i = bytes_written; i < buffer.size(); ++i) {
+ EXPECT_EQ(buffer.at(i), 0);
+ }
+ }
+}
+
+/// Test encoding to known LEB128 byte sequences with edge cases parameters.
+/// \see LEB128.KnownSuccessfulValues for other known values tested.
+TEST(LEB128, WriteEdgeCases) {
+ // Single byte value 0
+ TestLEB128Encode(0U, {0x00}, 1);
+ // Single byte value 127
+ TestLEB128Encode(127U, {0x7F}, 1);
+ // Three byte value 16384, encoded in larger buffer
+ TestLEB128Encode(16384U, {0x80, 0x80, 0x01}, 10);
+ // Two byte boundary values
+ TestLEB128Encode(129U, {0x81, 0x01}, 2);
+ TestLEB128Encode(16383U, {0xFF, 0x7F}, 2);
+ // Error case: Buffer too small for value 128 (needs 2 bytes but only 1 provided)
+ TestLEB128Encode(128U, {}, 1);
+ // Error case: Buffer too small for uint32_t max (needs 5 bytes but only 4 provided)
+ TestLEB128Encode(4294967295U, {}, 4);
Review Comment:
We have them in `LEB128.KnownSuccessfulValues`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2355593962
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ ARROW_DCHECK_LT(values_read, batch_size);
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+ auto const read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
return values_read;
}
-template <typename T, typename RunType, typename Converter>
-inline int RleDecoder::GetSpaced(Converter converter, int batch_size, int null_count,
- const uint8_t* valid_bits, int64_t valid_bits_offset,
- T* out) {
- if (ARROW_PREDICT_FALSE(null_count == batch_size)) {
- converter.FillZero(out, out + batch_size);
- return batch_size;
+namespace internal {
+
+/// Utility class to safely handle values and null count without too error-prone
+/// verbosity.
+class BatchCounter {
+ public:
+ using size_type = int32_t;
+
+ [[nodiscard]] static constexpr BatchCounter FromBatchSizeAndNulls(
+ size_type batch_size, size_type null_count) {
+ ARROW_DCHECK_LE(null_count, batch_size);
+ return {batch_size - null_count, null_count};
}
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
- int values_remaining = batch_size - null_count;
+ constexpr BatchCounter(size_type values_count, size_type null_count) noexcept
+ : values_count_(values_count), null_count_(null_count) {}
- // Assume no bits to start.
- arrow::internal::BitRunReader bit_reader(valid_bits, valid_bits_offset,
- /*length=*/batch_size);
- arrow::internal::BitRun valid_run = bit_reader.NextRun();
- while (values_read < batch_size) {
- if (ARROW_PREDICT_FALSE(valid_run.length == 0)) {
- valid_run = bit_reader.NextRun();
+ [[nodiscard]] constexpr size_type ValuesCount() const noexcept { return values_count_; }
+
+ [[nodiscard]] constexpr size_type ValuesRead() const noexcept { return values_read_; }
+
+ [[nodiscard]] constexpr size_type ValuesRemaining() const noexcept {
+ ARROW_DCHECK_LE(values_read_, values_count_);
+ return values_count_ - values_read_;
+ }
+
+ constexpr void AccrueReadValues(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, ValuesRemaining());
+ values_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type NullCount() const noexcept { return null_count_; }
+
+ [[nodiscard]] constexpr size_type NullRead() const noexcept { return null_read_; }
+
+ [[nodiscard]] constexpr size_type NullRemaining() const noexcept {
+ ARROW_DCHECK_LE(null_read_, null_count_);
+ return null_count_ - null_read_;
+ }
+
+ constexpr void AccrueReadNulls(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, NullRemaining());
+ null_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type TotalRemaining() const noexcept {
+ return ValuesRemaining() + NullRemaining();
+ }
+
+ [[nodiscard]] constexpr size_type TotalRead() const noexcept {
+ return values_read_ + null_read_;
+ }
+
+ [[nodiscard]] constexpr bool IsFullyNull() const noexcept {
+ return ValuesRemaining() == 0;
+ }
+
+ [[nodiscard]] constexpr bool IsDone() const noexcept { return TotalRemaining() == 0; }
+
+ private:
+ size_type values_count_ = 0;
+ size_type values_read_ = 0;
+ size_type null_count_ = 0;
+ size_type null_read_ = 0;
+};
+
+// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+/// Overload for GetSpaced for a single run in a RleDecoder
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ RleDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto values_remaining_run = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // Consume as much as possible from the repeated run.
+ // We only need to count the number of nulls and non-nulls because we can fill in the
+ // same value for nulls and non-nulls.
+ // This proves to be a big efficiency win.
+ while (values_remaining_run() > 0 && !batch.IsDone()) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const& validity_run_size = static_cast<values_count_type>(validity_run.length);
+
+ if (validity_run.set) {
+ // We may end the current RLE run in the middle of the validity run
+ auto update_size = std::min(validity_run_size, values_remaining_run());
+ batch.AccrueReadValues(update_size);
+ validity_run.length -= update_size;
+ } else {
+ // We can consume all nulls here because it does not matter if we consume on this
+ // RLE run, or an a next encoded run. The value filled does not matter.
+ auto update_size = std::min(validity_run_size, batch.NullRemaining());
+ batch.AccrueReadNulls(update_size);
+ validity_run.length -= update_size;
+ }
+
+ if (ARROW_PREDICT_TRUE(validity_run.length == 0)) {
+ validity_run = validity_reader.NextRun();
}
+ }
+
+ value_type const value = decoder.Value();
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(value))) {
+ return {0, 0};
+ }
+ converter.WriteRepeated(out, out + batch.TotalRead(), value);
+ auto const actual_values_read = decoder.Advance(batch.ValuesRead());
+ // We always cropped the number of values_read by the remaining values in the run.
+ // What's more the RLE decoder should not encounter any errors.
+ ARROW_DCHECK_EQ(actual_values_read, batch.ValuesRead());
- ARROW_DCHECK_GT(batch_size, 0);
- ARROW_DCHECK_GT(valid_run.length, 0);
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+template <typename T, typename... Ts>
+[[nodiscard]] constexpr T min(T x, Ts... ys) {
+ ((x = std::min(x, ys)), ...);
+ return x;
+}
+
+static_assert(min(5) == 5);
+static_assert(min(5, 4, -1) == -1);
+static_assert(min(5, 41) == 5);
+
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ BitPackedDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto run_values_remaining = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ while (run_values_remaining() > 0 && batch.ValuesRemaining() > 0) {
+ // TODO should this size be tune depending on sizeof(value_size)? cpu cache size?
Review Comment:
Under (quick) investigation for sizes 256, 512, 1024, 2048.
Edit: seems like 1024 values was a good pick.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2368350251
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +612,663 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/************************
+ * RleBitPackedParser *
+ ************************/
+
+template <typename Handler>
+void RleBitPackedParser::Parse(Handler&& handler) {
+ while (!exhausted()) {
+ auto [read, control] = PeekImpl(handler);
+ data_ += read;
+ data_size_ -= read;
+ if (ARROW_PREDICT_FALSE(control == ControlFlow::Break)) {
+ break;
+ }
+ }
+}
+
+namespace internal {
+/// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+} // namespace internal
+
+template <typename Handler>
+auto RleBitPackedParser::PeekImpl(Handler&& handler) const
+ -> std::pair<rle_size_t, ControlFlow> {
+ ARROW_DCHECK(!exhausted());
+
+ constexpr auto kMaxSize = bit_util::kMaxLEB128ByteLenFor<uint32_t>;
+ uint32_t run_len_type = 0;
+ const auto header_bytes = bit_util::ParseLeadingLEB128(data_, kMaxSize, &run_len_type);
+
+ if (ARROW_PREDICT_FALSE(header_bytes == 0)) {
+ // Malfomrmed LEB128 data
+ return {};
+ }
+
+ const bool is_bit_packed = run_len_type & 1;
+ const uint32_t count = run_len_type >> 1;
+ if (is_bit_packed) {
+ constexpr auto kMaxCount = bit_util::CeilDiv(internal::max_size_for_v<rle_size_t>, 8);
+ if (ARROW_PREDICT_FALSE(count == 0 || count > kMaxCount)) {
+ // Illegal number of encoded values
+ return {0, ControlFlow::Break};
+ }
+
+ const auto values_count = static_cast<rle_size_t>(count * 8);
+ ARROW_DCHECK_LT(count, internal::max_size_for_v<rle_size_t>);
+ // Count Already divided by 8
+ const auto bytes_read =
+ header_bytes + static_cast<rle_size_t>(count) * value_bit_width_;
+
+ auto control = handler.OnBitPackedRun(
+ BitPackedRun(data_ + header_bytes, values_count, value_bit_width_));
+
+ return {bytes_read, control};
+ }
+
+ if (ARROW_PREDICT_FALSE(
+ count == 0 ||
+ count > static_cast<uint32_t>(std::numeric_limits<rle_size_t>::max()))) {
+ // Illegal number of encoded values
+ return {0, ControlFlow::Break};
+ }
+
+ const auto values_count = static_cast<rle_size_t>(count);
+ const auto value_bytes = bit_util::BytesForBits(value_bit_width_);
+ ARROW_DCHECK_LT(value_bytes, internal::max_size_for_v<rle_size_t>);
+ const auto bytes_read = header_bytes + static_cast<rle_size_t>(value_bytes);
+
+ auto control =
+ handler.OnRleRun(RleRun(data_ + header_bytes, values_count, value_bit_width_));
+
+ return {bytes_read, control};
+}
+
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+template <typename Callable>
+void RleBitPackedDecoder<T>::ParseWithCallable(Callable&& func) {
+ struct {
+ Callable func;
+ auto OnBitPackedRun(BitPackedRun run) { return func(std::move(run)); }
+ auto OnRleRun(RleRun run) { return func(std::move(run)); }
+ } handler{std::move(func)};
+
+ parser_.Parse(std::move(handler));
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, rle_size_t batch_size)
+ -> rle_size_t {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ rle_size_t values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(run_remaining() > 0)) {
+ const auto read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(run_remaining() == 0);
}
+ ParseWithCallable([&](auto run) {
+ using RunDecoder = typename decltype(run)::template DecoderType<value_type>;
+
+ ARROW_DCHECK_LT(values_read, batch_size);
+ RunDecoder decoder(run);
+ const auto read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
Review Comment:
Ha, then this is for checking errors and/or actually when exactly 0 values are read (to avoid loop).
This should never happen here with the `RleDecoder` which is quite simple, however I don't have currently enough knowledge of the `BitReader` (used in the `BitPackedDecoder`) to confidently say it won't return 0. In particular the unpack functions can only can only process certain length and I don't know how that is checked.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2322047570
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -207,12 +209,310 @@ TEST(BitUtil, RoundTripIntValues) {
}
}
+/// A Rle run is a simple class owning some data and a repetition count.
+/// It does not know how to read such data.
+TEST(Rle, RleRun) {
+ const std::array<RleRun::byte, 4> value = {21, 2, 0, 0};
+
+ RleRun::values_count_type value_count = 12;
+
+ // 12 times the value 21 fitting over 5 bits
+ auto const run_5 = RleRun(value.data(), value_count, /* value_bit_width= */ 5);
+ EXPECT_EQ(run_5.ValuesCount(), value_count);
+ EXPECT_EQ(run_5.ValuesBitWidth(), 5);
+ EXPECT_EQ(run_5.RawDataSize(), 1); // 5 bits fit in one byte
+ EXPECT_EQ(*run_5.RawDataPtr(), 21);
+
+ // 12 times the value 21 fitting over 16 bits
+ auto const run_8 = RleRun(value.data(), value_count, /* value_bit_width= */ 8);
+ EXPECT_EQ(run_8.ValuesCount(), value_count);
+ EXPECT_EQ(run_8.ValuesBitWidth(), 8);
+ EXPECT_EQ(run_8.RawDataSize(), 1); // 8 bits fit in 1 byte
+ EXPECT_EQ(*run_8.RawDataPtr(), 21);
+
+ // 12 times the value {21, 2} fitting over 10 bits
+ auto const run_10 = RleRun(value.data(), value_count, /* value_bit_width= */ 10);
+
+ EXPECT_EQ(run_10.ValuesCount(), value_count);
+ EXPECT_EQ(run_10.ValuesBitWidth(), 10);
+ EXPECT_EQ(run_10.RawDataSize(), 2); // 10 bits fit in 2 bytes
+ EXPECT_EQ(*(run_10.RawDataPtr() + 0), 21);
+ EXPECT_EQ(*(run_10.RawDataPtr() + 1), 2);
+
+ // 12 times the value {21, 2} fitting over 32 bits
+ auto const run_32 = RleRun(value.data(), value_count, /* value_bit_width= */ 32);
+ EXPECT_EQ(run_32.ValuesCount(), value_count);
+ EXPECT_EQ(run_32.ValuesBitWidth(), 32);
+ EXPECT_EQ(run_32.RawDataSize(), 4); // 32 bits fit in 4 bytes
+ EXPECT_EQ(*(run_32.RawDataPtr() + 0), 21);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 1), 2);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 2), 0);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 3), 0);
+}
+
+/// A BitPacked run is a simple class owning some data and its size.
+/// It does not know how to read such data.
+TEST(BitPacked, BitPackedRun) {
+ const std::array<BitPackedRun::byte, 4> value = {0b10101010, 0, 0, 0b1111111};
+
+ /// 16 values of 1 bit for a total of 16 bits
+ BitPackedRun::values_count_type value_count_1 = 16;
+ auto const run_1 = BitPackedRun(value.data(), value_count_1, /* value_bit_width= */ 1);
+ EXPECT_EQ(run_1.ValuesCount(), value_count_1);
+ EXPECT_EQ(run_1.ValuesBitWidth(), 1);
+ EXPECT_EQ(run_1.RawDataSize(), 2); // 16 bits fit in 2 bytes
+ for (BitPackedRun::raw_data_size_type i = 0; i < run_1.RawDataSize(); ++i) {
+ EXPECT_EQ(*(run_1.RawDataPtr() + i), value[i]);
+ }
+
+ /// 8 values of 3 bits for a total of 24 bits
+ BitPackedRun::values_count_type value_count_3 = 8;
+ auto const run_3 = BitPackedRun(value.data(), value_count_3, /* value_bit_width= */ 3);
+ EXPECT_EQ(run_3.ValuesCount(), value_count_3);
+ EXPECT_EQ(run_3.ValuesBitWidth(), 3);
+ EXPECT_EQ(run_3.RawDataSize(), 3); // 24 bits fit in 3 bytes
+ for (BitPackedRun::raw_data_size_type i = 0; i < run_3.RawDataSize(); ++i) {
+ EXPECT_EQ(*(run_3.RawDataPtr() + i), value[i]);
+ }
+}
+
+template <typename T>
+void TestRleDecoder(std::vector<RleRun::byte> bytes,
+ RleRun::values_count_type value_count,
+ RleRun::bit_size_type bit_width) {
+ // Pre-requisite for this test
+ EXPECT_GT(value_count, 6);
+
+ // Compute value associated with bytes encoded as little endian
Review Comment:
Instead of computing it, how about passing it as an argument to this function? It would feel safer from a testing point of view.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2322275322
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -602,20 +1033,106 @@ struct GetBatchSpacedTestCase {
int bit_width;
};
-TEST(RleDecoder, GetBatchSpaced) {
- uint32_t kSeed = 1337;
- ::arrow::random::RandomArrayGenerator rand(kSeed);
-
- std::vector<GetBatchSpacedTestCase<int32_t>> int32_cases{
- {1, 100000, 0.01, 1}, {1, 100000, 0.1, 1}, {1, 100000, 0.5, 1},
- {4, 100000, 0.05, 3}, {100, 100000, 0.05, 7},
+template <typename T>
+void DoTestGetBatchSpacedRoundtrip() {
+ using Data = DataTestRleBitPacked<T>;
+ using ArrowType = typename Data::ArrowType;
+ using RandomPart = typename Data::RandomPart;
+ using NullPart = typename Data::NullPart;
+ using RepeatPart = typename Data::RepeatPart;
+
+ std::vector<Data> test_cases = {
+ {
+ {RandomPart{/* max=*/1, /* size=*/400, /* null_proba= */ 0.1}},
+ /* bit_width= */ 1,
+ },
+ {
+ {
+ RandomPart{/* max=*/7, /* size=*/10037, /* null_proba= */ 0.1},
+ NullPart{/* size= */ 1153},
+ RandomPart{/* max=*/7, /* size=*/800, /* null_proba= */ 0.5},
+ },
+ /* bit_width= */ 3,
+ },
+ {
+ {
+ NullPart{/* size= */ 80},
+ RandomPart{/* max=*/1023, /* size=*/800, /* null_proba= */ 0.01},
+ NullPart{/* size= */ 1023},
+ },
+ /* bit_width= */ 11,
+ },
+ {
+ {RepeatPart{/* value=*/13, /* size=*/100000}},
+ /* bit_width= */ 10,
+ },
+ {
+ {
+ NullPart{/* size= */ 1024},
+ RepeatPart{/* value=*/10000, /* size=*/100000},
+ NullPart{/* size= */ 77},
+ },
+ /* bit_width= */ 23,
+ },
+ {
+ {
+ RepeatPart{/* value=*/13, /* size=*/100000},
+ NullPart{/* size= */ 1153},
+ RepeatPart{/* value=*/72, /* size=*/100799},
+ },
+ /* bit_width= */ 10,
+ },
+ {
+ {
+ RandomPart{/* max=*/1, /* size=*/1013, /* null_proba= */ 0.01},
+ NullPart{/* size=*/8},
+ RepeatPart{1, /* size= */ 256},
+ NullPart{/* size=*/128},
+ RepeatPart{0, /* size= */ 256},
+ NullPart{/* size=*/15},
+ RandomPart{/* max=*/1, /* size=*/8 * 1024, /* null_proba= */ 0.01},
+ },
+ /* bit_width= */ 1,
+ },
};
- for (auto case_ : int32_cases) {
- auto arr = rand.Int32(case_.size, /*min=*/0, case_.max_value, case_.null_probability);
- CheckRoundTripSpaced<Int32Type>(*arr, case_.bit_width);
- CheckRoundTripSpaced<Int32Type>(*arr->Slice(1), case_.bit_width);
+
+ ::arrow::random::RandomArrayGenerator rand(/* seed= */ 12);
+ // FRAGILE: Large enough so that it can be indexed by any value in all cases
+ auto dict = std::static_pointer_cast<arrow::FloatArray>(rand.Float32(20000, -1.0, 1.0));
+
+ for (auto case_ : test_cases) {
+ if (static_cast<std::size_t>(case_.bit_width) > sizeof(T)) {
+ continue;
+ }
+
+ auto array = case_.MakeArray(rand);
+
+ // Tests for GetBatch
+ CheckRoundTrip<ArrowType, false, 1>(*array, case_.bit_width);
+ CheckRoundTrip<ArrowType, false, 3>(*array, case_.bit_width);
+
+ // Tests for GetBatchSpaced
+ CheckRoundTrip<ArrowType, true, 1>(*array, case_.bit_width);
+ CheckRoundTrip<ArrowType, true, 7>(*array, case_.bit_width);
+ CheckRoundTrip<ArrowType, true, 1>(*array->Slice(1), case_.bit_width);
+
+ // Cannot test GetBatchWithDict with this method since unknown null values
+
+ // Tests for GetBatchWithDictSpaced
+ CheckRoundTrip<ArrowType, true, 1>(*array, case_.bit_width, dict);
+ CheckRoundTrip<ArrowType, true, 5>(*array, case_.bit_width, dict);
}
}
+TEST(RleBitPacked, GetBatchSpacedRoundtripUint16) {
+ DoTestGetBatchSpacedRoundtrip<uint16_t>();
Review Comment:
We should also test `uint8_t`. Also, do we want to test signed integers or is that not supported by the RleBitPackedDecoder?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2319450787
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ ARROW_DCHECK_LT(values_read, batch_size);
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+ auto const read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
Review Comment:
Not sure about `ARROW_PREDICT_FALSE` either here. Its probability might depend on RLE stream and batching patterns in the upper layers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2319485372
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ ARROW_DCHECK_LT(values_read, batch_size);
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+ auto const read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
return values_read;
}
-template <typename T, typename RunType, typename Converter>
-inline int RleDecoder::GetSpaced(Converter converter, int batch_size, int null_count,
- const uint8_t* valid_bits, int64_t valid_bits_offset,
- T* out) {
- if (ARROW_PREDICT_FALSE(null_count == batch_size)) {
- converter.FillZero(out, out + batch_size);
- return batch_size;
+namespace internal {
+
+/// Utility class to safely handle values and null count without too error-prone
+/// verbosity.
+class BatchCounter {
+ public:
+ using size_type = int32_t;
+
+ [[nodiscard]] static constexpr BatchCounter FromBatchSizeAndNulls(
+ size_type batch_size, size_type null_count) {
+ ARROW_DCHECK_LE(null_count, batch_size);
+ return {batch_size - null_count, null_count};
}
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
- int values_remaining = batch_size - null_count;
+ constexpr BatchCounter(size_type values_count, size_type null_count) noexcept
+ : values_count_(values_count), null_count_(null_count) {}
- // Assume no bits to start.
- arrow::internal::BitRunReader bit_reader(valid_bits, valid_bits_offset,
- /*length=*/batch_size);
- arrow::internal::BitRun valid_run = bit_reader.NextRun();
- while (values_read < batch_size) {
- if (ARROW_PREDICT_FALSE(valid_run.length == 0)) {
- valid_run = bit_reader.NextRun();
+ [[nodiscard]] constexpr size_type ValuesCount() const noexcept { return values_count_; }
+
+ [[nodiscard]] constexpr size_type ValuesRead() const noexcept { return values_read_; }
+
+ [[nodiscard]] constexpr size_type ValuesRemaining() const noexcept {
+ ARROW_DCHECK_LE(values_read_, values_count_);
+ return values_count_ - values_read_;
+ }
+
+ constexpr void AccrueReadValues(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, ValuesRemaining());
+ values_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type NullCount() const noexcept { return null_count_; }
+
+ [[nodiscard]] constexpr size_type NullRead() const noexcept { return null_read_; }
+
+ [[nodiscard]] constexpr size_type NullRemaining() const noexcept {
+ ARROW_DCHECK_LE(null_read_, null_count_);
+ return null_count_ - null_read_;
+ }
+
+ constexpr void AccrueReadNulls(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, NullRemaining());
+ null_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type TotalRemaining() const noexcept {
+ return ValuesRemaining() + NullRemaining();
+ }
+
+ [[nodiscard]] constexpr size_type TotalRead() const noexcept {
+ return values_read_ + null_read_;
+ }
+
+ [[nodiscard]] constexpr bool IsFullyNull() const noexcept {
+ return ValuesRemaining() == 0;
+ }
+
+ [[nodiscard]] constexpr bool IsDone() const noexcept { return TotalRemaining() == 0; }
+
+ private:
+ size_type values_count_ = 0;
+ size_type values_read_ = 0;
+ size_type null_count_ = 0;
+ size_type null_read_ = 0;
+};
+
+// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+/// Overload for GetSpaced for a single run in a RleDecoder
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ RleDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto values_remaining_run = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // Consume as much as possible from the repeated run.
+ // We only need to count the number of nulls and non-nulls because we can fill in the
+ // same value for nulls and non-nulls.
+ // This proves to be a big efficiency win.
+ while (values_remaining_run() > 0 && !batch.IsDone()) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const& validity_run_size = static_cast<values_count_type>(validity_run.length);
+
+ if (validity_run.set) {
+ // We may end the current RLE run in the middle of the validity run
+ auto update_size = std::min(validity_run_size, values_remaining_run());
+ batch.AccrueReadValues(update_size);
+ validity_run.length -= update_size;
+ } else {
+ // We can consume all nulls here because it does not matter if we consume on this
+ // RLE run, or an a next encoded run. The value filled does not matter.
+ auto update_size = std::min(validity_run_size, batch.NullRemaining());
+ batch.AccrueReadNulls(update_size);
+ validity_run.length -= update_size;
+ }
+
+ if (ARROW_PREDICT_TRUE(validity_run.length == 0)) {
+ validity_run = validity_reader.NextRun();
}
+ }
+
+ value_type const value = decoder.Value();
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(value))) {
+ return {0, 0};
+ }
+ converter.WriteRepeated(out, out + batch.TotalRead(), value);
+ auto const actual_values_read = decoder.Advance(batch.ValuesRead());
+ // We always cropped the number of values_read by the remaining values in the run.
+ // What's more the RLE decoder should not encounter any errors.
+ ARROW_DCHECK_EQ(actual_values_read, batch.ValuesRead());
- ARROW_DCHECK_GT(batch_size, 0);
- ARROW_DCHECK_GT(valid_run.length, 0);
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+template <typename T, typename... Ts>
+[[nodiscard]] constexpr T min(T x, Ts... ys) {
+ ((x = std::min(x, ys)), ...);
+ return x;
+}
+
+static_assert(min(5) == 5);
+static_assert(min(5, 4, -1) == -1);
+static_assert(min(5, 41) == 5);
+
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ BitPackedDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto run_values_remaining = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ while (run_values_remaining() > 0 && batch.ValuesRemaining() > 0) {
+ // TODO should this size be tune depending on sizeof(value_size)? cpu cache size?
+ // Pull a batch of values from the bit packed encoded data and store it in a local
+ // buffer to benefit from unpacking intrinsics and data locality.
+ static constexpr values_count_type kBufferCapacity = 1024;
+ std::array<value_type, kBufferCapacity> buffer = {};
+
+ values_count_type buffer_start = 0;
+ values_count_type buffer_end = 0;
+ auto buffer_size = [&]() {
+ auto out = buffer_end - buffer_start;
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // buffer_start is 0 at this point so size is end
+ buffer_end = min(run_values_remaining(), batch.ValuesRemaining(), kBufferCapacity);
+ buffer_end = decoder.GetBatch(buffer.data(), buffer_size());
+ ARROW_DCHECK_LE(buffer_size(), kBufferCapacity);
+
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(buffer.data(), buffer_size()))) {
+ return {batch.ValuesRead(), batch.NullRead()};
+ }
- if (valid_run.set) {
- if ((repeat_count_ == 0) && (literal_count_ == 0)) {
- if (!NextCounts<RunType>()) return values_read;
- ARROW_DCHECK((repeat_count_ > 0) ^ (literal_count_ > 0));
+ // Copy chunks of valid values into the output, while adjusting spacing for null
+ // values.
+ while (buffer_size() > 0) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const validity_run_length =
+ static_cast<values_count_type>(validity_run.length);
+
+ // Copy as much as possible from the buffer into the output while not exceeding
+ // validity run
+ if (validity_run.set) {
+ auto const update_size = std::min(validity_run_length, buffer_size());
+ converter.WriteRange(out, buffer.data() + buffer_start, update_size);
+ buffer_start += update_size;
+ batch.AccrueReadValues(update_size);
+ out += update_size;
+ validity_run.length -= update_size;
+ // Simply write zeros in the output
+ } else {
+ auto const update_size = std::min(validity_run_length, batch.NullRemaining());
+ converter.WriteZero(out, out + update_size);
+ batch.AccrueReadNulls(update_size);
+ out += update_size;
+ validity_run.length -= update_size;
}
- if (repeat_count_ > 0) {
- int repeat_batch = 0;
- // Consume the entire repeat counts incrementing repeat_batch to
- // be the total of nulls + values consumed, we only need to
- // get the total count because we can fill in the same value for
- // nulls and non-nulls. This proves to be a big efficiency win.
- while (repeat_count_ > 0 && (values_read + repeat_batch) < batch_size) {
- ARROW_DCHECK_GT(valid_run.length, 0);
- if (valid_run.set) {
- int update_size = std::min(static_cast<int>(valid_run.length), repeat_count_);
- repeat_count_ -= update_size;
- repeat_batch += update_size;
- valid_run.length -= update_size;
- values_remaining -= update_size;
- } else {
- // We can consume all nulls here because we would do so on
- // the next loop anyways.
- repeat_batch += static_cast<int>(valid_run.length);
- valid_run.length = 0;
- }
- if (valid_run.length == 0) {
- valid_run = bit_reader.NextRun();
- }
- }
- RunType current_value = static_cast<RunType>(current_value_);
- if (ARROW_PREDICT_FALSE(!converter.IsValid(current_value))) {
- return values_read;
- }
- converter.Fill(out, out + repeat_batch, current_value);
- out += repeat_batch;
- values_read += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(values_remaining, literal_count_);
- ARROW_DCHECK_GT(literal_batch, 0);
-
- // Decode the literals
- constexpr int kBufferSize = 1024;
- RunType indices[kBufferSize];
- literal_batch = std::min(literal_batch, kBufferSize);
- int actual_read = bit_reader_.GetBatch(bit_width_, indices, literal_batch);
- if (ARROW_PREDICT_FALSE(actual_read != literal_batch)) {
- return values_read;
- }
- if (!converter.IsValid(indices, /*length=*/actual_read)) {
- return values_read;
- }
- int skipped = 0;
- int literals_read = 0;
- while (literals_read < literal_batch) {
- if (valid_run.set) {
- int update_size = std::min(literal_batch - literals_read,
- static_cast<int>(valid_run.length));
- converter.Copy(out, indices + literals_read, update_size);
- literals_read += update_size;
- out += update_size;
- valid_run.length -= update_size;
- } else {
- converter.FillZero(out, out + valid_run.length);
- out += valid_run.length;
- skipped += static_cast<int>(valid_run.length);
- valid_run.length = 0;
- }
- if (valid_run.length == 0) {
- valid_run = bit_reader.NextRun();
- }
- }
- literal_count_ -= literal_batch;
- values_remaining -= literal_batch;
- values_read += literal_batch + skipped;
+ if (validity_run.length == 0) {
+ validity_run = validity_reader.NextRun();
}
- } else {
- converter.FillZero(out, out + valid_run.length);
- out += valid_run.length;
- values_read += static_cast<int>(valid_run.length);
- valid_run.length = 0;
}
+
+ ARROW_DCHECK_EQ(buffer_size(), 0);
}
- ARROW_DCHECK_EQ(valid_run.length, 0);
- ARROW_DCHECK_EQ(values_remaining, 0);
- return values_read;
+
+ ARROW_DCHECK_EQ(values_available - decoder.Remaining(), batch.ValuesRead());
+ ARROW_DCHECK_LE(batch.TotalRead(), batch_size);
+ ARROW_DCHECK_LE(batch.NullRead(), batch.NullCount());
+
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+/// Overload for GetSpaced for a single run in a decoder variant
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(
+ Converter& converter, typename Converter::out_type* out, values_count_type batch_size,
+ values_count_type null_count, BitRunReader&& validity_reader, BitRun&& validity_run,
+ std::variant<RleDecoder<value_type>, BitPackedDecoder<value_type>>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ return std::visit(
+ [&](auto& dec) {
+ ARROW_DCHECK_GT(dec.Remaining(), 0);
+ return RunGetSpaced(converter, out, batch_size, null_count, validity_reader,
+ validity_run, dec);
+ },
+ decoder);
}
+} // namespace internal
+
+template <typename T>
+template <typename Converter>
+auto RleBitPackedDecoder<T>::GetSpaced(
+ Converter converter, typename Converter::out_type* out, values_count_type batch_size,
+ const byte* validity_bits, int64_t validity_bits_offset, values_count_type null_count)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
+
+ ARROW_DCHECK_GT(batch_size, 0);
+
+ auto batch = internal::BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ if (ARROW_PREDICT_FALSE(batch.IsFullyNull())) {
+ converter.WriteZero(out, out + batch.NullRemaining());
+ return batch.NullRemaining();
+ }
+
+ arrow::internal::BitRunReader validity_reader(validity_bits, validity_bits_offset,
+ /*length=*/batch.TotalRemaining());
+ arrow::internal::BitRun validity_run = validity_reader.NextRun();
+
+ auto const check_and_handle_fully_null_remaining = [&]() {
+ if (batch.IsFullyNull()) {
+ ARROW_DCHECK(validity_run.length == 0 || !validity_run.set);
+ ARROW_DCHECK_GE(validity_run.length, batch.NullRemaining());
+
+ converter.WriteZero(out, out + batch.NullRemaining());
+ out += batch.NullRemaining();
+ batch.AccrueReadNulls(batch.NullRemaining());
+ }
+ };
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const [values_read, null_read] =
+ RunGetSpaced(converter, out, batch.TotalRemaining(), batch.NullRemaining(),
+ validity_reader, validity_run, decoder_);
+
+ batch.AccrueReadNulls(null_read);
+ batch.AccrueReadValues(values_read);
+ out += values_read + null_read;
+
+ // Either we fulfilled all the batch values to be read
+ if (ARROW_PREDICT_FALSE(batch.ValuesRemaining() == 0)) {
+ // There may be remaining null if they are not greedily filled
+ check_and_handle_fully_null_remaining();
+ return batch.TotalRead();
+ }
+
+ /// We finished the remaining run
+ ARROW_DCHECK(RunRemaining() == 0);
+ }
+
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+
+ auto const [values_read, null_read] = internal::RunGetSpaced(
+ converter, out, batch.TotalRemaining(), batch.NullRemaining(),
+ validity_reader, validity_run, decoder);
+
+ batch.AccrueReadNulls(null_read);
+ batch.AccrueReadValues(values_read);
+ out += values_read + null_read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == 0 || batch.ValuesRemaining() == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
+ // There may be remaining null if they are not greedily filled by either decoder calls
+ check_and_handle_fully_null_remaining();
+
+ ARROW_DCHECK(batch.IsDone() || Exhausted());
+ // batch.Done() => batch.NullRemaining() == 0
+ ARROW_DCHECK(!batch.IsDone() || (batch.NullRemaining() == 0));
+ return batch.TotalRead();
+}
+
+namespace internal {
+
// Converter for GetSpaced that handles runs that get returned
// directly as output.
template <typename T>
-struct PlainRleConverter {
- T kZero = {};
- inline bool IsValid(const T& values) const { return true; }
- inline bool IsValid(const T* values, int32_t length) const { return true; }
- inline void Fill(T* begin, T* end, const T& run_value) const {
+struct NoOpConverter {
+ using in_type = T;
+ using out_type = T;
+ using size_type = int32_t;
+
+ static constexpr bool kIsIdentity = true;
+
+ [[nodiscard]] static constexpr bool InputIsValid(const in_type& values) { return true; }
+
+ [[nodiscard]] static constexpr bool InputIsValid(const in_type* values,
+ size_type length) {
+ return true;
+ }
+
+ static void WriteRepeated(out_type* begin, out_type* end, in_type run_value) {
std::fill(begin, end, run_value);
}
- inline void FillZero(T* begin, T* end) { std::fill(begin, end, kZero); }
- inline void Copy(T* out, const T* values, int length) const {
- std::memcpy(out, values, length * sizeof(T));
+
+ static void WriteZero(out_type* begin, out_type* end) {
+ std::fill(begin, end, out_type{});
+ }
+
+ static void WriteRange(out_type* out, const in_type* values, size_type length) {
+ std::memcpy(out, values, length * sizeof(out_type));
}
};
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatchSpaced(int batch_size, int null_count,
- const uint8_t* valid_bits,
- int64_t valid_bits_offset, T* out) {
+auto RleBitPackedDecoder<T>::GetBatchSpaced(values_count_type batch_size,
+ values_count_type null_count,
+ const byte* valid_bits,
+ int64_t valid_bits_offset, value_type* out)
+ -> values_count_type {
if (null_count == 0) {
- return GetBatch<T>(out, batch_size);
+ return GetBatch(out, batch_size);
}
- PlainRleConverter<T> converter;
- arrow::internal::BitBlockCounter block_counter(valid_bits, valid_bits_offset,
- batch_size);
-
- int total_processed = 0;
- int processed = 0;
- arrow::internal::BitBlockCount block;
+ internal::NoOpConverter<value_type> converter;
- do {
- block = block_counter.NextFourWords();
- if (block.length == 0) {
- break;
- }
- if (block.AllSet()) {
- processed = GetBatch<T>(out, block.length);
- } else if (block.NoneSet()) {
- converter.FillZero(out, out + block.length);
- processed = block.length;
- } else {
- processed = GetSpaced<T, /*RunType=*/T, PlainRleConverter<T>>(
- converter, block.length, block.length - block.popcount, valid_bits,
- valid_bits_offset, out);
- }
- total_processed += processed;
- out += block.length;
- valid_bits_offset += block.length;
- } while (processed == block.length);
- return total_processed;
+ return GetSpaced(converter, out, batch_size, valid_bits, valid_bits_offset, null_count);
}
-static inline bool IndexInRange(int32_t idx, int32_t dictionary_length) {
- return idx >= 0 && idx < dictionary_length;
+namespace internal {
+
+template <typename I>
+bool IndexInRange(I idx, int32_t dictionary_length) {
+ ARROW_DCHECK_GT(dictionary_length, 0);
+ using T = std::common_type_t<decltype(idx), decltype(dictionary_length)>;
+ return idx >= 0 && static_cast<T>(idx) < static_cast<T>(dictionary_length);
}
// Converter for GetSpaced that handles runs of returned dictionary
// indices.
-template <typename T>
+template <typename V, typename I>
struct DictionaryConverter {
- T kZero = {};
- const T* dictionary;
- int32_t dictionary_length;
-
- inline bool IsValid(int32_t value) { return IndexInRange(value, dictionary_length); }
-
- inline bool IsValid(const int32_t* values, int32_t length) const {
- using IndexType = int32_t;
- IndexType min_index = std::numeric_limits<IndexType>::max();
- IndexType max_index = std::numeric_limits<IndexType>::min();
- for (int x = 0; x < length; x++) {
- min_index = std::min(values[x], min_index);
- max_index = std::max(values[x], max_index);
+ using out_type = V;
+ using in_type = I;
+ using size_type = int32_t;
+
+ static constexpr bool kIsIdentity = false;
+
+ const out_type* dictionary;
+ size_type dictionary_length;
+
+ [[nodiscard]] bool InputIsValid(in_type idx) const {
+ return IndexInRange(idx, dictionary_length);
+ }
+
+ [[nodiscard]] bool InputIsValid(const in_type* indices, size_type length) const {
+ in_type min_index = std::numeric_limits<in_type>::max();
+ in_type max_index = std::numeric_limits<in_type>::min();
+ for (size_type x = 0; x < length; x++) {
+ min_index = std::min(indices[x], min_index);
+ max_index = std::max(indices[x], max_index);
}
return IndexInRange(min_index, dictionary_length) &&
IndexInRange(max_index, dictionary_length);
}
- inline void Fill(T* begin, T* end, const int32_t& run_value) const {
+
+ void WriteRepeated(out_type* begin, out_type* end, in_type run_value) const {
std::fill(begin, end, dictionary[run_value]);
}
- inline void FillZero(T* begin, T* end) { std::fill(begin, end, kZero); }
- inline void Copy(T* out, const int32_t* values, int length) const {
- for (int x = 0; x < length; x++) {
+ static void WriteZero(out_type* begin, out_type* end) {
+ std::fill(begin, end, out_type{});
+ }
+
+ void WriteRange(out_type* out, const in_type* values, size_type length) const {
+ for (size_type x = 0; x < length; x++) {
out[x] = dictionary[values[x]];
}
}
};
+/// Dummy imitation of BitRun that is all set.
+struct AllSetBitRun {
+ static constexpr bool set = true;
+ int64_t length = 0;
+};
+
+/// Dummy imitation of BitRunReader that should never be called.
+struct UnreachableBitRunReader {
+ constexpr static AllSetBitRun NextRun() { return {}; }
+};
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatchWithDict(const T* dictionary, int32_t dictionary_length,
- T* values, int batch_size) {
- // Per https://github.com/apache/parquet-format/blob/master/Encodings.md,
- // the maximum dictionary index width in Parquet is 32 bits.
- using IndexType = int32_t;
- DictionaryConverter<T> converter;
- converter.dictionary = dictionary;
- converter.dictionary_length = dictionary_length;
-
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) {
- auto idx = static_cast<IndexType>(current_value_);
- if (ARROW_PREDICT_FALSE(!IndexInRange(idx, dictionary_length))) {
- return values_read;
- }
- T val = dictionary[idx];
+template <typename V>
+auto RleBitPackedDecoder<T>::GetBatchWithDict(const V* dictionary,
+ int32_t dictionary_length, V* out,
+ values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
+
+ if (ARROW_PREDICT_FALSE(batch_size <= 0)) {
+ return 0;
+ }
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, val);
+ internal::DictionaryConverter<V, value_type> converter{dictionary, dictionary_length};
- /* Upkeep counters */
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- constexpr int kBufferSize = 1024;
- IndexType indices[kBufferSize];
+ // Make lightweight BitRun class to reuse previous methods.
+ constexpr internal::UnreachableBitRunReader validity_reader{};
+ internal::AllSetBitRun validity_run = {batch_size};
- int literal_batch = std::min(remaining, literal_count_);
- literal_batch = std::min(literal_batch, kBufferSize);
+ values_count_type values_read = 0;
+ auto batch_values_remaining = [&]() {
+ ARROW_DCHECK_LE(values_read, batch_size);
+ return batch_size - values_read;
+ };
- int actual_read = bit_reader_.GetBatch(bit_width_, indices, literal_batch);
- if (ARROW_PREDICT_FALSE(actual_read != literal_batch)) {
- return values_read;
- }
- if (ARROW_PREDICT_FALSE(!converter.IsValid(indices, /*length=*/literal_batch))) {
- return values_read;
- }
- converter.Copy(out, indices, literal_batch);
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const [run_values_read, run_null_read] =
+ RunGetSpaced(converter, out, batch_size, /* null_count= */ 0, validity_reader,
+ validity_run, decoder_);
- /* Upkeep counters */
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<IndexType>()) return values_read;
+ ARROW_DCHECK_EQ(run_null_read, 0);
+ values_read += run_values_read;
+ out += run_values_read;
+
+ // Either we fulfilled all the batch values to be read
+ if (ARROW_PREDICT_FALSE(values_read >= batch_size)) {
+ // There may be remaining null if they are not greedily filled
+ return values_read;
}
+
+ /// We finished the remaining run
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+
+ auto const [run_values_read, run_null_read] = internal::RunGetSpaced(
+ converter, out, batch_values_remaining(), /* null_count= */ 0,
+ validity_reader, validity_run, decoder);
+
+ ARROW_DCHECK_EQ(run_null_read, 0);
+ values_read += run_values_read;
+ out += run_values_read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(run_values_read == 0 || values_read == batch_size)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
return values_read;
}
template <typename T>
-inline int RleDecoder::GetBatchWithDictSpaced(const T* dictionary,
- int32_t dictionary_length, T* out,
- int batch_size, int null_count,
- const uint8_t* valid_bits,
- int64_t valid_bits_offset) {
+template <typename V>
+auto RleBitPackedDecoder<T>::GetBatchWithDictSpaced(
+ const V* dictionary, int32_t dictionary_length, V* out, values_count_type batch_size,
+ values_count_type null_count, const uint8_t* valid_bits, int64_t valid_bits_offset)
+ -> values_count_type {
if (null_count == 0) {
- return GetBatchWithDict<T>(dictionary, dictionary_length, out, batch_size);
- }
- arrow::internal::BitBlockCounter block_counter(valid_bits, valid_bits_offset,
- batch_size);
- using IndexType = int32_t;
- DictionaryConverter<T> converter;
- converter.dictionary = dictionary;
- converter.dictionary_length = dictionary_length;
-
- int total_processed = 0;
- int processed = 0;
- arrow::internal::BitBlockCount block;
- do {
- block = block_counter.NextFourWords();
- if (block.length == 0) {
- break;
+ return GetBatchWithDict<V>(dictionary, dictionary_length, out, batch_size);
+ }
+ internal::DictionaryConverter<V, value_type> converter{dictionary, dictionary_length};
+
+ return GetSpaced(converter, out, batch_size, valid_bits, valid_bits_offset, null_count);
+}
+
+/************
+ * RleRun *
+ ************/
+
+inline RleRun::RleRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept
+ : values_count_(values_count), value_bit_width_(value_bit_width) {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_GE(values_count, 0);
+ std::copy(data, data + RawDataSize(), data_.begin());
+}
+
+constexpr auto RleRun::ValuesCount() const noexcept -> values_count_type {
+ return values_count_;
+}
+
+constexpr auto RleRun::ValuesBitWidth() const noexcept -> bit_size_type {
+ return value_bit_width_;
+}
+
+constexpr auto RleRun::RawDataPtr() const noexcept -> raw_data_const_pointer {
+ return data_.data();
+}
+
+constexpr auto RleRun::RawDataSize() const noexcept -> raw_data_size_type {
+ auto out = bit_util::BytesForBits(value_bit_width_);
+ ARROW_DCHECK_LE(out, std::numeric_limits<raw_data_size_type>::max());
+ return static_cast<raw_data_size_type>(out);
+};
+
+/******************
+ * BitPackedRun *
+ ******************/
+
+constexpr BitPackedRun::BitPackedRun(raw_data_const_pointer data,
+ values_count_type values_count,
+ bit_size_type value_bit_width) noexcept
+ : data_(data), values_count_(values_count), value_bit_width_(value_bit_width) {
+ ARROW_CHECK_GE(value_bit_width_, 0);
+ ARROW_CHECK_GE(values_count_, 0);
+}
+
+constexpr auto BitPackedRun::ValuesCount() const noexcept -> values_count_type {
+ return values_count_;
+}
+
+constexpr auto BitPackedRun::ValuesBitWidth() const noexcept -> bit_size_type {
+ return value_bit_width_;
+}
+
+constexpr auto BitPackedRun::RawDataPtr() const noexcept -> raw_data_const_pointer {
+ return data_;
+}
+
+constexpr auto BitPackedRun::RawDataSize() const noexcept -> raw_data_size_type {
+ auto out = bit_util::BytesForBits(static_cast<int64_t>(value_bit_width_) *
+ static_cast<int64_t>(values_count_));
+ ARROW_CHECK_LE(out, std::numeric_limits<raw_data_size_type>::max());
+ return static_cast<raw_data_size_type>(out);
+}
+
+/************************
+ * RleBitPackedParser *
+ ************************/
+
+constexpr RleBitPackedParser::RleBitPackedParser(raw_data_const_pointer data,
+ raw_data_size_type size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, size, value_bit_width);
+}
+
+constexpr void RleBitPackedParser::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ data_ = data;
+ data_size_ = data_size;
+ value_bit_width_ = value_bit_width;
+}
+
+inline auto RleBitPackedParser::Peek() const -> std::optional<dynamic_run_type> {
+ if (ARROW_PREDICT_FALSE(Exhausted())) {
+ return {};
+ }
+
+ auto out = std::optional<dynamic_run_type>{};
+ auto handler = internal::LambdaHandler{[&](auto run) {
+ out = run;
+ return ControlFlow::Break;
+ }};
+ PeekImpl(handler);
+ return out;
+}
+
+inline auto RleBitPackedParser::Next() -> std::optional<dynamic_run_type> {
+ if (ARROW_PREDICT_FALSE(Exhausted())) {
+ return {};
+ }
+
+ auto out = std::optional<dynamic_run_type>{};
+ auto handler = internal::LambdaHandler{[&](auto run) {
+ out = run;
+ return ControlFlow::Break;
+ }};
+ PeekImpl(handler);
+ auto [read, control] = PeekImpl(handler);
+ data_ += read;
+ data_size_ -= read;
+ return out;
+}
+
+inline bool RleBitPackedParser::Advance() { return Next().has_value(); }
+
+inline bool RleBitPackedParser::Exhausted() const { return data_size_ == 0; }
+
+template <typename Handler>
+auto RleBitPackedParser::PeekImpl(Handler&& handler) const
+ -> std::pair<raw_data_size_type, ControlFlow> {
+ ARROW_DCHECK(!Exhausted());
+
+ constexpr auto kMaxSize = bit_util::MaxLEB128ByteLenFor<uint32_t>;
+ uint32_t run_len_type = 0;
+ auto const header_bytes = bit_util::ParseLeadingLEB128(data_, kMaxSize, &run_len_type);
+
+ if (ARROW_PREDICT_FALSE(header_bytes == 0)) {
+ // Malfomrmed LEB128 data
+ return {};
+ }
+
+ bool const is_bit_packed = run_len_type & 1;
+ uint32_t const count = run_len_type >> 1;
+ if (is_bit_packed) {
+ using values_count_type = BitPackedRun::values_count_type;
+ constexpr auto kMaxCount =
+ bit_util::CeilDiv(internal::max_size_for_v<values_count_type>, 8);
+ if (ARROW_PREDICT_FALSE(count == 0 || count > kMaxCount)) {
+ /// Illegal number of encoded values
Review Comment:
Let's use `//` for comments and `///` for actual docstrings.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2319504021
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ ARROW_DCHECK_LT(values_read, batch_size);
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+ auto const read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
return values_read;
}
-template <typename T, typename RunType, typename Converter>
-inline int RleDecoder::GetSpaced(Converter converter, int batch_size, int null_count,
- const uint8_t* valid_bits, int64_t valid_bits_offset,
- T* out) {
- if (ARROW_PREDICT_FALSE(null_count == batch_size)) {
- converter.FillZero(out, out + batch_size);
- return batch_size;
+namespace internal {
+
+/// Utility class to safely handle values and null count without too error-prone
+/// verbosity.
+class BatchCounter {
+ public:
+ using size_type = int32_t;
+
+ [[nodiscard]] static constexpr BatchCounter FromBatchSizeAndNulls(
+ size_type batch_size, size_type null_count) {
+ ARROW_DCHECK_LE(null_count, batch_size);
+ return {batch_size - null_count, null_count};
}
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
- int values_remaining = batch_size - null_count;
+ constexpr BatchCounter(size_type values_count, size_type null_count) noexcept
+ : values_count_(values_count), null_count_(null_count) {}
- // Assume no bits to start.
- arrow::internal::BitRunReader bit_reader(valid_bits, valid_bits_offset,
- /*length=*/batch_size);
- arrow::internal::BitRun valid_run = bit_reader.NextRun();
- while (values_read < batch_size) {
- if (ARROW_PREDICT_FALSE(valid_run.length == 0)) {
- valid_run = bit_reader.NextRun();
+ [[nodiscard]] constexpr size_type ValuesCount() const noexcept { return values_count_; }
+
+ [[nodiscard]] constexpr size_type ValuesRead() const noexcept { return values_read_; }
+
+ [[nodiscard]] constexpr size_type ValuesRemaining() const noexcept {
+ ARROW_DCHECK_LE(values_read_, values_count_);
+ return values_count_ - values_read_;
+ }
+
+ constexpr void AccrueReadValues(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, ValuesRemaining());
+ values_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type NullCount() const noexcept { return null_count_; }
+
+ [[nodiscard]] constexpr size_type NullRead() const noexcept { return null_read_; }
+
+ [[nodiscard]] constexpr size_type NullRemaining() const noexcept {
+ ARROW_DCHECK_LE(null_read_, null_count_);
+ return null_count_ - null_read_;
+ }
+
+ constexpr void AccrueReadNulls(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, NullRemaining());
+ null_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type TotalRemaining() const noexcept {
+ return ValuesRemaining() + NullRemaining();
+ }
+
+ [[nodiscard]] constexpr size_type TotalRead() const noexcept {
+ return values_read_ + null_read_;
+ }
+
+ [[nodiscard]] constexpr bool IsFullyNull() const noexcept {
+ return ValuesRemaining() == 0;
+ }
+
+ [[nodiscard]] constexpr bool IsDone() const noexcept { return TotalRemaining() == 0; }
+
+ private:
+ size_type values_count_ = 0;
+ size_type values_read_ = 0;
+ size_type null_count_ = 0;
+ size_type null_read_ = 0;
+};
+
+// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+/// Overload for GetSpaced for a single run in a RleDecoder
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
Review Comment:
The rrefs here are a bit unexpected, it's not obvious what the API contract is? Perhaps passing pointers would make clearer that these are in-out arguments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2336229247
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -84,32 +85,278 @@ namespace util {
/// (total 26 bytes, 1 byte overhead)
//
+class RleRun {
+ public:
+ using byte = uint8_t;
+ /// Enough space to store a 64bit value
+ using raw_data_storage = std::array<byte, 8>;
+ using raw_data_const_pointer = const byte*;
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr RleRun() noexcept = default;
+ constexpr RleRun(RleRun const&) noexcept = default;
+ constexpr RleRun(RleRun&&) noexcept = default;
+
+ explicit RleRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr RleRun& operator=(RleRun const&) noexcept = default;
+ constexpr RleRun& operator=(RleRun&&) noexcept = default;
+
+ /// The number of repeated values in this run.
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ /// A pointer to the repeated value raw bytes.
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ /// The number of bytes used for the raw repeated value.
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The repeated value raw bytes stored inside the class
+ raw_data_storage data_ = {};
+ /// The number of time the value is repeated
+ values_count_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+class BitPackedRun {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// According to the Parquet thrift definition the page size can be written into an
+ /// int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr BitPackedRun() noexcept = default;
+ constexpr BitPackedRun(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun(BitPackedRun&&) noexcept = default;
+
+ constexpr BitPackedRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr BitPackedRun& operator=(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun& operator=(BitPackedRun&&) noexcept = default;
+
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Number of values in this run.
+ raw_data_size_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+/// A parser that emits either a ``BitPackedRun`` or a ``RleRun``.
+class RleBitPackedParser {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// By Parquet thrift definition the page size can be written into an int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+ /// The different types of runs emitted by the parser
+ using dynamic_run_type = std::variant<RleRun, BitPackedRun>;
+
+ constexpr RleBitPackedParser() noexcept = default;
+
+ constexpr RleBitPackedParser(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr void Reset(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width_) noexcept;
+
+ /// Get the current run with a small parsing cost without advancing the iteration.
+ [[nodiscard]] std::optional<dynamic_run_type> Peek() const;
+
+ /// Move to the next run.
+ [[nodiscard]] bool Advance();
+
+ /// Advance and return the current run.
+ [[nodiscard]] std::optional<dynamic_run_type> Next();
+
+ /// Whether there is still runs to iterate over.
+ ///
+ /// WARN: Due to lack of proper error handling, iteration with Next and Peek could
+ /// return not data while the parser is not exhausted.
+ /// This is how one can check for errors.
+ [[nodiscard]] bool Exhausted() const;
+
+ /// Enum to return from an ``Parse`` handler.
+ ///
+ /// Since a callback has no way to know when to stop, the handler must return
+ /// a value indicating to the ``Parse`` function whether to stop or continue.
+ enum class ControlFlow {
+ Continue,
+ Break,
+ };
+
+ /// A callback approach to parsing.
+ ///
+ /// This approach is used to reduce the number of dynamic lookups involved with using a
+ /// variant.
+ ///
+ /// The handler must be of the form
+ /// ```cpp`
+ /// struct Handler {
+ /// ControlFlow OnBitPackedRun(BitPackedRun run);
+ ///
+ /// ControlFlow OnRleRun(RleRun run);
+ /// };
+ /// ```
+ template <typename Handler>
+ void Parse(Handler&& handler);
Review Comment:
Originally it was an iteration API, then for performance we switched to callback to avoid hiding the run type inside a variant.
I now removed what was left of the iteration API.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2336229903
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -84,32 +85,278 @@ namespace util {
/// (total 26 bytes, 1 byte overhead)
//
+class RleRun {
+ public:
+ using byte = uint8_t;
+ /// Enough space to store a 64bit value
+ using raw_data_storage = std::array<byte, 8>;
+ using raw_data_const_pointer = const byte*;
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr RleRun() noexcept = default;
+ constexpr RleRun(RleRun const&) noexcept = default;
+ constexpr RleRun(RleRun&&) noexcept = default;
+
+ explicit RleRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr RleRun& operator=(RleRun const&) noexcept = default;
+ constexpr RleRun& operator=(RleRun&&) noexcept = default;
+
+ /// The number of repeated values in this run.
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ /// A pointer to the repeated value raw bytes.
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ /// The number of bytes used for the raw repeated value.
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The repeated value raw bytes stored inside the class
+ raw_data_storage data_ = {};
+ /// The number of time the value is repeated
+ values_count_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+class BitPackedRun {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// According to the Parquet thrift definition the page size can be written into an
+ /// int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr BitPackedRun() noexcept = default;
+ constexpr BitPackedRun(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun(BitPackedRun&&) noexcept = default;
+
+ constexpr BitPackedRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr BitPackedRun& operator=(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun& operator=(BitPackedRun&&) noexcept = default;
+
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Number of values in this run.
+ raw_data_size_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+/// A parser that emits either a ``BitPackedRun`` or a ``RleRun``.
+class RleBitPackedParser {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// By Parquet thrift definition the page size can be written into an int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+ /// The different types of runs emitted by the parser
+ using dynamic_run_type = std::variant<RleRun, BitPackedRun>;
+
+ constexpr RleBitPackedParser() noexcept = default;
+
+ constexpr RleBitPackedParser(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr void Reset(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width_) noexcept;
+
+ /// Get the current run with a small parsing cost without advancing the iteration.
+ [[nodiscard]] std::optional<dynamic_run_type> Peek() const;
+
+ /// Move to the next run.
+ [[nodiscard]] bool Advance();
+
+ /// Advance and return the current run.
+ [[nodiscard]] std::optional<dynamic_run_type> Next();
Review Comment:
Removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2337159964
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ ARROW_DCHECK_LT(values_read, batch_size);
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+ auto const read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
return values_read;
}
-template <typename T, typename RunType, typename Converter>
-inline int RleDecoder::GetSpaced(Converter converter, int batch_size, int null_count,
- const uint8_t* valid_bits, int64_t valid_bits_offset,
- T* out) {
- if (ARROW_PREDICT_FALSE(null_count == batch_size)) {
- converter.FillZero(out, out + batch_size);
- return batch_size;
+namespace internal {
+
+/// Utility class to safely handle values and null count without too error-prone
+/// verbosity.
+class BatchCounter {
+ public:
+ using size_type = int32_t;
+
+ [[nodiscard]] static constexpr BatchCounter FromBatchSizeAndNulls(
+ size_type batch_size, size_type null_count) {
+ ARROW_DCHECK_LE(null_count, batch_size);
+ return {batch_size - null_count, null_count};
}
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
- int values_remaining = batch_size - null_count;
+ constexpr BatchCounter(size_type values_count, size_type null_count) noexcept
+ : values_count_(values_count), null_count_(null_count) {}
- // Assume no bits to start.
- arrow::internal::BitRunReader bit_reader(valid_bits, valid_bits_offset,
- /*length=*/batch_size);
- arrow::internal::BitRun valid_run = bit_reader.NextRun();
- while (values_read < batch_size) {
- if (ARROW_PREDICT_FALSE(valid_run.length == 0)) {
- valid_run = bit_reader.NextRun();
+ [[nodiscard]] constexpr size_type ValuesCount() const noexcept { return values_count_; }
+
+ [[nodiscard]] constexpr size_type ValuesRead() const noexcept { return values_read_; }
+
+ [[nodiscard]] constexpr size_type ValuesRemaining() const noexcept {
+ ARROW_DCHECK_LE(values_read_, values_count_);
+ return values_count_ - values_read_;
+ }
+
+ constexpr void AccrueReadValues(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, ValuesRemaining());
+ values_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type NullCount() const noexcept { return null_count_; }
+
+ [[nodiscard]] constexpr size_type NullRead() const noexcept { return null_read_; }
+
+ [[nodiscard]] constexpr size_type NullRemaining() const noexcept {
+ ARROW_DCHECK_LE(null_read_, null_count_);
+ return null_count_ - null_read_;
+ }
+
+ constexpr void AccrueReadNulls(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, NullRemaining());
+ null_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type TotalRemaining() const noexcept {
+ return ValuesRemaining() + NullRemaining();
+ }
+
+ [[nodiscard]] constexpr size_type TotalRead() const noexcept {
+ return values_read_ + null_read_;
+ }
+
+ [[nodiscard]] constexpr bool IsFullyNull() const noexcept {
+ return ValuesRemaining() == 0;
+ }
+
+ [[nodiscard]] constexpr bool IsDone() const noexcept { return TotalRemaining() == 0; }
+
+ private:
+ size_type values_count_ = 0;
+ size_type values_read_ = 0;
+ size_type null_count_ = 0;
+ size_type null_read_ = 0;
+};
+
+// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+/// Overload for GetSpaced for a single run in a RleDecoder
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ RleDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto values_remaining_run = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // Consume as much as possible from the repeated run.
+ // We only need to count the number of nulls and non-nulls because we can fill in the
+ // same value for nulls and non-nulls.
+ // This proves to be a big efficiency win.
+ while (values_remaining_run() > 0 && !batch.IsDone()) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const& validity_run_size = static_cast<values_count_type>(validity_run.length);
+
+ if (validity_run.set) {
+ // We may end the current RLE run in the middle of the validity run
+ auto update_size = std::min(validity_run_size, values_remaining_run());
+ batch.AccrueReadValues(update_size);
+ validity_run.length -= update_size;
+ } else {
+ // We can consume all nulls here because it does not matter if we consume on this
+ // RLE run, or an a next encoded run. The value filled does not matter.
+ auto update_size = std::min(validity_run_size, batch.NullRemaining());
+ batch.AccrueReadNulls(update_size);
+ validity_run.length -= update_size;
+ }
+
+ if (ARROW_PREDICT_TRUE(validity_run.length == 0)) {
+ validity_run = validity_reader.NextRun();
}
+ }
+
+ value_type const value = decoder.Value();
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(value))) {
+ return {0, 0};
+ }
+ converter.WriteRepeated(out, out + batch.TotalRead(), value);
+ auto const actual_values_read = decoder.Advance(batch.ValuesRead());
+ // We always cropped the number of values_read by the remaining values in the run.
+ // What's more the RLE decoder should not encounter any errors.
+ ARROW_DCHECK_EQ(actual_values_read, batch.ValuesRead());
- ARROW_DCHECK_GT(batch_size, 0);
- ARROW_DCHECK_GT(valid_run.length, 0);
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+template <typename T, typename... Ts>
+[[nodiscard]] constexpr T min(T x, Ts... ys) {
+ ((x = std::min(x, ys)), ...);
+ return x;
+}
+
+static_assert(min(5) == 5);
+static_assert(min(5, 4, -1) == -1);
+static_assert(min(5, 41) == 5);
+
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ BitPackedDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto run_values_remaining = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ while (run_values_remaining() > 0 && batch.ValuesRemaining() > 0) {
Review Comment:
I'm not sure that works. It seems easy but there quite a few different indices, triggering different things when they reach zero.
I prefer to keep things separate depending on where they come from.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2337010039
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
Review Comment:
Yes (move assignment), the intent is to clear the decoder (make sure we don't hold on to old pointers).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2349616714
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -207,12 +209,310 @@ TEST(BitUtil, RoundTripIntValues) {
}
}
+/// A Rle run is a simple class owning some data and a repetition count.
+/// It does not know how to read such data.
+TEST(Rle, RleRun) {
+ const std::array<RleRun::byte, 4> value = {21, 2, 0, 0};
+
+ RleRun::values_count_type value_count = 12;
+
+ // 12 times the value 21 fitting over 5 bits
+ auto const run_5 = RleRun(value.data(), value_count, /* value_bit_width= */ 5);
+ EXPECT_EQ(run_5.ValuesCount(), value_count);
+ EXPECT_EQ(run_5.ValuesBitWidth(), 5);
+ EXPECT_EQ(run_5.RawDataSize(), 1); // 5 bits fit in one byte
+ EXPECT_EQ(*run_5.RawDataPtr(), 21);
+
+ // 12 times the value 21 fitting over 16 bits
+ auto const run_8 = RleRun(value.data(), value_count, /* value_bit_width= */ 8);
+ EXPECT_EQ(run_8.ValuesCount(), value_count);
+ EXPECT_EQ(run_8.ValuesBitWidth(), 8);
+ EXPECT_EQ(run_8.RawDataSize(), 1); // 8 bits fit in 1 byte
+ EXPECT_EQ(*run_8.RawDataPtr(), 21);
+
+ // 12 times the value {21, 2} fitting over 10 bits
+ auto const run_10 = RleRun(value.data(), value_count, /* value_bit_width= */ 10);
+
+ EXPECT_EQ(run_10.ValuesCount(), value_count);
+ EXPECT_EQ(run_10.ValuesBitWidth(), 10);
+ EXPECT_EQ(run_10.RawDataSize(), 2); // 10 bits fit in 2 bytes
+ EXPECT_EQ(*(run_10.RawDataPtr() + 0), 21);
+ EXPECT_EQ(*(run_10.RawDataPtr() + 1), 2);
+
+ // 12 times the value {21, 2} fitting over 32 bits
+ auto const run_32 = RleRun(value.data(), value_count, /* value_bit_width= */ 32);
+ EXPECT_EQ(run_32.ValuesCount(), value_count);
+ EXPECT_EQ(run_32.ValuesBitWidth(), 32);
+ EXPECT_EQ(run_32.RawDataSize(), 4); // 32 bits fit in 4 bytes
+ EXPECT_EQ(*(run_32.RawDataPtr() + 0), 21);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 1), 2);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 2), 0);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 3), 0);
+}
+
+/// A BitPacked run is a simple class owning some data and its size.
+/// It does not know how to read such data.
+TEST(BitPacked, BitPackedRun) {
+ const std::array<BitPackedRun::byte, 4> value = {0b10101010, 0, 0, 0b1111111};
+
+ /// 16 values of 1 bit for a total of 16 bits
+ BitPackedRun::values_count_type value_count_1 = 16;
+ auto const run_1 = BitPackedRun(value.data(), value_count_1, /* value_bit_width= */ 1);
+ EXPECT_EQ(run_1.ValuesCount(), value_count_1);
+ EXPECT_EQ(run_1.ValuesBitWidth(), 1);
+ EXPECT_EQ(run_1.RawDataSize(), 2); // 16 bits fit in 2 bytes
+ for (BitPackedRun::raw_data_size_type i = 0; i < run_1.RawDataSize(); ++i) {
+ EXPECT_EQ(*(run_1.RawDataPtr() + i), value[i]);
+ }
+
+ /// 8 values of 3 bits for a total of 24 bits
+ BitPackedRun::values_count_type value_count_3 = 8;
+ auto const run_3 = BitPackedRun(value.data(), value_count_3, /* value_bit_width= */ 3);
+ EXPECT_EQ(run_3.ValuesCount(), value_count_3);
+ EXPECT_EQ(run_3.ValuesBitWidth(), 3);
+ EXPECT_EQ(run_3.RawDataSize(), 3); // 24 bits fit in 3 bytes
+ for (BitPackedRun::raw_data_size_type i = 0; i < run_3.RawDataSize(); ++i) {
+ EXPECT_EQ(*(run_3.RawDataPtr() + i), value[i]);
+ }
+}
+
+template <typename T>
+void TestRleDecoder(std::vector<RleRun::byte> bytes,
+ RleRun::values_count_type value_count,
+ RleRun::bit_size_type bit_width) {
+ // Pre-requisite for this test
+ EXPECT_GT(value_count, 6);
+
+ // Compute value associated with bytes encoded as little endian
+ T value = 0;
+ for (std::size_t i = 0; i < bytes.size(); ++i) {
+ value += static_cast<T>(bytes.at(i)) << (8 * i);
+ }
+
+ auto const run = RleRun(bytes.data(), value_count, bit_width);
+
+ auto decoder = RleDecoder<T>(run);
+ std::vector<T> vals = {0, 0};
+
+ EXPECT_EQ(decoder.Remaining(), value_count);
+
+ typename decltype(decoder)::values_count_type read = 0;
+ EXPECT_EQ(decoder.Get(vals.data()), 1);
+ read += 1;
+ EXPECT_EQ(vals.at(0), value);
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ EXPECT_EQ(decoder.Advance(3), 3);
+ read += 3;
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ vals = {0, 0};
+ EXPECT_EQ(decoder.GetBatch(vals.data(), 2), vals.size());
+ EXPECT_EQ(vals.at(0), value);
+ EXPECT_EQ(vals.at(1), value);
+ read += static_cast<decltype(read)>(vals.size());
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ // Exhaust iteration
+ EXPECT_EQ(decoder.Advance(value_count - read), value_count - read);
+ EXPECT_EQ(decoder.Remaining(), 0);
+ EXPECT_EQ(decoder.Advance(1), 0);
+ vals = {0, 0};
+ EXPECT_EQ(decoder.Get(vals.data()), 0);
+ EXPECT_EQ(vals.at(0), 0);
+
+ // Reset the decoder
+ decoder.Reset(run);
+ EXPECT_EQ(decoder.Remaining(), value_count);
+ vals = {0, 0};
+ EXPECT_EQ(decoder.GetBatch(vals.data(), 2), vals.size());
+ EXPECT_EQ(vals.at(0), value);
+ EXPECT_EQ(vals.at(1), value);
+}
+
+TEST(Rle, RleDecoder) {
+ TestRleDecoder<uint32_t>({21, 0, 0}, /* value_count= */ 21, /* bit_width= */ 5);
+ TestRleDecoder<uint16_t>({1, 0}, /* value_count= */ 13, /* bit_width= */ 1);
+ TestRleDecoder<uint64_t>({21, 2, 0, 1}, /* value_count= */ 20, /* bit_width= */ 30);
Review Comment:
Yes and done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "conbench-apache-arrow[bot] (via GitHub)" <gi...@apache.org>.
conbench-apache-arrow[bot] commented on PR #47294:
URL: https://github.com/apache/arrow/pull/47294#issuecomment-3299584564
Thanks for your patience. Conbench analyzed the 3 benchmarking runs that have been run so far on PR commit 3537727814ce5d3d795d8084f518c2704187ff1e.
There weren't enough matching historic benchmark results to make a call on whether there were regressions.
The [full Conbench report](https://github.com/apache/arrow/runs/50513758181) has more details.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "voltrondatabot (via GitHub)" <gi...@apache.org>.
voltrondatabot commented on PR #47294:
URL: https://github.com/apache/arrow/pull/47294#issuecomment-3299609550
Commit 3537727814ce5d3d795d8084f518c2704187ff1e already has scheduled benchmark runs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2349465815
##########
cpp/src/arrow/util/bit_util.h:
##########
@@ -365,5 +365,118 @@ void PackBits(const uint32_t* values, uint8_t* out) {
}
}
+constexpr int64_t MaxLEB128ByteLen(int64_t n_bits) { return CeilDiv(n_bits, 7); }
+
+template <typename Int>
+constexpr int64_t MaxLEB128ByteLenFor = MaxLEB128ByteLen(sizeof(Int) * 8);
+
+/// Write a integer as LEB128
+///
+/// Write the input value as LEB128 into the outptu buffer and return the number of bytes
+/// written.
+/// If the output buffer size is insufficient, return 0 but the output may have been
+/// written to.
+///
+/// \see https://en.wikipedia.org/wiki/LEB128
+/// \see MaxLEB128ByteLenFor
+template <typename Int>
+constexpr int32_t WriteLEB128(Int value, uint8_t* out, int32_t max_out_size) {
+ constexpr Int kLow7Mask = Int(0x7F);
+ constexpr Int kHigh7Mask = ~kLow7Mask;
+ constexpr uint8_t kContinuationBit = 0x80;
+
+ auto const out_first = out;
+
+ // Write as many bytes as the could be for the given input
+ while ((value & kHigh7Mask) != Int(0)) {
+ // We do not have enough room to write the LEB128
+ if (out - out_first >= max_out_size) {
Review Comment:
I had done a branch-less version of the LEB128 reading for in32 on the fast path that we can read all needed bytes at once. Turned out to be slower.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou merged PR #47294:
URL: https://github.com/apache/arrow/pull/47294
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2307017424
##########
cpp/src/arrow/util/bit_util.h:
##########
@@ -365,5 +365,118 @@ void PackBits(const uint32_t* values, uint8_t* out) {
}
}
+constexpr int64_t MaxLEB128ByteLen(int64_t n_bits) { return CeilDiv(n_bits, 7); }
+
+template <typename Int>
+constexpr int64_t MaxLEB128ByteLenFor = MaxLEB128ByteLen(sizeof(Int) * 8);
+
+/// Write a integer as LEB128
+///
+/// Write the input value as LEB128 into the outptu buffer and return the number of bytes
+/// written.
+/// If the output buffer size is insufficient, return 0 but the output may have been
+/// written to.
+///
+/// \see https://en.wikipedia.org/wiki/LEB128
+/// \see MaxLEB128ByteLenFor
+template <typename Int>
+constexpr int32_t WriteLEB128(Int value, uint8_t* out, int32_t max_out_size) {
+ constexpr Int kLow7Mask = Int(0x7F);
+ constexpr Int kHigh7Mask = ~kLow7Mask;
+ constexpr uint8_t kContinuationBit = 0x80;
+
+ auto const out_first = out;
+
+ // Write as many bytes as the could be for the given input
+ while ((value & kHigh7Mask) != Int(0)) {
+ // We do not have enough room to write the LEB128
+ if (out - out_first >= max_out_size) {
+ return 0;
+ }
+
+ // Write the encoded byte with continuation bit
+ *out = static_cast<uint8_t>(value & kLow7Mask) | kContinuationBit;
+ ++out;
+ // Shift remaining data
+ value >>= 7;
+ }
+
+ // We do not have enough room to write the LEB128
+ if (out - out_first >= max_out_size) {
+ return 0;
+ }
+
+ // Write last non-continuing byte
+ *out = static_cast<uint8_t>(value & kLow7Mask);
+ ++out;
+
+ return static_cast<int32_t>(out - out_first);
+}
+
+/// Parse a leading LEB128
+///
+/// Take as input a data pointer and the maximum number of bytes that can be read from it
+/// (typically the array size).
+/// When a valid LEB128 is found at the start of the data, the function writes it to the
+/// out pointer and return the number of bytes read.
+/// Otherwise, the out pointer is unmodified and zero is returned.
+///
+/// \see https://en.wikipedia.org/wiki/LEB128
+/// \see MaxLEB128ByteLenFor
+template <typename Int>
+constexpr int32_t ParseLeadingLEB128(uint8_t const* data, int32_t max_data_size,
+ Int* out) {
+ constexpr auto kMaxBytes = static_cast<int32_t>(MaxLEB128ByteLenFor<Int>);
+ static_assert(kMaxBytes >= 1);
+ constexpr uint8_t kLow7Mask = 0x7F;
+ constexpr uint8_t kContinuationBit = 0x80;
+ constexpr int32_t kSignBitCount = std::is_signed_v<Int> ? 1 : 0;
+ // Number of bits allowed for encoding data on the last byte to avoid overflow
+ constexpr uint8_t kHighBitCount = (8 * sizeof(Int) - kSignBitCount) % 7;
+ // kHighBitCount least significant `0` bits and the rest with `1`
+ constexpr uint8_t kHighForbiddenMask = ~((1 << kHighBitCount) - 1);
+
+ // Iteratively building the value
+ std::make_unsigned_t<Int> value = 0;
+
+ // Read as many bytes as the could be for the give output
+ for (int32_t i = 0; i < kMaxBytes - 1; i++) {
+ // We have not finished reading a valid LEB128, yet we run out of data
+ if (ARROW_PREDICT_FALSE(i >= max_data_size)) {
+ return 0;
+ }
+
+ // Read the byte and set its 7 LSB to in the final value
+ uint8_t const byte = data[i];
+ value |= static_cast<Int>(byte & kLow7Mask) << (7 * i);
+
+ // Check for lack of continuation flag in MSB
+ if ((byte & kContinuationBit) == 0) {
+ *out = value;
+ return i + 1;
+ }
+ }
+
+ // Process the last index avoiding overflowing
+ constexpr int32_t last = kMaxBytes - 1;
+
+ // We have not finished reading a valid LEB128, yet we run out of data
+ if (ARROW_PREDICT_FALSE(last >= max_data_size)) {
+ return 0;
+ }
+
+ uint8_t const byte = data[last];
+
+ // Need to check if there are bits that would overflow the output.
+ // Also checks that there is no continuation.
+ if (ARROW_PREDICT_FALSE((byte & kHighForbiddenMask) != 0)) {
Review Comment:
Take a uint32, that four bytes, it can be written in up to five bytes in LEB128.
But not all five bytes LEB128 fit in a uint32.
On the last byte, a possible `01111111` would be shifted by 4*7=28 overflowing some of the bits.
I added a test for that.
Writing does not have this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on PR #47294:
URL: https://github.com/apache/arrow/pull/47294#issuecomment-3223849889
Some benchmarks on Linux x86_64 cloud instance with 8 CPU 16 Gb memory and dependencies/compilers from Conda-Forge.
<details>
<summary><code>archery benchmark diff --benchmark-suite=parquet-arrow --benchmark-filter=Read --repetitions=3</code></summary>
```
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Non-regressions: (81)
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
benchmark baseline contender change % counters
BM_ReadColumn<false,BooleanType>/-1/0 120.497 MiB/sec 172.809 MiB/sec 43.414 {'family_index': 6, 'per_family_instance_index': 0, 'run_name': 'BM_ReadColumn<false,BooleanType>/-1/0', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 68}
BM_ReadStructOfStructColumn/50 1.379 GiB/sec 1.650 GiB/sec 19.712 {'family_index': 19, 'per_family_instance_index': 2, 'run_name': 'BM_ReadStructOfStructColumn/50', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 41}
BM_ReadColumn<false,BooleanType>/1/20 53.301 MiB/sec 60.698 MiB/sec 13.878 {'family_index': 6, 'per_family_instance_index': 1, 'run_name': 'BM_ReadColumn<false,BooleanType>/1/20', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 29}
BM_ReadStructColumn/50 1.251 GiB/sec 1.367 GiB/sec 9.297 {'family_index': 18, 'per_family_instance_index': 2, 'run_name': 'BM_ReadStructColumn/50', 'repetitions': 3, 'repetition_index': 2, 'threads': 1, 'iterations': 75}
BM_ReadColumn<true,BooleanType>/-1/1 29.265 MiB/sec 31.087 MiB/sec 6.227 {'family_index': 7, 'per_family_instance_index': 0, 'run_name': 'BM_ReadColumn<true,BooleanType>/-1/1', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 16}
BM_ReadColumn<false,Int32Type>/-1/1 14.031 GiB/sec 14.775 GiB/sec 5.302 {'family_index': 0, 'per_family_instance_index': 0, 'run_name': 'BM_ReadColumn<false,Int32Type>/-1/1', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 258}
BM_ReadColumn<true,DoubleType>/-1/0 1.945 GiB/sec 2.048 GiB/sec 5.261 {'family_index': 5, 'per_family_instance_index': 0, 'run_name': 'BM_ReadColumn<true,DoubleType>/-1/0', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 18}
BM_ReadColumn<true,Int64Type>/-1/0 1.957 GiB/sec 2.046 GiB/sec 4.527 {'family_index': 3, 'per_family_instance_index': 0, 'run_name': 'BM_ReadColumn<true,Int64Type>/-1/0', 'repetitions': 3, 'repetition_index': 2, 'threads': 1, 'iterations': 18}
BM_ReadColumnPlain<true,Int32Type>/null_probability:0 3.356 GiB/sec 3.504 GiB/sec 4.401 {'family_index': 9, 'per_family_instance_index': 0, 'run_name': 'BM_ReadColumnPlain<true,Int32Type>/null_probability:0', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 58}
BM_ReadStructColumn/0 10.081 GiB/sec 10.508 GiB/sec 4.236 {'family_index': 18, 'per_family_instance_index': 0, 'run_name': 'BM_ReadStructColumn/0', 'repetitions': 3, 'repetition_index': 2, 'threads': 1, 'iterations': 601}
BM_ReadListOfListColumn/99 1.452 GiB/sec 1.513 GiB/sec 4.181 {'family_index': 23, 'per_family_instance_index': 3, 'run_name': 'BM_ReadListOfListColumn/99', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 123}
BM_ReadColumn<false,Int32Type>/-1/10 13.915 GiB/sec 14.484 GiB/sec 4.083 {'family_index': 0, 'per_family_instance_index': 1, 'run_name': 'BM_ReadColumn<false,Int32Type>/-1/10', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 255}
BM_ReadColumn<false,Int64Type>/-1/50 11.318 GiB/sec 11.744 GiB/sec 3.761 {'family_index': 2, 'per_family_instance_index': 2, 'run_name': 'BM_ReadColumn<false,Int64Type>/-1/50', 'repetitions': 3, 'repetition_index': 2, 'threads': 1, 'iterations': 103}
BM_ReadColumnPlain<true,Float16LogicalType>/null_probability:100 2.162 GiB/sec 2.229 GiB/sec 3.099 {'family_index': 11, 'per_family_instance_index': 4, 'run_name': 'BM_ReadColumnPlain<true,Float16LogicalType>/null_probability:100', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 79}
BM_ReadIndividualRowGroups 3.753 GiB/sec 3.822 GiB/sec 1.818 {'family_index': 24, 'per_family_instance_index': 0, 'run_name': 'BM_ReadIndividualRowGroups', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 34}
BM_ReadColumn<false,Int64Type>/-1/1 11.464 GiB/sec 11.670 GiB/sec 1.799 {'family_index': 2, 'per_family_instance_index': 0, 'run_name': 'BM_ReadColumn<false,Int64Type>/-1/1', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 103}
BM_ReadBinaryViewColumn/null_probability:0/unique_values:32 1.070 GiB/sec 1.089 GiB/sec 1.786 {'family_index': 15, 'per_family_instance_index': 0, 'run_name': 'BM_ReadBinaryViewColumn/null_probability:0/unique_values:32', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 3}
BM_ReadListColumn/99 1.734 GiB/sec 1.763 GiB/sec 1.703 {'family_index': 21, 'per_family_instance_index': 3, 'run_name': 'BM_ReadListColumn/99', 'repetitions': 3, 'repetition_index': 2, 'threads': 1, 'iterations': 155}
BM_ReadBinaryViewColumn/null_probability:0/unique_values:-1 1.935 GiB/sec 1.967 GiB/sec 1.654 {'family_index': 15, 'per_family_instance_index': 1, 'run_name': 'BM_ReadBinaryViewColumn/null_probability:0/unique_values:-1', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 5}
BM_ReadColumnByteStreamSplit<true,Float16LogicalType>/null_probability:100 2.167 GiB/sec 2.203 GiB/sec 1.639 {'family_index': 13, 'per_family_instance_index': 4, 'run_name': 'BM_ReadColumnByteStreamSplit<true,Float16LogicalType>/null_probability:100', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 78}
BM_ReadColumnPlain<true,Float16LogicalType>/null_probability:0 1.976 GiB/sec 2.008 GiB/sec 1.606 {'family_index': 11, 'per_family_instance_index': 0, 'run_name': 'BM_ReadColumnPlain<true,Float16LogicalType>/null_probability:0', 'repetitions': 3, 'repetition_index': 2, 'threads': 1, 'iterations': 69}
BM_ReadColumnPlain<false,Int32Type>/null_probability:-1 14.423 GiB/sec 14.654 GiB/sec 1.605 {'family_index': 8, 'per_family_instance_index': 0, 'run_name': 'BM_ReadColumnPlain<false,Int32Type>/null_probability:-1', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 263}
BM_ReadColumnByteStreamSplit<false,Float16LogicalType>/null_probability:-1 18.385 GiB/sec 18.650 GiB/sec 1.444 {'family_index': 12, 'per_family_instance_index': 0, 'run_name': 'BM_ReadColumnByteStreamSplit<false,Float16LogicalType>/null_probability:-1', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 662}
BM_ReadMultipleRowGroups 3.706 GiB/sec 3.752 GiB/sec 1.248 {'family_index': 25, 'per_family_instance_index': 0, 'run_name': 'BM_ReadMultipleRowGroups', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 33}
BM_ReadStructOfStructColumn/0 8.740 GiB/sec 8.838 GiB/sec 1.118 {'family_index': 19, 'per_family_instance_index': 0, 'run_name': 'BM_ReadStructOfStructColumn/0', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 252}
BM_ReadColumn<true,Int32Type>/0/1 3.461 GiB/sec 3.498 GiB/sec 1.072 {'family_index': 1, 'per_family_instance_index': 1, 'run_name': 'BM_ReadColumn<true,Int32Type>/0/1', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 60}
BM_ReadStructOfListColumn/99 1.130 GiB/sec 1.141 GiB/sec 0.995 {'family_index': 20, 'per_family_instance_index': 3, 'run_name': 'BM_ReadStructOfListColumn/99', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 65}
BM_ReadMultipleRowGroupsGenerator 3.724 GiB/sec 3.742 GiB/sec 0.482 {'family_index': 26, 'per_family_instance_index': 0, 'run_name': 'BM_ReadMultipleRowGroupsGenerator', 'repetitions': 3, 'repetition_index': 2, 'threads': 1, 'iterations': 33}
BM_ReadBinaryViewColumn/null_probability:50/unique_values:-1 1.055 GiB/sec 1.059 GiB/sec 0.324 {'family_index': 15, 'per_family_instance_index': 6, 'run_name': 'BM_ReadBinaryViewColumn/null_probability:50/unique_values:-1', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 5}
BM_ReadStructOfListColumn/50 528.189 MiB/sec 529.803 MiB/sec 0.306 {'family_index': 20, 'per_family_instance_index': 2, 'run_name': 'BM_ReadStructOfListColumn/50', 'repetitions': 3, 'repetition_index': 2, 'threads': 1, 'iterations': 32}
BM_ReadBinaryColumnDeltaByteArray/null_probability:50/unique_values:-1 778.888 MiB/sec 780.710 MiB/sec 0.234 {'family_index': 16, 'per_family_instance_index': 2, 'run_name': 'BM_ReadBinaryColumnDeltaByteArray/null_probability:50/unique_values:-1', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 4}
BM_ReadColumn<true,Int64Type>/50/50 2.109 GiB/sec 2.113 GiB/sec 0.203 {'family_index': 3, 'per_family_instance_index': 8, 'run_name': 'BM_ReadColumn<true,Int64Type>/50/50', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 19}
BM_ReadBinaryViewColumnDeltaByteArray/null_probability:50/unique_values:-1 1.040 GiB/sec 1.041 GiB/sec 0.164 {'family_index': 17, 'per_family_instance_index': 2, 'run_name': 'BM_ReadBinaryViewColumnDeltaByteArray/null_probability:50/unique_values:-1', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 4}
BM_ReadBinaryViewColumnDeltaByteArray/null_probability:1/unique_values:-1 1.725 GiB/sec 1.724 GiB/sec -0.069 {'family_index': 17, 'per_family_instance_index': 1, 'run_name': 'BM_ReadBinaryViewColumnDeltaByteArray/null_probability:1/unique_values:-1', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 5}
BM_ReadColumn<false,Int32Type>/-1/50 14.614 GiB/sec 14.597 GiB/sec -0.116 {'family_index': 0, 'per_family_instance_index': 2, 'run_name': 'BM_ReadColumn<false,Int32Type>/-1/50', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 264}
BM_ReadColumn<true,Int64Type>/50/1 2.116 GiB/sec 2.112 GiB/sec -0.186 {'family_index': 3, 'per_family_instance_index': 9, 'run_name': 'BM_ReadColumn<true,Int64Type>/50/1', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 19}
BM_ReadColumnPlain<true,Int32Type>/null_probability:100 4.013 GiB/sec 4.004 GiB/sec -0.233 {'family_index': 9, 'per_family_instance_index': 4, 'run_name': 'BM_ReadColumnPlain<true,Int32Type>/null_probability:100', 'repetitions': 3, 'repetition_index': 2, 'threads': 1, 'iterations': 69}
BM_ReadBinaryColumnDeltaByteArray/null_probability:0/unique_values:-1 1.298 GiB/sec 1.294 GiB/sec -0.247 {'family_index': 16, 'per_family_instance_index': 0, 'run_name': 'BM_ReadBinaryColumnDeltaByteArray/null_probability:0/unique_values:-1', 'repetitions': 3, 'repetition_index': 2, 'threads': 1, 'iterations': 4}
BM_ReadColumn<true,Int32Type>/-1/0 1.086 GiB/sec 1.083 GiB/sec -0.279 {'family_index': 1, 'per_family_instance_index': 0, 'run_name': 'BM_ReadColumn<true,Int32Type>/-1/0', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 19}
BM_ReadListOfStructColumn/50 656.332 MiB/sec 654.402 MiB/sec -0.294 {'family_index': 22, 'per_family_instance_index': 2, 'run_name': 'BM_ReadListOfStructColumn/50', 'repetitions': 3, 'repetition_index': 2, 'threads': 1, 'iterations': 38}
BM_ReadColumnPlain<false,Float16LogicalType>/null_probability:-1 18.674 GiB/sec 18.573 GiB/sec -0.539 {'family_index': 10, 'per_family_instance_index': 0, 'run_name': 'BM_ReadColumnPlain<false,Float16LogicalType>/null_probability:-1', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 633}
BM_ReadColumnPlain<true,Int32Type>/null_probability:50 1.129 GiB/sec 1.123 GiB/sec -0.566 {'family_index': 9, 'per_family_instance_index': 2, 'run_name': 'BM_ReadColumnPlain<true,Int32Type>/null_probability:50', 'repetitions': 3, 'repetition_index': 2, 'threads': 1, 'iterations': 20}
BM_ReadColumn<true,Int32Type>/50/50 1.128 GiB/sec 1.122 GiB/sec -0.571 {'family_index': 1, 'per_family_instance_index': 5, 'run_name': 'BM_ReadColumn<true,Int32Type>/50/50', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 20}
BM_ReadColumn<true,Int32Type>/50/0 1.135 GiB/sec 1.127 GiB/sec -0.710 {'family_index': 1, 'per_family_instance_index': 6, 'run_name': 'BM_ReadColumn<true,Int32Type>/50/0', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 20}
BM_ReadBinaryColumnDeltaByteArray/null_probability:1/unique_values:-1 1.221 GiB/sec 1.212 GiB/sec -0.749 {'family_index': 16, 'per_family_instance_index': 1, 'run_name': 'BM_ReadBinaryColumnDeltaByteArray/null_probability:1/unique_values:-1', 'repetitions': 3, 'repetition_index': 2, 'threads': 1, 'iterations': 4}
BM_ReadColumnByteStreamSplit<true,Float16LogicalType>/null_probability:0 2.104 GiB/sec 2.085 GiB/sec -0.881 {'family_index': 13, 'per_family_instance_index': 0, 'run_name': 'BM_ReadColumnByteStreamSplit<true,Float16LogicalType>/null_probability:0', 'repetitions': 3, 'repetition_index': 2, 'threads': 1, 'iterations': 74}
BM_ReadBinaryColumn/null_probability:50/unique_values:-1 794.836 MiB/sec 787.189 MiB/sec -0.962 {'family_index': 14, 'per_family_instance_index': 6, 'run_name': 'BM_ReadBinaryColumn/null_probability:50/unique_values:-1', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 4}
BM_ReadBinaryViewColumnDeltaByteArray/null_probability:0/unique_values:-1 1.929 GiB/sec 1.910 GiB/sec -0.969 {'family_index': 17, 'per_family_instance_index': 0, 'run_name': 'BM_ReadBinaryViewColumnDeltaByteArray/null_probability:0/unique_values:-1', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 5}
BM_ReadColumn<true,BooleanType>/5/10 30.798 MiB/sec 30.487 MiB/sec -1.010 {'family_index': 7, 'per_family_instance_index': 1, 'run_name': 'BM_ReadColumn<true,BooleanType>/5/10', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 17}
BM_ReadBinaryColumn/null_probability:1/unique_values:-1 1.216 GiB/sec 1.203 GiB/sec -1.018 {'family_index': 14, 'per_family_instance_index': 5, 'run_name': 'BM_ReadBinaryColumn/null_probability:1/unique_values:-1', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 4}
BM_ReadColumn<false,Int64Type>/-1/10 11.593 GiB/sec 11.459 GiB/sec -1.161 {'family_index': 2, 'per_family_instance_index': 1, 'run_name': 'BM_ReadColumn<false,Int64Type>/-1/10', 'repetitions': 3, 'repetition_index': 2, 'threads': 1, 'iterations': 105}
BM_ReadBinaryViewColumn/null_probability:1/unique_values:-1 1.736 GiB/sec 1.716 GiB/sec -1.174 {'family_index': 15, 'per_family_instance_index': 5, 'run_name': 'BM_ReadBinaryViewColumn/null_probability:1/unique_values:-1', 'repetitions': 3, 'repetition_index': 2, 'threads': 1, 'iterations': 5}
BM_ReadBinaryViewColumn/null_probability:1/unique_values:32 1023.246 MiB/sec 1009.122 MiB/sec -1.380 {'family_index': 15, 'per_family_instance_index': 2, 'run_name': 'BM_ReadBinaryViewColumn/null_probability:1/unique_values:32', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 3}
BM_ReadBinaryViewColumn/null_probability:50/unique_values:32 857.872 MiB/sec 845.888 MiB/sec -1.397 {'family_index': 15, 'per_family_instance_index': 3, 'run_name': 'BM_ReadBinaryViewColumn/null_probability:50/unique_values:32', 'repetitions': 3, 'repetition_index': 2, 'threads': 1, 'iterations': 4}
BM_ReadBinaryColumn/null_probability:50/unique_values:32 567.955 MiB/sec 559.978 MiB/sec -1.404 {'family_index': 14, 'per_family_instance_index': 3, 'run_name': 'BM_ReadBinaryColumn/null_probability:50/unique_values:32', 'repetitions': 3, 'repetition_index': 2, 'threads': 1, 'iterations': 4}
BM_ReadColumnPlain<true,Float16LogicalType>/null_probability:50 614.041 MiB/sec 603.984 MiB/sec -1.638 {'family_index': 11, 'per_family_instance_index': 2, 'run_name': 'BM_ReadColumnPlain<true,Float16LogicalType>/null_probability:50', 'repetitions': 3, 'repetition_index': 2, 'threads': 1, 'iterations': 22}
BM_ReadColumn<true,Int64Type>/45/25 2.107 GiB/sec 2.073 GiB/sec -1.641 {'family_index': 3, 'per_family_instance_index': 7, 'run_name': 'BM_ReadColumn<true,Int64Type>/45/25', 'repetitions': 3, 'repetition_index': 2, 'threads': 1, 'iterations': 19}
BM_ReadBinaryViewColumn/null_probability:99/unique_values:-1 1.397 GiB/sec 1.372 GiB/sec -1.783 {'family_index': 15, 'per_family_instance_index': 7, 'run_name': 'BM_ReadBinaryViewColumn/null_probability:99/unique_values:-1', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 12}
BM_ReadBinaryColumn/null_probability:99/unique_values:-1 741.638 MiB/sec 728.032 MiB/sec -1.835 {'family_index': 14, 'per_family_instance_index': 7, 'run_name': 'BM_ReadBinaryColumn/null_probability:99/unique_values:-1', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 12}
BM_ReadBinaryColumn/null_probability:0/unique_values:32 754.669 MiB/sec 740.571 MiB/sec -1.868 {'family_index': 14, 'per_family_instance_index': 0, 'run_name': 'BM_ReadBinaryColumn/null_probability:0/unique_values:32', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 3}
BM_ReadColumn<false,DoubleType>/-1/0 11.838 GiB/sec 11.607 GiB/sec -1.946 {'family_index': 4, 'per_family_instance_index': 0, 'run_name': 'BM_ReadColumn<false,DoubleType>/-1/0', 'repetitions': 3, 'repetition_index': 2, 'threads': 1, 'iterations': 104}
BM_ReadListColumn/50 718.113 MiB/sec 704.088 MiB/sec -1.953 {'family_index': 21, 'per_family_instance_index': 2, 'run_name': 'BM_ReadListColumn/50', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 62}
BM_ReadColumn<true,Int64Type>/35/10 2.061 GiB/sec 2.020 GiB/sec -1.983 {'family_index': 3, 'per_family_instance_index': 6, 'run_name': 'BM_ReadColumn<true,Int64Type>/35/10', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 18}
BM_ReadBinaryViewColumn/null_probability:99/unique_values:32 1.370 GiB/sec 1.342 GiB/sec -2.059 {'family_index': 15, 'per_family_instance_index': 4, 'run_name': 'BM_ReadBinaryViewColumn/null_probability:99/unique_values:32', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 12}
BM_ReadListOfStructColumn/99 1.339 GiB/sec 1.311 GiB/sec -2.086 {'family_index': 22, 'per_family_instance_index': 3, 'run_name': 'BM_ReadListOfStructColumn/99', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 74}
BM_ReadListOfListColumn/50 631.471 MiB/sec 618.070 MiB/sec -2.122 {'family_index': 23, 'per_family_instance_index': 2, 'run_name': 'BM_ReadListOfListColumn/50', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 55}
BM_ReadBinaryColumnDeltaByteArray/null_probability:99/unique_values:-1 743.722 MiB/sec 723.739 MiB/sec -2.687 {'family_index': 16, 'per_family_instance_index': 3, 'run_name': 'BM_ReadBinaryColumnDeltaByteArray/null_probability:99/unique_values:-1', 'repetitions': 3, 'repetition_index': 2, 'threads': 1, 'iterations': 12}
BM_ReadBinaryColumn/null_probability:1/unique_values:32 718.629 MiB/sec 698.738 MiB/sec -2.768 {'family_index': 14, 'per_family_instance_index': 2, 'run_name': 'BM_ReadBinaryColumn/null_probability:1/unique_values:32', 'repetitions': 3, 'repetition_index': 2, 'threads': 1, 'iterations': 3}
BM_ReadColumn<true,Int64Type>/30/10 2.033 GiB/sec 1.974 GiB/sec -2.888 {'family_index': 3, 'per_family_instance_index': 5, 'run_name': 'BM_ReadColumn<true,Int64Type>/30/10', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 18}
BM_ReadBinaryColumn/null_probability:99/unique_values:32 729.705 MiB/sec 707.958 MiB/sec -2.980 {'family_index': 14, 'per_family_instance_index': 4, 'run_name': 'BM_ReadBinaryColumn/null_probability:99/unique_values:32', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 13}
BM_ReadBinaryColumn/null_probability:0/unique_values:-1 1.327 GiB/sec 1.285 GiB/sec -3.171 {'family_index': 14, 'per_family_instance_index': 1, 'run_name': 'BM_ReadBinaryColumn/null_probability:0/unique_values:-1', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 4}
BM_ReadBinaryViewColumnDeltaByteArray/null_probability:99/unique_values:-1 1.398 GiB/sec 1.353 GiB/sec -3.218 {'family_index': 17, 'per_family_instance_index': 3, 'run_name': 'BM_ReadBinaryViewColumnDeltaByteArray/null_probability:99/unique_values:-1', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 12}
BM_ReadColumn<true,DoubleType>/25/25 1.972 GiB/sec 1.891 GiB/sec -4.113 {'family_index': 5, 'per_family_instance_index': 2, 'run_name': 'BM_ReadColumn<true,DoubleType>/25/25', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 18}
BM_ReadColumn<true,Int32Type>/25/5 1.060 GiB/sec 1.015 GiB/sec -4.257 {'family_index': 1, 'per_family_instance_index': 4, 'run_name': 'BM_ReadColumn<true,Int32Type>/25/5', 'repetitions': 3, 'repetition_index': 2, 'threads': 1, 'iterations': 19}
BM_ReadColumnByteStreamSplit<true,Float16LogicalType>/null_probability:50 615.394 MiB/sec 588.496 MiB/sec -4.371 {'family_index': 13, 'per_family_instance_index': 2, 'run_name': 'BM_ReadColumnByteStreamSplit<true,Float16LogicalType>/null_probability:50', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 21}
BM_ReadListOfListColumn/1 907.752 MiB/sec 866.414 MiB/sec -4.554 {'family_index': 23, 'per_family_instance_index': 1, 'run_name': 'BM_ReadListOfListColumn/1', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 79}
BM_ReadColumnPlain<true,Int32Type>/null_probability:99 2.424 GiB/sec 2.314 GiB/sec -4.566 {'family_index': 9, 'per_family_instance_index': 3, 'run_name': 'BM_ReadColumnPlain<true,Int32Type>/null_probability:99', 'repetitions': 3, 'repetition_index': 2, 'threads': 1, 'iterations': 40}
BM_ReadColumn<true,Int64Type>/25/10 1.986 GiB/sec 1.893 GiB/sec -4.669 {'family_index': 3, 'per_family_instance_index': 4, 'run_name': 'BM_ReadColumn<true,Int64Type>/25/10', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 18}
BM_ReadColumn<true,Int64Type>/1/1 3.363 GiB/sec 3.201 GiB/sec -4.798 {'family_index': 3, 'per_family_instance_index': 1, 'run_name': 'BM_ReadColumn<true,Int64Type>/1/1', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 31}
BM_ReadColumn<true,Int64Type>/75/1 2.404 GiB/sec 2.287 GiB/sec -4.859 {'family_index': 3, 'per_family_instance_index': 10, 'run_name': 'BM_ReadColumn<true,Int64Type>/75/1', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 22}
BM_ReadColumnPlain<true,Float16LogicalType>/null_probability:99 1.297 GiB/sec 1.233 GiB/sec -4.942 {'family_index': 11, 'per_family_instance_index': 3, 'run_name': 'BM_ReadColumnPlain<true,Float16LogicalType>/null_probability:99', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 47}
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Regressions: (25)
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
benchmark baseline contender change % counters
BM_ReadColumn<false,DoubleType>/-1/20 12.151 GiB/sec 11.520 GiB/sec -5.190 {'family_index': 4, 'per_family_instance_index': 1, 'run_name': 'BM_ReadColumn<false,DoubleType>/-1/20', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 99}
BM_ReadStructOfListColumn/0 835.554 MiB/sec 790.684 MiB/sec -5.370 {'family_index': 20, 'per_family_instance_index': 0, 'run_name': 'BM_ReadStructOfListColumn/0', 'repetitions': 3, 'repetition_index': 2, 'threads': 1, 'iterations': 50}
BM_ReadColumn<true,Int64Type>/99/0 4.479 GiB/sec 4.226 GiB/sec -5.661 {'family_index': 3, 'per_family_instance_index': 12, 'run_name': 'BM_ReadColumn<true,Int64Type>/99/0', 'repetitions': 3, 'repetition_index': 2, 'threads': 1, 'iterations': 40}
BM_ReadColumnPlain<true,Int32Type>/null_probability:1 2.104 GiB/sec 1.984 GiB/sec -5.717 {'family_index': 9, 'per_family_instance_index': 1, 'run_name': 'BM_ReadColumnPlain<true,Int32Type>/null_probability:1', 'repetitions': 3, 'repetition_index': 2, 'threads': 1, 'iterations': 37}
BM_ReadStructColumn/1 1.942 GiB/sec 1.830 GiB/sec -5.763 {'family_index': 18, 'per_family_instance_index': 1, 'run_name': 'BM_ReadStructColumn/1', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 115}
BM_ReadColumn<true,Int64Type>/99/50 4.444 GiB/sec 4.183 GiB/sec -5.878 {'family_index': 3, 'per_family_instance_index': 11, 'run_name': 'BM_ReadColumn<true,Int64Type>/99/50', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 40}
BM_ReadListColumn/1 1.087 GiB/sec 1.023 GiB/sec -5.903 {'family_index': 21, 'per_family_instance_index': 1, 'run_name': 'BM_ReadListColumn/1', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 94}
BM_ReadListOfListColumn/0 1.009 GiB/sec 968.501 MiB/sec -6.252 {'family_index': 23, 'per_family_instance_index': 0, 'run_name': 'BM_ReadListOfListColumn/0', 'repetitions': 3, 'repetition_index': 2, 'threads': 1, 'iterations': 90}
BM_ReadColumnByteStreamSplit<true,Float16LogicalType>/null_probability:1 1.184 GiB/sec 1.109 GiB/sec -6.347 {'family_index': 13, 'per_family_instance_index': 1, 'run_name': 'BM_ReadColumnByteStreamSplit<true,Float16LogicalType>/null_probability:1', 'repetitions': 3, 'repetition_index': 2, 'threads': 1, 'iterations': 42}
BM_ReadColumnByteStreamSplit<true,Float16LogicalType>/null_probability:99 1.293 GiB/sec 1.208 GiB/sec -6.597 {'family_index': 13, 'per_family_instance_index': 3, 'run_name': 'BM_ReadColumnByteStreamSplit<true,Float16LogicalType>/null_probability:99', 'repetitions': 3, 'repetition_index': 2, 'threads': 1, 'iterations': 46}
BM_ReadStructOfListColumn/1 694.703 MiB/sec 648.480 MiB/sec -6.654 {'family_index': 20, 'per_family_instance_index': 1, 'run_name': 'BM_ReadStructOfListColumn/1', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 43}
BM_ReadColumn<true,Int32Type>/99/0 2.536 GiB/sec 2.357 GiB/sec -7.061 {'family_index': 1, 'per_family_instance_index': 8, 'run_name': 'BM_ReadColumn<true,Int32Type>/99/0', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 45}
BM_ReadColumnPlain<true,Float16LogicalType>/null_probability:1 1.181 GiB/sec 1.097 GiB/sec -7.096 {'family_index': 11, 'per_family_instance_index': 1, 'run_name': 'BM_ReadColumnPlain<true,Float16LogicalType>/null_probability:1', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 42}
BM_ReadListOfStructColumn/0 990.568 MiB/sec 920.118 MiB/sec -7.112 {'family_index': 22, 'per_family_instance_index': 0, 'run_name': 'BM_ReadListOfStructColumn/0', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 58}
BM_ReadColumn<true,Int32Type>/99/50 2.531 GiB/sec 2.337 GiB/sec -7.683 {'family_index': 1, 'per_family_instance_index': 7, 'run_name': 'BM_ReadColumn<true,Int32Type>/99/50', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 45}
BM_ReadColumn<true,Int32Type>/1/1 2.133 GiB/sec 1.967 GiB/sec -7.780 {'family_index': 1, 'per_family_instance_index': 2, 'run_name': 'BM_ReadColumn<true,Int32Type>/1/1', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 38}
BM_ReadStructColumn/99 3.581 GiB/sec 3.262 GiB/sec -8.906 {'family_index': 18, 'per_family_instance_index': 3, 'run_name': 'BM_ReadStructColumn/99', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 216}
BM_ReadStructOfStructColumn/1 1.621 GiB/sec 1.476 GiB/sec -8.945 {'family_index': 19, 'per_family_instance_index': 1, 'run_name': 'BM_ReadStructOfStructColumn/1', 'repetitions': 3, 'repetition_index': 2, 'threads': 1, 'iterations': 49}
BM_ReadListOfStructColumn/1 804.476 MiB/sec 731.014 MiB/sec -9.132 {'family_index': 22, 'per_family_instance_index': 1, 'run_name': 'BM_ReadListOfStructColumn/1', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 47}
BM_ReadStructOfStructColumn/99 3.142 GiB/sec 2.838 GiB/sec -9.691 {'family_index': 19, 'per_family_instance_index': 3, 'run_name': 'BM_ReadStructOfStructColumn/99', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 93}
BM_ReadColumn<true,Int64Type>/5/5 2.330 GiB/sec 2.073 GiB/sec -11.041 {'family_index': 3, 'per_family_instance_index': 2, 'run_name': 'BM_ReadColumn<true,Int64Type>/5/5', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 21}
BM_ReadListColumn/0 1.300 GiB/sec 1.149 GiB/sec -11.568 {'family_index': 21, 'per_family_instance_index': 0, 'run_name': 'BM_ReadListColumn/0', 'repetitions': 3, 'repetition_index': 0, 'threads': 1, 'iterations': 112}
BM_ReadColumn<true,DoubleType>/10/50 2.022 GiB/sec 1.787 GiB/sec -11.608 {'family_index': 5, 'per_family_instance_index': 1, 'run_name': 'BM_ReadColumn<true,DoubleType>/10/50', 'repetitions': 3, 'repetition_index': 2, 'threads': 1, 'iterations': 18}
BM_ReadColumn<true,Int64Type>/10/5 2.035 GiB/sec 1.775 GiB/sec -12.771 {'family_index': 3, 'per_family_instance_index': 3, 'run_name': 'BM_ReadColumn<true,Int64Type>/10/5', 'repetitions': 3, 'repetition_index': 2, 'threads': 1, 'iterations': 18}
BM_ReadColumn<true,Int32Type>/10/10 1.086 GiB/sec 970.014 MiB/sec -12.778 {'family_index': 1, 'per_family_instance_index': 3, 'run_name': 'BM_ReadColumn<true,Int32Type>/10/10', 'repetitions': 3, 'repetition_index': 1, 'threads': 1, 'iterations': 20}
```
</details>
----------
The new implementation is on-par with the previous one. Sometimes better, sometimes worse, possibly slightly worse on average.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "voltrondatabot (via GitHub)" <gi...@apache.org>.
voltrondatabot commented on PR #47294:
URL: https://github.com/apache/arrow/pull/47294#issuecomment-3299478150
Benchmark runs are scheduled for commit 3537727814ce5d3d795d8084f518c2704187ff1e. Watch https://buildkite.com/apache-arrow and https://conbench.ursa.dev for updates. A comment will be posted here when the runs are complete.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2349117516
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -602,20 +1033,106 @@ struct GetBatchSpacedTestCase {
int bit_width;
};
-TEST(RleDecoder, GetBatchSpaced) {
- uint32_t kSeed = 1337;
- ::arrow::random::RandomArrayGenerator rand(kSeed);
-
- std::vector<GetBatchSpacedTestCase<int32_t>> int32_cases{
- {1, 100000, 0.01, 1}, {1, 100000, 0.1, 1}, {1, 100000, 0.5, 1},
- {4, 100000, 0.05, 3}, {100, 100000, 0.05, 7},
+template <typename T>
+void DoTestGetBatchSpacedRoundtrip() {
+ using Data = DataTestRleBitPacked<T>;
+ using ArrowType = typename Data::ArrowType;
+ using RandomPart = typename Data::RandomPart;
+ using NullPart = typename Data::NullPart;
+ using RepeatPart = typename Data::RepeatPart;
+
+ std::vector<Data> test_cases = {
+ {
+ {RandomPart{/* max=*/1, /* size=*/400, /* null_proba= */ 0.1}},
+ /* bit_width= */ 1,
+ },
+ {
+ {
+ RandomPart{/* max=*/7, /* size=*/10037, /* null_proba= */ 0.1},
+ NullPart{/* size= */ 1153},
+ RandomPart{/* max=*/7, /* size=*/800, /* null_proba= */ 0.5},
+ },
+ /* bit_width= */ 3,
+ },
+ {
+ {
+ NullPart{/* size= */ 80},
+ RandomPart{/* max=*/1023, /* size=*/800, /* null_proba= */ 0.01},
+ NullPart{/* size= */ 1023},
+ },
+ /* bit_width= */ 11,
+ },
+ {
+ {RepeatPart{/* value=*/13, /* size=*/100000}},
+ /* bit_width= */ 10,
+ },
+ {
+ {
+ NullPart{/* size= */ 1024},
+ RepeatPart{/* value=*/10000, /* size=*/100000},
+ NullPart{/* size= */ 77},
+ },
+ /* bit_width= */ 23,
+ },
+ {
+ {
+ RepeatPart{/* value=*/13, /* size=*/100000},
+ NullPart{/* size= */ 1153},
+ RepeatPart{/* value=*/72, /* size=*/100799},
+ },
+ /* bit_width= */ 10,
+ },
+ {
+ {
+ RandomPart{/* max=*/1, /* size=*/1013, /* null_proba= */ 0.01},
+ NullPart{/* size=*/8},
+ RepeatPart{1, /* size= */ 256},
+ NullPart{/* size=*/128},
+ RepeatPart{0, /* size= */ 256},
+ NullPart{/* size=*/15},
+ RandomPart{/* max=*/1, /* size=*/8 * 1024, /* null_proba= */ 0.01},
+ },
+ /* bit_width= */ 1,
+ },
};
- for (auto case_ : int32_cases) {
- auto arr = rand.Int32(case_.size, /*min=*/0, case_.max_value, case_.null_probability);
- CheckRoundTripSpaced<Int32Type>(*arr, case_.bit_width);
- CheckRoundTripSpaced<Int32Type>(*arr->Slice(1), case_.bit_width);
+
+ ::arrow::random::RandomArrayGenerator rand(/* seed= */ 12);
+ // FRAGILE: Large enough so that it can be indexed by any value in all cases
+ auto dict = std::static_pointer_cast<arrow::FloatArray>(rand.Float32(20000, -1.0, 1.0));
+
+ for (auto case_ : test_cases) {
+ if (static_cast<std::size_t>(case_.bit_width) > sizeof(T)) {
+ continue;
+ }
+
+ auto array = case_.MakeArray(rand);
+
+ // Tests for GetBatch
+ CheckRoundTrip<ArrowType, false, 1>(*array, case_.bit_width);
+ CheckRoundTrip<ArrowType, false, 3>(*array, case_.bit_width);
+
+ // Tests for GetBatchSpaced
+ CheckRoundTrip<ArrowType, true, 1>(*array, case_.bit_width);
+ CheckRoundTrip<ArrowType, true, 7>(*array, case_.bit_width);
+ CheckRoundTrip<ArrowType, true, 1>(*array->Slice(1), case_.bit_width);
+
+ // Cannot test GetBatchWithDict with this method since unknown null values
+
+ // Tests for GetBatchWithDictSpaced
+ CheckRoundTrip<ArrowType, true, 1>(*array, case_.bit_width, dict);
+ CheckRoundTrip<ArrowType, true, 5>(*array, case_.bit_width, dict);
}
}
+TEST(RleBitPacked, GetBatchSpacedRoundtripUint16) {
+ DoTestGetBatchSpacedRoundtrip<uint16_t>();
Review Comment:
I added `uint8_t`. We already have a test for `int32_t` here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2352588002
##########
cpp/src/arrow/util/bit_util_test.cc:
##########
@@ -1997,11 +1997,191 @@ TEST(BitUtil, RoundUpToPowerOf2) {
#undef U64
#undef S64
+/// Test the maximum number of bytes needed to write a LEB128 of a give size.
+TEST(LEB128, MaxLEB128ByteLenFor) {
+ EXPECT_EQ(bit_util::kMaxLEB128ByteLenFor<int16_t>, 3);
+ EXPECT_EQ(bit_util::kMaxLEB128ByteLenFor<int32_t>, 5);
+ EXPECT_EQ(bit_util::kMaxLEB128ByteLenFor<int64_t>, 10);
+}
+
+/// Utility function to test LEB128 encoding with known input value and expected byte
+/// array
+template <typename Int>
+void TestLEB128Encode(Int input_value, const std::vector<uint8_t>& expected_data,
+ std::size_t buffer_size) {
+ std::vector<uint8_t> buffer(buffer_size);
+ auto bytes_written = bit_util::WriteLEB128(input_value, buffer.data(),
+ static_cast<int32_t>(buffer.size()));
+
+ EXPECT_EQ(bytes_written, expected_data.size());
+ // Encoded data
+ for (std::size_t i = 0; i < expected_data.size(); ++i) {
+ EXPECT_EQ(buffer.at(i), expected_data.at(i));
+ }
+
+ // When the value is successfully encoded, the remaining of the buffer is untouched
+ if (bytes_written > 0) {
+ for (std::size_t i = bytes_written; i < buffer.size(); ++i) {
+ EXPECT_EQ(buffer.at(i), 0);
+ }
+ }
+}
+
+/// Test encoding to known LEB128 byte sequences with edge cases parameters.
+/// \see LEB128.KnownSuccessfulValues for other known values tested.
+TEST(LEB128, WriteEdgeCases) {
+ // Single byte value 0
+ TestLEB128Encode(0U, {0x00}, 1);
+ // Single byte value 127
+ TestLEB128Encode(127U, {0x7F}, 1);
+ // Three byte value 16384, encoded in larger buffer
+ TestLEB128Encode(16384U, {0x80, 0x80, 0x01}, 10);
+ // Two byte boundary values
Review Comment:
128U would be the actual boundary value, right? Perhaps test it too?
##########
cpp/src/arrow/util/bit_util_test.cc:
##########
@@ -1997,11 +1997,191 @@ TEST(BitUtil, RoundUpToPowerOf2) {
#undef U64
#undef S64
+/// Test the maximum number of bytes needed to write a LEB128 of a give size.
+TEST(LEB128, MaxLEB128ByteLenFor) {
+ EXPECT_EQ(bit_util::kMaxLEB128ByteLenFor<int16_t>, 3);
+ EXPECT_EQ(bit_util::kMaxLEB128ByteLenFor<int32_t>, 5);
+ EXPECT_EQ(bit_util::kMaxLEB128ByteLenFor<int64_t>, 10);
Review Comment:
Also test the unsigned types?
##########
cpp/src/arrow/util/bit_util_test.cc:
##########
@@ -1997,11 +1997,191 @@ TEST(BitUtil, RoundUpToPowerOf2) {
#undef U64
#undef S64
+/// Test the maximum number of bytes needed to write a LEB128 of a give size.
+TEST(LEB128, MaxLEB128ByteLenFor) {
+ EXPECT_EQ(bit_util::kMaxLEB128ByteLenFor<int16_t>, 3);
+ EXPECT_EQ(bit_util::kMaxLEB128ByteLenFor<int32_t>, 5);
+ EXPECT_EQ(bit_util::kMaxLEB128ByteLenFor<int64_t>, 10);
+}
+
+/// Utility function to test LEB128 encoding with known input value and expected byte
+/// array
+template <typename Int>
+void TestLEB128Encode(Int input_value, const std::vector<uint8_t>& expected_data,
+ std::size_t buffer_size) {
+ std::vector<uint8_t> buffer(buffer_size);
+ auto bytes_written = bit_util::WriteLEB128(input_value, buffer.data(),
+ static_cast<int32_t>(buffer.size()));
+
+ EXPECT_EQ(bytes_written, expected_data.size());
+ // Encoded data
+ for (std::size_t i = 0; i < expected_data.size(); ++i) {
+ EXPECT_EQ(buffer.at(i), expected_data.at(i));
+ }
+
+ // When the value is successfully encoded, the remaining of the buffer is untouched
+ if (bytes_written > 0) {
+ for (std::size_t i = bytes_written; i < buffer.size(); ++i) {
+ EXPECT_EQ(buffer.at(i), 0);
+ }
+ }
+}
+
+/// Test encoding to known LEB128 byte sequences with edge cases parameters.
+/// \see LEB128.KnownSuccessfulValues for other known values tested.
+TEST(LEB128, WriteEdgeCases) {
+ // Single byte value 0
+ TestLEB128Encode(0U, {0x00}, 1);
+ // Single byte value 127
+ TestLEB128Encode(127U, {0x7F}, 1);
+ // Three byte value 16384, encoded in larger buffer
+ TestLEB128Encode(16384U, {0x80, 0x80, 0x01}, 10);
+ // Two byte boundary values
+ TestLEB128Encode(129U, {0x81, 0x01}, 2);
+ TestLEB128Encode(16383U, {0xFF, 0x7F}, 2);
+ // Error case: Buffer too small for value 128 (needs 2 bytes but only 1 provided)
+ TestLEB128Encode(128U, {}, 1);
+ // Error case: Buffer too small for uint32_t max (needs 5 bytes but only 4 provided)
+ TestLEB128Encode(4294967295U, {}, 4);
+ // Error case: Zero buffer size
+ TestLEB128Encode(52U, {}, 0);
+ // Error case: Negative value
+ TestLEB128Encode(-3, {}, 1);
+}
+
+/// Utility function to test LEB128 decoding with known byte array and expected result
+template <typename Int>
+void TestLEB128Decode(const std::vector<uint8_t>& data, Int expected_value,
+ int32_t expected_bytes_read) {
+ Int result = 0;
+ auto bytes_read = bit_util::ParseLeadingLEB128(
+ data.data(), static_cast<int32_t>(data.size()), &result);
+ EXPECT_EQ(bytes_read, expected_bytes_read);
+ if (expected_bytes_read > 0) {
+ EXPECT_EQ(result, expected_value);
+ }
+}
+
+template <typename Int>
+void TestLEB128Decode(const std::vector<uint8_t>& data, Int expected_value,
+ std::size_t expected_bytes_read) {
Review Comment:
Do we really need this override? It doesn't seem very useful to me.
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -553,47 +853,183 @@ TEST(BitRle, Overflow) {
}
}
+/// Check RleBitPacked encoding/decoding round trip.
+///
+/// \param spaced If set to false, treat Nulls in the input array as regular data.
+/// \param parts The number of parts in which the data will be decoded.
+/// For number greater than one, this ensure that the decoder intermediary state
+/// is valid.
template <typename Type>
-void CheckRoundTripSpaced(const Array& data, int bit_width) {
+void CheckRoundTrip(const Array& data, int bit_width, bool spaced, int32_t parts,
+ std::shared_ptr<FloatArray> dict = {}) {
using ArrayType = typename TypeTraits<Type>::ArrayType;
- using T = typename Type::c_type;
+ using value_type = typename Type::c_type;
- int num_values = static_cast<int>(data.length());
- int buffer_size = RleEncoder::MaxBufferSize(bit_width, num_values);
+ const int data_size = static_cast<int>(data.length());
+ const int data_values_count =
+ static_cast<int>(data.length() - spaced * data.null_count());
+ const int buffer_size = RleBitPackedEncoder::MaxBufferSize(bit_width, data_size);
+ ASSERT_GE(parts, 1);
+ ASSERT_LE(parts, data_size);
- const T* values = static_cast<const ArrayType&>(data).raw_values();
+ const value_type* data_values = static_cast<const ArrayType&>(data).raw_values();
+ // Encode the data into `buffer` using the encoder.
std::vector<uint8_t> buffer(buffer_size);
- RleEncoder encoder(buffer.data(), buffer_size, bit_width);
- for (int i = 0; i < num_values; ++i) {
- if (data.IsValid(i)) {
- if (!encoder.Put(static_cast<uint64_t>(values[i]))) {
- FAIL() << "Encoding failed";
- }
+ RleBitPackedEncoder encoder(buffer.data(), buffer_size, bit_width);
+ int32_t encoded_values_size = 0;
+ for (int i = 0; i < data_size; ++i) {
+ // Depending on `spaced` we treat nulls as regular values.
+ if (data.IsValid(i) || !spaced) {
+ bool success = encoder.Put(static_cast<uint64_t>(data_values[i]));
+ ASSERT_TRUE(success) << "Encoding failed in pos " << i;
+ ++encoded_values_size;
}
}
- int encoded_size = encoder.Flush();
+ int encoded_byte_size = encoder.Flush();
+ ASSERT_EQ(encoded_values_size, data_values_count)
+ << "All values input were not encoded successfully by the encoder";
+
+ // Now we verify batch read
+ RleBitPackedDecoder<value_type> decoder(buffer.data(), encoded_byte_size, bit_width);
+ // We will only use one of them depending on whether this is a dictonnary tests
+ std::vector<float> dict_read;
+ std::vector<value_type> values_read;
+ if (dict) {
+ dict_read.resize(data_size);
+ } else {
+ values_read.resize(data_size);
+ }
- // Verify batch read
- RleDecoder decoder(buffer.data(), encoded_size, bit_width);
- std::vector<T> values_read(num_values);
+ // We will read the data in `parts` calls to make sure intermediate states are valid
+ int32_t actual_read_count = 0;
+ int32_t requested_read_count = 0;
+ while (requested_read_count < data_size) {
+ const auto remaining = data_size - requested_read_count;
+ auto to_read = data_size / parts;
+ if (remaining / to_read == 1) {
+ to_read = remaining;
+ }
- if (num_values != decoder.GetBatchSpaced(
- num_values, static_cast<int>(data.null_count()),
- data.null_bitmap_data(), data.offset(), values_read.data())) {
- FAIL();
- }
+ auto read = 0;
+ if (spaced) {
+ // We need to slice the input array get the proper null count and bitmap
+ auto data_remaining = data.Slice(requested_read_count, to_read);
+
+ if (dict) {
+ auto* out = dict_read.data() + requested_read_count;
+ read = decoder.GetBatchWithDictSpaced(
+ dict->raw_values(), static_cast<int32_t>(dict->length()), out, to_read,
+ static_cast<int32_t>(data_remaining->null_count()),
+ data_remaining->null_bitmap_data(), data_remaining->offset());
+ } else {
+ auto* out = values_read.data() + requested_read_count;
+ read = decoder.GetBatchSpaced(
+ to_read, static_cast<int32_t>(data_remaining->null_count()),
+ data_remaining->null_bitmap_data(), data_remaining->offset(), out);
+ }
+ } else {
+ if (dict) {
+ auto* out = dict_read.data() + requested_read_count;
+ read = decoder.GetBatchWithDict(
+ dict->raw_values(), static_cast<int32_t>(dict->length()), out, to_read);
+ } else {
+ auto* out = values_read.data() + requested_read_count;
+ read = decoder.GetBatch(out, to_read);
+ }
+ }
+ ASSERT_EQ(read, to_read) << "Decoder did not read as many values as requested";
- for (int64_t i = 0; i < num_values; ++i) {
- if (data.IsValid(i)) {
- if (values_read[i] != values[i]) {
- FAIL() << "Index " << i << " read " << values_read[i] << " but should be "
- << values[i];
+ actual_read_count += read;
+ requested_read_count += to_read;
+ }
+ EXPECT_EQ(requested_read_count, data_size) << "This test logic is wrong";
+ EXPECT_EQ(actual_read_count, data_size) << "Total number of values read is off";
+
+ // Verify the round trip: encoded-decoded values must equal the original one
+ for (int64_t i = 0; i < data_size; ++i) {
+ if (data.IsValid(i) || !spaced) {
+ if (dict) {
+ EXPECT_EQ(dict_read.at(i), dict->Value(data_values[i]))
+ << "Encoded then decoded and mapped value at position " << i << " ("
+ << values_read[i] << ") differs from original value (" << data_values[i]
+ << " mapped to " << dict->Value(data_values[i]) << ")";
+ } else {
+ EXPECT_EQ(values_read.at(i), data_values[i])
+ << "Encoded then decoded value at position " << i << " (" << values_read.at(i)
+ << ") differs from original value (" << data_values[i] << ")";
}
}
}
}
+template <typename T>
+struct DataTestRleBitPackedRandomPart {
+ using value_type = T;
+
+ value_type max;
+ int32_t size;
+ double null_probability;
+};
+
+template <typename T>
+struct DataTestRleBitPackedRepeatPart {
+ using value_type = T;
+
+ value_type value;
+ int32_t size;
+ double null_probability;
+};
+
+template <typename T>
+struct DataTestRleBitPackedNullPart {
+ using value_type = T;
+
+ int32_t size;
+};
+
+template <typename T>
+struct DataTestRleBitPacked {
+ using value_type = T;
+ using ArrowType = typename arrow::CTypeTraits<value_type>::ArrowType;
+ using RandomPart = DataTestRleBitPackedRandomPart<value_type>;
+ using RepeatPart = DataTestRleBitPackedRepeatPart<value_type>;
+ using NullPart = DataTestRleBitPackedNullPart<value_type>;
+ using AnyPart = std::variant<RandomPart, RepeatPart, NullPart>;
+
+ std::vector<AnyPart> parts;
+ int32_t bit_width;
+
+ std::shared_ptr<::arrow::Array> MakeArray(
+ ::arrow::random::RandomArrayGenerator& rand) const {
+ using Traits = arrow::TypeTraits<ArrowType>;
+
+ std::vector<std::shared_ptr<::arrow::Array>> arrays = {};
+
+ for (const auto& dyn_part : parts) {
+ if (auto* part = std::get_if<RandomPart>(&dyn_part)) {
+ auto arr = rand.Numeric<ArrowType>(part->size, /* min= */ value_type(0),
+ part->max, part->null_probability);
+ arrays.push_back(std::move(arr));
+
+ } else if (auto* part = std::get_if<RepeatPart>(&dyn_part)) {
+ auto arr =
+ rand.Numeric<ArrowType>(part->size, /* min= */ part->value,
+ /* max= */ part->value, part->null_probability);
+ arrays.push_back(std::move(arr));
+
+ } else if (auto* part = std::get_if<NullPart>(&dyn_part)) {
+ EXPECT_OK_AND_ASSIGN(
+ auto arr, ::arrow::MakeArrayOfNull(Traits::type_singleton(), part->size));
+ arrays.push_back(std::move(arr));
+ }
+ }
+ ARROW_DCHECK_EQ(parts.size(), arrays.size());
+
+ return ::arrow::Concatenate(arrays).ValueOrDie();
+ }
+};
+
template <typename T>
struct GetBatchSpacedTestCase {
Review Comment:
Is this one still used?
##########
cpp/src/arrow/util/bit_stream_utils_internal.h:
##########
@@ -439,91 +424,84 @@ inline bool BitReader::Advance(int64_t num_bits) {
return true;
}
-inline bool BitWriter::PutVlqInt(uint32_t v) {
- bool result = true;
- while ((v & 0xFFFFFF80UL) != 0UL) {
- result &= PutAligned<uint8_t>(static_cast<uint8_t>((v & 0x7F) | 0x80), 1);
- v >>= 7;
- }
- result &= PutAligned<uint8_t>(static_cast<uint8_t>(v & 0x7F), 1);
- return result;
-}
+template <typename Int>
+inline bool BitWriter::PutVlqInt(Int v) {
+ static_assert(std::is_integral_v<Int>);
Review Comment:
Same here: assert it's unsigned?
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -84,32 +83,279 @@ namespace util {
/// (total 26 bytes, 1 byte overhead)
//
-/// Decoder class for RLE encoded data.
-class RleDecoder {
+template <typename T>
+class RleRunDecoder;
+
+class RleRun {
public:
- /// Create a decoder object. buffer/buffer_len is the decoded data.
- /// bit_width is the width of each value (before encoding).
- RleDecoder(const uint8_t* buffer, int buffer_len, int bit_width)
- : bit_reader_(buffer, buffer_len),
- bit_width_(bit_width),
- current_value_(0),
- repeat_count_(0),
- literal_count_(0) {
- ARROW_DCHECK_GE(bit_width_, 0);
- ARROW_DCHECK_LE(bit_width_, 64);
- }
+ /// Enough space to store a 64bit value
+ using raw_data_storage = std::array<uint8_t, 8>;
+ using raw_data_const_pointer = const uint8_t*;
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ /// The decoder class used to decode a single run in the given type.
+ template <typename T>
+ using DecoderType = RleRunDecoder<T>;
- RleDecoder() : bit_width_(-1) {}
+ constexpr RleRun() noexcept = default;
+ constexpr RleRun(const RleRun&) noexcept = default;
+ constexpr RleRun(RleRun&&) noexcept = default;
- void Reset(const uint8_t* buffer, int buffer_len, int bit_width) {
- ARROW_DCHECK_GE(bit_width, 0);
- ARROW_DCHECK_LE(bit_width, 64);
- bit_reader_.Reset(buffer, buffer_len);
- bit_width_ = bit_width;
- current_value_ = 0;
- repeat_count_ = 0;
- literal_count_ = 0;
- }
+ explicit RleRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr RleRun& operator=(const RleRun&) noexcept = default;
+ constexpr RleRun& operator=(RleRun&&) noexcept = default;
+
+ /// The number of repeated values in this run.
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
Review Comment:
Again, `[[nodiscard]]` does not seem useful on such simple accessor methods. Nothing dangerous happens if you ignore the result...
##########
cpp/src/parquet/decoder.cc:
##########
@@ -1898,7 +1900,7 @@ class RleBooleanDecoder : public TypedDecoderImpl<BooleanType>, public BooleanDe
}
private:
- std::shared_ptr<::arrow::util::RleDecoder> decoder_;
+ std::shared_ptr<::arrow::util::RleBitPackedDecoder<bool>> decoder_;
Review Comment:
Fair enough.
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -207,12 +209,310 @@ TEST(BitUtil, RoundTripIntValues) {
}
}
+/// A Rle run is a simple class owning some data and a repetition count.
+/// It does not know how to read such data.
+TEST(Rle, RleRun) {
+ const std::array<RleRun::byte, 4> value = {21, 2, 0, 0};
+
+ RleRun::values_count_type value_count = 12;
+
+ // 12 times the value 21 fitting over 5 bits
+ auto const run_5 = RleRun(value.data(), value_count, /* value_bit_width= */ 5);
+ EXPECT_EQ(run_5.ValuesCount(), value_count);
+ EXPECT_EQ(run_5.ValuesBitWidth(), 5);
+ EXPECT_EQ(run_5.RawDataSize(), 1); // 5 bits fit in one byte
+ EXPECT_EQ(*run_5.RawDataPtr(), 21);
+
+ // 12 times the value 21 fitting over 16 bits
+ auto const run_8 = RleRun(value.data(), value_count, /* value_bit_width= */ 8);
+ EXPECT_EQ(run_8.ValuesCount(), value_count);
+ EXPECT_EQ(run_8.ValuesBitWidth(), 8);
+ EXPECT_EQ(run_8.RawDataSize(), 1); // 8 bits fit in 1 byte
+ EXPECT_EQ(*run_8.RawDataPtr(), 21);
+
+ // 12 times the value {21, 2} fitting over 10 bits
+ auto const run_10 = RleRun(value.data(), value_count, /* value_bit_width= */ 10);
+
+ EXPECT_EQ(run_10.ValuesCount(), value_count);
+ EXPECT_EQ(run_10.ValuesBitWidth(), 10);
+ EXPECT_EQ(run_10.RawDataSize(), 2); // 10 bits fit in 2 bytes
+ EXPECT_EQ(*(run_10.RawDataPtr() + 0), 21);
+ EXPECT_EQ(*(run_10.RawDataPtr() + 1), 2);
+
+ // 12 times the value {21, 2} fitting over 32 bits
+ auto const run_32 = RleRun(value.data(), value_count, /* value_bit_width= */ 32);
+ EXPECT_EQ(run_32.ValuesCount(), value_count);
+ EXPECT_EQ(run_32.ValuesBitWidth(), 32);
+ EXPECT_EQ(run_32.RawDataSize(), 4); // 32 bits fit in 4 bytes
+ EXPECT_EQ(*(run_32.RawDataPtr() + 0), 21);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 1), 2);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 2), 0);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 3), 0);
+}
+
+/// A BitPacked run is a simple class owning some data and its size.
+/// It does not know how to read such data.
+TEST(BitPacked, BitPackedRun) {
+ const std::array<BitPackedRun::byte, 4> value = {0b10101010, 0, 0, 0b1111111};
+
+ /// 16 values of 1 bit for a total of 16 bits
+ BitPackedRun::values_count_type value_count_1 = 16;
+ auto const run_1 = BitPackedRun(value.data(), value_count_1, /* value_bit_width= */ 1);
+ EXPECT_EQ(run_1.ValuesCount(), value_count_1);
+ EXPECT_EQ(run_1.ValuesBitWidth(), 1);
+ EXPECT_EQ(run_1.RawDataSize(), 2); // 16 bits fit in 2 bytes
+ for (BitPackedRun::raw_data_size_type i = 0; i < run_1.RawDataSize(); ++i) {
+ EXPECT_EQ(*(run_1.RawDataPtr() + i), value[i]);
Review Comment:
Hmm, I'm still wondering about this (see question above).
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ ARROW_DCHECK_LT(values_read, batch_size);
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+ auto const read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
return values_read;
}
-template <typename T, typename RunType, typename Converter>
-inline int RleDecoder::GetSpaced(Converter converter, int batch_size, int null_count,
- const uint8_t* valid_bits, int64_t valid_bits_offset,
- T* out) {
- if (ARROW_PREDICT_FALSE(null_count == batch_size)) {
- converter.FillZero(out, out + batch_size);
- return batch_size;
+namespace internal {
+
+/// Utility class to safely handle values and null count without too error-prone
+/// verbosity.
+class BatchCounter {
+ public:
+ using size_type = int32_t;
+
+ [[nodiscard]] static constexpr BatchCounter FromBatchSizeAndNulls(
+ size_type batch_size, size_type null_count) {
+ ARROW_DCHECK_LE(null_count, batch_size);
+ return {batch_size - null_count, null_count};
}
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
- int values_remaining = batch_size - null_count;
+ constexpr BatchCounter(size_type values_count, size_type null_count) noexcept
+ : values_count_(values_count), null_count_(null_count) {}
- // Assume no bits to start.
- arrow::internal::BitRunReader bit_reader(valid_bits, valid_bits_offset,
- /*length=*/batch_size);
- arrow::internal::BitRun valid_run = bit_reader.NextRun();
- while (values_read < batch_size) {
- if (ARROW_PREDICT_FALSE(valid_run.length == 0)) {
- valid_run = bit_reader.NextRun();
+ [[nodiscard]] constexpr size_type ValuesCount() const noexcept { return values_count_; }
+
+ [[nodiscard]] constexpr size_type ValuesRead() const noexcept { return values_read_; }
+
+ [[nodiscard]] constexpr size_type ValuesRemaining() const noexcept {
+ ARROW_DCHECK_LE(values_read_, values_count_);
+ return values_count_ - values_read_;
+ }
+
+ constexpr void AccrueReadValues(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, ValuesRemaining());
+ values_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type NullCount() const noexcept { return null_count_; }
+
+ [[nodiscard]] constexpr size_type NullRead() const noexcept { return null_read_; }
+
+ [[nodiscard]] constexpr size_type NullRemaining() const noexcept {
+ ARROW_DCHECK_LE(null_read_, null_count_);
+ return null_count_ - null_read_;
+ }
+
+ constexpr void AccrueReadNulls(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, NullRemaining());
+ null_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type TotalRemaining() const noexcept {
+ return ValuesRemaining() + NullRemaining();
+ }
+
+ [[nodiscard]] constexpr size_type TotalRead() const noexcept {
+ return values_read_ + null_read_;
+ }
+
+ [[nodiscard]] constexpr bool IsFullyNull() const noexcept {
+ return ValuesRemaining() == 0;
+ }
+
+ [[nodiscard]] constexpr bool IsDone() const noexcept { return TotalRemaining() == 0; }
+
+ private:
+ size_type values_count_ = 0;
+ size_type values_read_ = 0;
+ size_type null_count_ = 0;
+ size_type null_read_ = 0;
+};
+
+// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+/// Overload for GetSpaced for a single run in a RleDecoder
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ RleDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto values_remaining_run = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // Consume as much as possible from the repeated run.
+ // We only need to count the number of nulls and non-nulls because we can fill in the
+ // same value for nulls and non-nulls.
+ // This proves to be a big efficiency win.
+ while (values_remaining_run() > 0 && !batch.IsDone()) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const& validity_run_size = static_cast<values_count_type>(validity_run.length);
+
+ if (validity_run.set) {
+ // We may end the current RLE run in the middle of the validity run
+ auto update_size = std::min(validity_run_size, values_remaining_run());
+ batch.AccrueReadValues(update_size);
+ validity_run.length -= update_size;
+ } else {
+ // We can consume all nulls here because it does not matter if we consume on this
+ // RLE run, or an a next encoded run. The value filled does not matter.
+ auto update_size = std::min(validity_run_size, batch.NullRemaining());
+ batch.AccrueReadNulls(update_size);
+ validity_run.length -= update_size;
+ }
+
+ if (ARROW_PREDICT_TRUE(validity_run.length == 0)) {
+ validity_run = validity_reader.NextRun();
}
+ }
+
+ value_type const value = decoder.Value();
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(value))) {
+ return {0, 0};
+ }
+ converter.WriteRepeated(out, out + batch.TotalRead(), value);
+ auto const actual_values_read = decoder.Advance(batch.ValuesRead());
+ // We always cropped the number of values_read by the remaining values in the run.
+ // What's more the RLE decoder should not encounter any errors.
+ ARROW_DCHECK_EQ(actual_values_read, batch.ValuesRead());
- ARROW_DCHECK_GT(batch_size, 0);
- ARROW_DCHECK_GT(valid_run.length, 0);
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+template <typename T, typename... Ts>
+[[nodiscard]] constexpr T min(T x, Ts... ys) {
+ ((x = std::min(x, ys)), ...);
+ return x;
+}
+
+static_assert(min(5) == 5);
+static_assert(min(5, 4, -1) == -1);
+static_assert(min(5, 41) == 5);
+
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ BitPackedDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto run_values_remaining = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ while (run_values_remaining() > 0 && batch.ValuesRemaining() > 0) {
+ // TODO should this size be tune depending on sizeof(value_size)? cpu cache size?
+ // Pull a batch of values from the bit packed encoded data and store it in a local
+ // buffer to benefit from unpacking intrinsics and data locality.
+ static constexpr values_count_type kBufferCapacity = 1024;
+ std::array<value_type, kBufferCapacity> buffer = {};
+
+ values_count_type buffer_start = 0;
+ values_count_type buffer_end = 0;
+ auto buffer_size = [&]() {
+ auto out = buffer_end - buffer_start;
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // buffer_start is 0 at this point so size is end
+ buffer_end = min(run_values_remaining(), batch.ValuesRemaining(), kBufferCapacity);
+ buffer_end = decoder.GetBatch(buffer.data(), buffer_size());
+ ARROW_DCHECK_LE(buffer_size(), kBufferCapacity);
+
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(buffer.data(), buffer_size()))) {
+ return {batch.ValuesRead(), batch.NullRead()};
Review Comment:
If easily doable then yes.
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -553,47 +853,183 @@ TEST(BitRle, Overflow) {
}
}
+/// Check RleBitPacked encoding/decoding round trip.
+///
+/// \param spaced If set to false, treat Nulls in the input array as regular data.
+/// \param parts The number of parts in which the data will be decoded.
+/// For number greater than one, this ensure that the decoder intermediary state
Review Comment:
```suggestion
/// For number greater than one, this ensure that the decoder intermediate state
```
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -207,12 +209,310 @@ TEST(BitUtil, RoundTripIntValues) {
}
}
+/// A Rle run is a simple class owning some data and a repetition count.
+/// It does not know how to read such data.
+TEST(Rle, RleRun) {
+ const std::array<uint8_t, 4> value = {21, 2, 0, 0};
+
+ RleRun::values_count_type value_count = 12;
+
+ // 12 times the value 21 fitting over 5 bits
+ const auto run_5 = RleRun(value.data(), value_count, /* value_bit_width= */ 5);
+ EXPECT_EQ(run_5.ValuesCount(), value_count);
+ EXPECT_EQ(run_5.ValuesBitWidth(), 5);
+ EXPECT_EQ(run_5.RawDataSize(), 1); // 5 bits fit in one byte
+ EXPECT_EQ(*run_5.RawDataPtr(), 21);
+
+ // 12 times the value 21 fitting over 16 bits
+ const auto run_8 = RleRun(value.data(), value_count, /* value_bit_width= */ 8);
+ EXPECT_EQ(run_8.ValuesCount(), value_count);
+ EXPECT_EQ(run_8.ValuesBitWidth(), 8);
+ EXPECT_EQ(run_8.RawDataSize(), 1); // 8 bits fit in 1 byte
+ EXPECT_EQ(*run_8.RawDataPtr(), 21);
+
+ // 12 times the value {21, 2} fitting over 10 bits
Review Comment:
You mean the value 533?
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -553,47 +853,183 @@ TEST(BitRle, Overflow) {
}
}
+/// Check RleBitPacked encoding/decoding round trip.
+///
+/// \param spaced If set to false, treat Nulls in the input array as regular data.
+/// \param parts The number of parts in which the data will be decoded.
+/// For number greater than one, this ensure that the decoder intermediary state
+/// is valid.
template <typename Type>
-void CheckRoundTripSpaced(const Array& data, int bit_width) {
+void CheckRoundTrip(const Array& data, int bit_width, bool spaced, int32_t parts,
+ std::shared_ptr<FloatArray> dict = {}) {
using ArrayType = typename TypeTraits<Type>::ArrayType;
- using T = typename Type::c_type;
+ using value_type = typename Type::c_type;
- int num_values = static_cast<int>(data.length());
- int buffer_size = RleEncoder::MaxBufferSize(bit_width, num_values);
+ const int data_size = static_cast<int>(data.length());
+ const int data_values_count =
+ static_cast<int>(data.length() - spaced * data.null_count());
+ const int buffer_size = RleBitPackedEncoder::MaxBufferSize(bit_width, data_size);
+ ASSERT_GE(parts, 1);
+ ASSERT_LE(parts, data_size);
- const T* values = static_cast<const ArrayType&>(data).raw_values();
+ const value_type* data_values = static_cast<const ArrayType&>(data).raw_values();
+ // Encode the data into `buffer` using the encoder.
std::vector<uint8_t> buffer(buffer_size);
- RleEncoder encoder(buffer.data(), buffer_size, bit_width);
- for (int i = 0; i < num_values; ++i) {
- if (data.IsValid(i)) {
- if (!encoder.Put(static_cast<uint64_t>(values[i]))) {
- FAIL() << "Encoding failed";
- }
+ RleBitPackedEncoder encoder(buffer.data(), buffer_size, bit_width);
+ int32_t encoded_values_size = 0;
+ for (int i = 0; i < data_size; ++i) {
+ // Depending on `spaced` we treat nulls as regular values.
+ if (data.IsValid(i) || !spaced) {
+ bool success = encoder.Put(static_cast<uint64_t>(data_values[i]));
+ ASSERT_TRUE(success) << "Encoding failed in pos " << i;
+ ++encoded_values_size;
}
}
- int encoded_size = encoder.Flush();
+ int encoded_byte_size = encoder.Flush();
+ ASSERT_EQ(encoded_values_size, data_values_count)
+ << "All values input were not encoded successfully by the encoder";
+
+ // Now we verify batch read
+ RleBitPackedDecoder<value_type> decoder(buffer.data(), encoded_byte_size, bit_width);
+ // We will only use one of them depending on whether this is a dictonnary tests
+ std::vector<float> dict_read;
+ std::vector<value_type> values_read;
+ if (dict) {
+ dict_read.resize(data_size);
+ } else {
+ values_read.resize(data_size);
+ }
- // Verify batch read
- RleDecoder decoder(buffer.data(), encoded_size, bit_width);
- std::vector<T> values_read(num_values);
+ // We will read the data in `parts` calls to make sure intermediate states are valid
+ int32_t actual_read_count = 0;
+ int32_t requested_read_count = 0;
+ while (requested_read_count < data_size) {
+ const auto remaining = data_size - requested_read_count;
+ auto to_read = data_size / parts;
+ if (remaining / to_read == 1) {
+ to_read = remaining;
+ }
- if (num_values != decoder.GetBatchSpaced(
- num_values, static_cast<int>(data.null_count()),
- data.null_bitmap_data(), data.offset(), values_read.data())) {
- FAIL();
- }
+ auto read = 0;
Review Comment:
Let's put the type (aka `int` or `int32_t`) explicitly here?
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -553,47 +853,183 @@ TEST(BitRle, Overflow) {
}
}
+/// Check RleBitPacked encoding/decoding round trip.
+///
+/// \param spaced If set to false, treat Nulls in the input array as regular data.
+/// \param parts The number of parts in which the data will be decoded.
+/// For number greater than one, this ensure that the decoder intermediary state
+/// is valid.
template <typename Type>
-void CheckRoundTripSpaced(const Array& data, int bit_width) {
+void CheckRoundTrip(const Array& data, int bit_width, bool spaced, int32_t parts,
+ std::shared_ptr<FloatArray> dict = {}) {
using ArrayType = typename TypeTraits<Type>::ArrayType;
- using T = typename Type::c_type;
+ using value_type = typename Type::c_type;
- int num_values = static_cast<int>(data.length());
- int buffer_size = RleEncoder::MaxBufferSize(bit_width, num_values);
+ const int data_size = static_cast<int>(data.length());
+ const int data_values_count =
+ static_cast<int>(data.length() - spaced * data.null_count());
+ const int buffer_size = RleBitPackedEncoder::MaxBufferSize(bit_width, data_size);
+ ASSERT_GE(parts, 1);
+ ASSERT_LE(parts, data_size);
- const T* values = static_cast<const ArrayType&>(data).raw_values();
+ const value_type* data_values = static_cast<const ArrayType&>(data).raw_values();
+ // Encode the data into `buffer` using the encoder.
std::vector<uint8_t> buffer(buffer_size);
- RleEncoder encoder(buffer.data(), buffer_size, bit_width);
- for (int i = 0; i < num_values; ++i) {
- if (data.IsValid(i)) {
- if (!encoder.Put(static_cast<uint64_t>(values[i]))) {
- FAIL() << "Encoding failed";
- }
+ RleBitPackedEncoder encoder(buffer.data(), buffer_size, bit_width);
+ int32_t encoded_values_size = 0;
+ for (int i = 0; i < data_size; ++i) {
+ // Depending on `spaced` we treat nulls as regular values.
+ if (data.IsValid(i) || !spaced) {
+ bool success = encoder.Put(static_cast<uint64_t>(data_values[i]));
+ ASSERT_TRUE(success) << "Encoding failed in pos " << i;
+ ++encoded_values_size;
}
}
- int encoded_size = encoder.Flush();
+ int encoded_byte_size = encoder.Flush();
+ ASSERT_EQ(encoded_values_size, data_values_count)
+ << "All values input were not encoded successfully by the encoder";
+
+ // Now we verify batch read
+ RleBitPackedDecoder<value_type> decoder(buffer.data(), encoded_byte_size, bit_width);
+ // We will only use one of them depending on whether this is a dictonnary tests
Review Comment:
```suggestion
// We will only use one of them depending on whether this is a dictionary test
```
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -207,12 +209,310 @@ TEST(BitUtil, RoundTripIntValues) {
}
}
+/// A Rle run is a simple class owning some data and a repetition count.
+/// It does not know how to read such data.
+TEST(Rle, RleRun) {
+ const std::array<uint8_t, 4> value = {21, 2, 0, 0};
+
+ RleRun::values_count_type value_count = 12;
+
+ // 12 times the value 21 fitting over 5 bits
+ const auto run_5 = RleRun(value.data(), value_count, /* value_bit_width= */ 5);
+ EXPECT_EQ(run_5.ValuesCount(), value_count);
+ EXPECT_EQ(run_5.ValuesBitWidth(), 5);
+ EXPECT_EQ(run_5.RawDataSize(), 1); // 5 bits fit in one byte
+ EXPECT_EQ(*run_5.RawDataPtr(), 21);
+
+ // 12 times the value 21 fitting over 16 bits
+ const auto run_8 = RleRun(value.data(), value_count, /* value_bit_width= */ 8);
+ EXPECT_EQ(run_8.ValuesCount(), value_count);
+ EXPECT_EQ(run_8.ValuesBitWidth(), 8);
+ EXPECT_EQ(run_8.RawDataSize(), 1); // 8 bits fit in 1 byte
+ EXPECT_EQ(*run_8.RawDataPtr(), 21);
+
+ // 12 times the value {21, 2} fitting over 10 bits
+ const auto run_10 = RleRun(value.data(), value_count, /* value_bit_width= */ 10);
+
+ EXPECT_EQ(run_10.ValuesCount(), value_count);
+ EXPECT_EQ(run_10.ValuesBitWidth(), 10);
+ EXPECT_EQ(run_10.RawDataSize(), 2); // 10 bits fit in 2 bytes
+ EXPECT_EQ(*(run_10.RawDataPtr() + 0), 21);
+ EXPECT_EQ(*(run_10.RawDataPtr() + 1), 2);
+
+ // 12 times the value {21, 2} fitting over 32 bits
+ const auto run_32 = RleRun(value.data(), value_count, /* value_bit_width= */ 32);
+ EXPECT_EQ(run_32.ValuesCount(), value_count);
+ EXPECT_EQ(run_32.ValuesBitWidth(), 32);
+ EXPECT_EQ(run_32.RawDataSize(), 4); // 32 bits fit in 4 bytes
+ EXPECT_EQ(*(run_32.RawDataPtr() + 0), 21);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 1), 2);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 2), 0);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 3), 0);
+}
+
+/// A BitPacked run is a simple class owning some data and its size.
+/// It does not know how to read such data.
+TEST(BitPacked, BitPackedRun) {
+ const std::array<uint8_t, 4> value = {0b10101010, 0, 0, 0b1111111};
+
+ // 16 values of 1 bit for a total of 16 bits
+ BitPackedRun::values_count_type value_count_1 = 16;
+ const auto run_1 = BitPackedRun(value.data(), value_count_1, /* value_bit_width= */ 1);
+ EXPECT_EQ(run_1.ValuesCount(), value_count_1);
+ EXPECT_EQ(run_1.ValuesBitWidth(), 1);
+ EXPECT_EQ(run_1.RawDataSize(), 2); // 16 bits fit in 2 bytes
+ for (BitPackedRun::raw_data_size_type i = 0; i < run_1.RawDataSize(); ++i) {
+ EXPECT_EQ(*(run_1.RawDataPtr() + i), value[i]);
+ }
+
+ // 8 values of 3 bits for a total of 24 bits
+ BitPackedRun::values_count_type value_count_3 = 8;
+ const auto run_3 = BitPackedRun(value.data(), value_count_3, /* value_bit_width= */ 3);
+ EXPECT_EQ(run_3.ValuesCount(), value_count_3);
+ EXPECT_EQ(run_3.ValuesBitWidth(), 3);
+ EXPECT_EQ(run_3.RawDataSize(), 3); // 24 bits fit in 3 bytes
+ for (BitPackedRun::raw_data_size_type i = 0; i < run_3.RawDataSize(); ++i) {
+ EXPECT_EQ(*(run_3.RawDataPtr() + i), value[i]);
+ }
+}
+
+template <typename T>
+void TestRleDecoder(std::vector<uint8_t> bytes, RleRun::values_count_type value_count,
+ RleRun::bit_size_type bit_width, T expected_value) {
+ // Pre-requisite for this test
+ EXPECT_GT(value_count, 6);
+
+ const auto run = RleRun(bytes.data(), value_count, bit_width);
+
+ auto decoder = RleRunDecoder<T>(run);
+ std::vector<T> vals = {0, 0};
+
+ EXPECT_EQ(decoder.Remaining(), value_count);
+
+ typename decltype(decoder)::values_count_type read = 0;
+ EXPECT_EQ(decoder.Get(vals.data()), 1);
+ read += 1;
+ EXPECT_EQ(vals.at(0), expected_value);
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ EXPECT_EQ(decoder.Advance(3), 3);
+ read += 3;
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ vals = {0, 0};
+ EXPECT_EQ(decoder.GetBatch(vals.data(), 2), vals.size());
+ EXPECT_EQ(vals.at(0), expected_value);
+ EXPECT_EQ(vals.at(1), expected_value);
+ read += static_cast<decltype(read)>(vals.size());
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ // Exhaust iteration
+ EXPECT_EQ(decoder.Advance(value_count - read), value_count - read);
+ EXPECT_EQ(decoder.Remaining(), 0);
+ EXPECT_EQ(decoder.Advance(1), 0);
+ vals = {0, 0};
+ EXPECT_EQ(decoder.Get(vals.data()), 0);
+ EXPECT_EQ(vals.at(0), 0);
+
+ // Reset the decoder
+ decoder.Reset(run);
+ EXPECT_EQ(decoder.Remaining(), value_count);
+ vals = {0, 0};
+ EXPECT_EQ(decoder.GetBatch(vals.data(), 2), vals.size());
+ EXPECT_EQ(vals.at(0), expected_value);
+ EXPECT_EQ(vals.at(1), expected_value);
+}
+
+TEST(Rle, RleDecoder) {
+ TestRleDecoder<uint8_t>({21, 0, 0}, /* value_count= */ 21, /* bit_width= */ 5,
+ /* expected_value= */ 21);
+ TestRleDecoder<uint16_t>({1, 0}, /* value_count= */ 13, /* bit_width= */ 1,
+ /* expected_value= */ 1);
+ TestRleDecoder<uint32_t>({21, 0, 0}, /* value_count= */ 21, /* bit_width= */ 5,
+ /* expected_value= */ 21);
Review Comment:
It's a bit weird to test with `value_count == expected_value`.
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -602,20 +1038,126 @@ struct GetBatchSpacedTestCase {
int bit_width;
};
-TEST(RleDecoder, GetBatchSpaced) {
- uint32_t kSeed = 1337;
- ::arrow::random::RandomArrayGenerator rand(kSeed);
-
- std::vector<GetBatchSpacedTestCase<int32_t>> int32_cases{
- {1, 100000, 0.01, 1}, {1, 100000, 0.1, 1}, {1, 100000, 0.5, 1},
- {4, 100000, 0.05, 3}, {100, 100000, 0.05, 7},
+template <typename T>
+void DoTestGetBatchSpacedRoundtrip() {
+ using Data = DataTestRleBitPacked<T>;
+ using ArrowType = typename Data::ArrowType;
+ using RandomPart = typename Data::RandomPart;
+ using NullPart = typename Data::NullPart;
+ using RepeatPart = typename Data::RepeatPart;
+
+ std::vector<Data> test_cases = {
+ {
+ {RandomPart{/* max=*/1, /* size=*/400, /* null_proba= */ 0.1}},
+ /* bit_width= */ 1,
+ },
+ {
+ {
+ RandomPart{/* max=*/7, /* size=*/10037, /* null_proba= */ 0.0},
+ NullPart{/* size= */ 1153},
+ RandomPart{/* max=*/7, /* size=*/800, /* null_proba= */ 0.5},
+ },
+ /* bit_width= */ 3,
+ },
+ {
+ {
+ NullPart{/* size= */ 80},
+ RandomPart{/* max=*/static_cast<T>(1023), /* size=*/800,
+ /* null_proba= */ 0.01},
+ NullPart{/* size= */ 1023},
+ },
+ /* bit_width= */ 11,
+ },
+ {
+ {RepeatPart{/* value=*/13, /* size=*/100000, /* null_proba= */ 0.01}},
Review Comment:
We probably want to reduce the sizes a bit to keep the test durations in check (especially in ASAN and Valgrind builds). I don't think it's useful to test more than 1k repeated values, for example.
##########
cpp/src/arrow/util/bit_stream_utils_internal.h:
##########
@@ -439,91 +424,84 @@ inline bool BitReader::Advance(int64_t num_bits) {
return true;
}
-inline bool BitWriter::PutVlqInt(uint32_t v) {
- bool result = true;
- while ((v & 0xFFFFFF80UL) != 0UL) {
- result &= PutAligned<uint8_t>(static_cast<uint8_t>((v & 0x7F) | 0x80), 1);
- v >>= 7;
- }
- result &= PutAligned<uint8_t>(static_cast<uint8_t>(v & 0x7F), 1);
- return result;
-}
+template <typename Int>
+inline bool BitWriter::PutVlqInt(Int v) {
+ static_assert(std::is_integral_v<Int>);
-inline bool BitReader::GetVlqInt(uint32_t* v) {
- uint32_t tmp = 0;
+ constexpr auto kBufferSize = kMaxLEB128ByteLenFor<Int>;
- for (int i = 0; i < kMaxVlqByteLength; i++) {
- uint8_t byte = 0;
- if (ARROW_PREDICT_FALSE(!GetAligned<uint8_t>(1, &byte))) {
- return false;
- }
- tmp |= static_cast<uint32_t>(byte & 0x7F) << (7 * i);
+ uint8_t buffer[kBufferSize] = {};
+ const auto bytes_written = WriteLEB128(v, buffer, kBufferSize);
+ ARROW_DCHECK_LE(bytes_written, kBufferSize);
+ ARROW_DCHECK_GT(bytes_written, 0); // Cannot fail since we gave max space
- if ((byte & 0x80) == 0) {
- *v = tmp;
- return true;
+ for (int i = 0; i < bytes_written; ++i) {
+ const bool success = PutAligned(buffer[i], 1);
+ if (ARROW_PREDICT_FALSE(!success)) {
+ return false;
}
}
- return false;
-}
-
-inline bool BitWriter::PutZigZagVlqInt(int32_t v) {
- uint32_t u_v = ::arrow::util::SafeCopy<uint32_t>(v);
- u_v = (u_v << 1) ^ static_cast<uint32_t>(v >> 31);
- return PutVlqInt(u_v);
-}
-
-inline bool BitReader::GetZigZagVlqInt(int32_t* v) {
- uint32_t u;
- if (!GetVlqInt(&u)) return false;
- u = (u >> 1) ^ (~(u & 1) + 1);
- *v = ::arrow::util::SafeCopy<int32_t>(u);
return true;
}
-inline bool BitWriter::PutVlqInt(uint64_t v) {
- bool result = true;
- while ((v & 0xFFFFFFFFFFFFFF80ULL) != 0ULL) {
- result &= PutAligned<uint8_t>(static_cast<uint8_t>((v & 0x7F) | 0x80), 1);
- v >>= 7;
+template <typename Int>
+inline bool BitReader::GetVlqInt(Int* v) {
+ static_assert(std::is_integral_v<Int>);
+
+ // The data that we will pass to the LEB128 parser
+ // In all case, we read an byte-aligned value, skipping remaining bits
Review Comment:
```suggestion
// In all case, we read a byte-aligned value, skipping remaining bits
```
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -553,47 +853,183 @@ TEST(BitRle, Overflow) {
}
}
+/// Check RleBitPacked encoding/decoding round trip.
+///
+/// \param spaced If set to false, treat Nulls in the input array as regular data.
+/// \param parts The number of parts in which the data will be decoded.
+/// For number greater than one, this ensure that the decoder intermediary state
+/// is valid.
template <typename Type>
-void CheckRoundTripSpaced(const Array& data, int bit_width) {
+void CheckRoundTrip(const Array& data, int bit_width, bool spaced, int32_t parts,
+ std::shared_ptr<FloatArray> dict = {}) {
using ArrayType = typename TypeTraits<Type>::ArrayType;
- using T = typename Type::c_type;
+ using value_type = typename Type::c_type;
- int num_values = static_cast<int>(data.length());
- int buffer_size = RleEncoder::MaxBufferSize(bit_width, num_values);
+ const int data_size = static_cast<int>(data.length());
+ const int data_values_count =
+ static_cast<int>(data.length() - spaced * data.null_count());
+ const int buffer_size = RleBitPackedEncoder::MaxBufferSize(bit_width, data_size);
+ ASSERT_GE(parts, 1);
+ ASSERT_LE(parts, data_size);
- const T* values = static_cast<const ArrayType&>(data).raw_values();
+ const value_type* data_values = static_cast<const ArrayType&>(data).raw_values();
+ // Encode the data into `buffer` using the encoder.
std::vector<uint8_t> buffer(buffer_size);
- RleEncoder encoder(buffer.data(), buffer_size, bit_width);
- for (int i = 0; i < num_values; ++i) {
- if (data.IsValid(i)) {
- if (!encoder.Put(static_cast<uint64_t>(values[i]))) {
- FAIL() << "Encoding failed";
- }
+ RleBitPackedEncoder encoder(buffer.data(), buffer_size, bit_width);
+ int32_t encoded_values_size = 0;
+ for (int i = 0; i < data_size; ++i) {
+ // Depending on `spaced` we treat nulls as regular values.
+ if (data.IsValid(i) || !spaced) {
+ bool success = encoder.Put(static_cast<uint64_t>(data_values[i]));
+ ASSERT_TRUE(success) << "Encoding failed in pos " << i;
+ ++encoded_values_size;
}
}
- int encoded_size = encoder.Flush();
+ int encoded_byte_size = encoder.Flush();
+ ASSERT_EQ(encoded_values_size, data_values_count)
+ << "All values input were not encoded successfully by the encoder";
+
+ // Now we verify batch read
+ RleBitPackedDecoder<value_type> decoder(buffer.data(), encoded_byte_size, bit_width);
+ // We will only use one of them depending on whether this is a dictonnary tests
+ std::vector<float> dict_read;
+ std::vector<value_type> values_read;
+ if (dict) {
+ dict_read.resize(data_size);
+ } else {
+ values_read.resize(data_size);
+ }
- // Verify batch read
- RleDecoder decoder(buffer.data(), encoded_size, bit_width);
- std::vector<T> values_read(num_values);
+ // We will read the data in `parts` calls to make sure intermediate states are valid
+ int32_t actual_read_count = 0;
+ int32_t requested_read_count = 0;
+ while (requested_read_count < data_size) {
+ const auto remaining = data_size - requested_read_count;
+ auto to_read = data_size / parts;
+ if (remaining / to_read == 1) {
+ to_read = remaining;
+ }
- if (num_values != decoder.GetBatchSpaced(
- num_values, static_cast<int>(data.null_count()),
- data.null_bitmap_data(), data.offset(), values_read.data())) {
- FAIL();
- }
+ auto read = 0;
+ if (spaced) {
+ // We need to slice the input array get the proper null count and bitmap
+ auto data_remaining = data.Slice(requested_read_count, to_read);
+
+ if (dict) {
+ auto* out = dict_read.data() + requested_read_count;
+ read = decoder.GetBatchWithDictSpaced(
+ dict->raw_values(), static_cast<int32_t>(dict->length()), out, to_read,
+ static_cast<int32_t>(data_remaining->null_count()),
+ data_remaining->null_bitmap_data(), data_remaining->offset());
+ } else {
+ auto* out = values_read.data() + requested_read_count;
+ read = decoder.GetBatchSpaced(
+ to_read, static_cast<int32_t>(data_remaining->null_count()),
+ data_remaining->null_bitmap_data(), data_remaining->offset(), out);
+ }
+ } else {
+ if (dict) {
+ auto* out = dict_read.data() + requested_read_count;
+ read = decoder.GetBatchWithDict(
+ dict->raw_values(), static_cast<int32_t>(dict->length()), out, to_read);
+ } else {
+ auto* out = values_read.data() + requested_read_count;
+ read = decoder.GetBatch(out, to_read);
+ }
+ }
+ ASSERT_EQ(read, to_read) << "Decoder did not read as many values as requested";
- for (int64_t i = 0; i < num_values; ++i) {
- if (data.IsValid(i)) {
- if (values_read[i] != values[i]) {
- FAIL() << "Index " << i << " read " << values_read[i] << " but should be "
- << values[i];
+ actual_read_count += read;
+ requested_read_count += to_read;
Review Comment:
Since we asserted just above that `read` and `to_read` were equal, is there a reason to keep track of both `actual_read_count` and `requested_read_count`? It's impossible for them to get out of sync.
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -84,32 +85,278 @@ namespace util {
/// (total 26 bytes, 1 byte overhead)
//
+class RleRun {
+ public:
+ using byte = uint8_t;
+ /// Enough space to store a 64bit value
+ using raw_data_storage = std::array<byte, 8>;
+ using raw_data_const_pointer = const byte*;
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr RleRun() noexcept = default;
Review Comment:
Well, you can have a simple `struct` with attributes *plus* some helper methods.
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -84,32 +85,278 @@ namespace util {
/// (total 26 bytes, 1 byte overhead)
//
+class RleRun {
+ public:
+ using byte = uint8_t;
+ /// Enough space to store a 64bit value
+ using raw_data_storage = std::array<byte, 8>;
+ using raw_data_const_pointer = const byte*;
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr RleRun() noexcept = default;
+ constexpr RleRun(RleRun const&) noexcept = default;
+ constexpr RleRun(RleRun&&) noexcept = default;
+
+ explicit RleRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr RleRun& operator=(RleRun const&) noexcept = default;
+ constexpr RleRun& operator=(RleRun&&) noexcept = default;
+
+ /// The number of repeated values in this run.
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ /// A pointer to the repeated value raw bytes.
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ /// The number of bytes used for the raw repeated value.
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The repeated value raw bytes stored inside the class
+ raw_data_storage data_ = {};
+ /// The number of time the value is repeated
+ values_count_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+class BitPackedRun {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// According to the Parquet thrift definition the page size can be written into an
+ /// int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr BitPackedRun() noexcept = default;
+ constexpr BitPackedRun(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun(BitPackedRun&&) noexcept = default;
+
+ constexpr BitPackedRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr BitPackedRun& operator=(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun& operator=(BitPackedRun&&) noexcept = default;
+
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Number of values in this run.
+ raw_data_size_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+/// A parser that emits either a ``BitPackedRun`` or a ``RleRun``.
+class RleBitPackedParser {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// By Parquet thrift definition the page size can be written into an int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+ /// The different types of runs emitted by the parser
+ using dynamic_run_type = std::variant<RleRun, BitPackedRun>;
+
+ constexpr RleBitPackedParser() noexcept = default;
+
+ constexpr RleBitPackedParser(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr void Reset(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width_) noexcept;
+
+ /// Get the current run with a small parsing cost without advancing the iteration.
+ [[nodiscard]] std::optional<dynamic_run_type> Peek() const;
+
+ /// Move to the next run.
+ [[nodiscard]] bool Advance();
+
+ /// Advance and return the current run.
+ [[nodiscard]] std::optional<dynamic_run_type> Next();
+
+ /// Whether there is still runs to iterate over.
+ ///
+ /// WARN: Due to lack of proper error handling, iteration with Next and Peek could
+ /// return not data while the parser is not exhausted.
+ /// This is how one can check for errors.
+ [[nodiscard]] bool Exhausted() const;
+
+ /// Enum to return from an ``Parse`` handler.
+ ///
+ /// Since a callback has no way to know when to stop, the handler must return
+ /// a value indicating to the ``Parse`` function whether to stop or continue.
+ enum class ControlFlow {
+ Continue,
+ Break,
+ };
+
+ /// A callback approach to parsing.
+ ///
+ /// This approach is used to reduce the number of dynamic lookups involved with using a
+ /// variant.
+ ///
+ /// The handler must be of the form
+ /// ```cpp`
+ /// struct Handler {
+ /// ControlFlow OnBitPackedRun(BitPackedRun run);
+ ///
+ /// ControlFlow OnRleRun(RleRun run);
+ /// };
+ /// ```
+ template <typename Handler>
+ void Parse(Handler&& handler);
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Size in bytes of the run.
+ raw_data_size_type data_size_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+
+ /// Run the handler on the run read and return the number of values read.
+ /// Does not advance the parser.
+ template <typename Handler>
+ std::pair<raw_data_size_type, ControlFlow> PeekImpl(Handler&&) const;
+};
+
/// Decoder class for RLE encoded data.
+template <typename T>
class RleDecoder {
public:
- /// Create a decoder object. buffer/buffer_len is the decoded data.
- /// bit_width is the width of each value (before encoding).
- RleDecoder(const uint8_t* buffer, int buffer_len, int bit_width)
- : bit_reader_(buffer, buffer_len),
- bit_width_(bit_width),
- current_value_(0),
- repeat_count_(0),
- literal_count_(0) {
- ARROW_DCHECK_GE(bit_width_, 0);
- ARROW_DCHECK_LE(bit_width_, 64);
- }
+ /// The type in which the data should be decoded.
+ using value_type = T;
+ /// The type of run that can be decoded.
+ using run_type = RleRun;
+ using values_count_type = run_type::values_count_type;
Review Comment:
> The thing I think that remain very useful in reading in the future is answering questions like "_why does this only has 32 bits?_", "_Why is it signed?_"...
Agreed. Adding explanatory comments can be part of the answer here.
> We could keep a few namespace level alias like `using RleBitPackedSize = int32_t;` along with their documentation. What do you think?
Why not.
##########
cpp/src/arrow/util/bit_stream_utils_internal.h:
##########
@@ -439,91 +424,84 @@ inline bool BitReader::Advance(int64_t num_bits) {
return true;
}
-inline bool BitWriter::PutVlqInt(uint32_t v) {
- bool result = true;
- while ((v & 0xFFFFFF80UL) != 0UL) {
- result &= PutAligned<uint8_t>(static_cast<uint8_t>((v & 0x7F) | 0x80), 1);
- v >>= 7;
- }
- result &= PutAligned<uint8_t>(static_cast<uint8_t>(v & 0x7F), 1);
- return result;
-}
+template <typename Int>
+inline bool BitWriter::PutVlqInt(Int v) {
+ static_assert(std::is_integral_v<Int>);
-inline bool BitReader::GetVlqInt(uint32_t* v) {
- uint32_t tmp = 0;
+ constexpr auto kBufferSize = kMaxLEB128ByteLenFor<Int>;
- for (int i = 0; i < kMaxVlqByteLength; i++) {
- uint8_t byte = 0;
- if (ARROW_PREDICT_FALSE(!GetAligned<uint8_t>(1, &byte))) {
- return false;
- }
- tmp |= static_cast<uint32_t>(byte & 0x7F) << (7 * i);
+ uint8_t buffer[kBufferSize] = {};
+ const auto bytes_written = WriteLEB128(v, buffer, kBufferSize);
+ ARROW_DCHECK_LE(bytes_written, kBufferSize);
+ ARROW_DCHECK_GT(bytes_written, 0); // Cannot fail since we gave max space
- if ((byte & 0x80) == 0) {
- *v = tmp;
- return true;
+ for (int i = 0; i < bytes_written; ++i) {
+ const bool success = PutAligned(buffer[i], 1);
+ if (ARROW_PREDICT_FALSE(!success)) {
+ return false;
}
}
- return false;
-}
-
-inline bool BitWriter::PutZigZagVlqInt(int32_t v) {
- uint32_t u_v = ::arrow::util::SafeCopy<uint32_t>(v);
- u_v = (u_v << 1) ^ static_cast<uint32_t>(v >> 31);
- return PutVlqInt(u_v);
-}
-
-inline bool BitReader::GetZigZagVlqInt(int32_t* v) {
- uint32_t u;
- if (!GetVlqInt(&u)) return false;
- u = (u >> 1) ^ (~(u & 1) + 1);
- *v = ::arrow::util::SafeCopy<int32_t>(u);
return true;
}
-inline bool BitWriter::PutVlqInt(uint64_t v) {
- bool result = true;
- while ((v & 0xFFFFFFFFFFFFFF80ULL) != 0ULL) {
- result &= PutAligned<uint8_t>(static_cast<uint8_t>((v & 0x7F) | 0x80), 1);
- v >>= 7;
+template <typename Int>
+inline bool BitReader::GetVlqInt(Int* v) {
+ static_assert(std::is_integral_v<Int>);
Review Comment:
Can we also assert that it's unsigned?
##########
cpp/src/arrow/util/bit_util_test.cc:
##########
@@ -1997,11 +1997,191 @@ TEST(BitUtil, RoundUpToPowerOf2) {
#undef U64
#undef S64
+/// Test the maximum number of bytes needed to write a LEB128 of a give size.
+TEST(LEB128, MaxLEB128ByteLenFor) {
+ EXPECT_EQ(bit_util::kMaxLEB128ByteLenFor<int16_t>, 3);
+ EXPECT_EQ(bit_util::kMaxLEB128ByteLenFor<int32_t>, 5);
+ EXPECT_EQ(bit_util::kMaxLEB128ByteLenFor<int64_t>, 10);
+}
+
+/// Utility function to test LEB128 encoding with known input value and expected byte
+/// array
+template <typename Int>
+void TestLEB128Encode(Int input_value, const std::vector<uint8_t>& expected_data,
+ std::size_t buffer_size) {
+ std::vector<uint8_t> buffer(buffer_size);
+ auto bytes_written = bit_util::WriteLEB128(input_value, buffer.data(),
+ static_cast<int32_t>(buffer.size()));
+
+ EXPECT_EQ(bytes_written, expected_data.size());
+ // Encoded data
+ for (std::size_t i = 0; i < expected_data.size(); ++i) {
+ EXPECT_EQ(buffer.at(i), expected_data.at(i));
+ }
+
+ // When the value is successfully encoded, the remaining of the buffer is untouched
+ if (bytes_written > 0) {
+ for (std::size_t i = bytes_written; i < buffer.size(); ++i) {
+ EXPECT_EQ(buffer.at(i), 0);
+ }
+ }
+}
+
+/// Test encoding to known LEB128 byte sequences with edge cases parameters.
+/// \see LEB128.KnownSuccessfulValues for other known values tested.
+TEST(LEB128, WriteEdgeCases) {
+ // Single byte value 0
+ TestLEB128Encode(0U, {0x00}, 1);
+ // Single byte value 127
+ TestLEB128Encode(127U, {0x7F}, 1);
+ // Three byte value 16384, encoded in larger buffer
+ TestLEB128Encode(16384U, {0x80, 0x80, 0x01}, 10);
+ // Two byte boundary values
+ TestLEB128Encode(129U, {0x81, 0x01}, 2);
+ TestLEB128Encode(16383U, {0xFF, 0x7F}, 2);
+ // Error case: Buffer too small for value 128 (needs 2 bytes but only 1 provided)
+ TestLEB128Encode(128U, {}, 1);
+ // Error case: Buffer too small for uint32_t max (needs 5 bytes but only 4 provided)
+ TestLEB128Encode(4294967295U, {}, 4);
Review Comment:
Can we test a couple large 64-bit ints?
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -84,32 +85,278 @@ namespace util {
/// (total 26 bytes, 1 byte overhead)
//
+class RleRun {
+ public:
+ using byte = uint8_t;
+ /// Enough space to store a 64bit value
+ using raw_data_storage = std::array<byte, 8>;
+ using raw_data_const_pointer = const byte*;
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr RleRun() noexcept = default;
+ constexpr RleRun(RleRun const&) noexcept = default;
+ constexpr RleRun(RleRun&&) noexcept = default;
+
+ explicit RleRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr RleRun& operator=(RleRun const&) noexcept = default;
+ constexpr RleRun& operator=(RleRun&&) noexcept = default;
+
+ /// The number of repeated values in this run.
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ /// A pointer to the repeated value raw bytes.
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ /// The number of bytes used for the raw repeated value.
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The repeated value raw bytes stored inside the class
+ raw_data_storage data_ = {};
+ /// The number of time the value is repeated
+ values_count_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+class BitPackedRun {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// According to the Parquet thrift definition the page size can be written into an
+ /// int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr BitPackedRun() noexcept = default;
+ constexpr BitPackedRun(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun(BitPackedRun&&) noexcept = default;
+
+ constexpr BitPackedRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr BitPackedRun& operator=(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun& operator=(BitPackedRun&&) noexcept = default;
+
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Number of values in this run.
+ raw_data_size_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+/// A parser that emits either a ``BitPackedRun`` or a ``RleRun``.
+class RleBitPackedParser {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// By Parquet thrift definition the page size can be written into an int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+ /// The different types of runs emitted by the parser
+ using dynamic_run_type = std::variant<RleRun, BitPackedRun>;
+
+ constexpr RleBitPackedParser() noexcept = default;
+
+ constexpr RleBitPackedParser(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr void Reset(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width_) noexcept;
+
+ /// Get the current run with a small parsing cost without advancing the iteration.
+ [[nodiscard]] std::optional<dynamic_run_type> Peek() const;
+
+ /// Move to the next run.
+ [[nodiscard]] bool Advance();
+
+ /// Advance and return the current run.
+ [[nodiscard]] std::optional<dynamic_run_type> Next();
+
+ /// Whether there is still runs to iterate over.
+ ///
+ /// WARN: Due to lack of proper error handling, iteration with Next and Peek could
+ /// return not data while the parser is not exhausted.
+ /// This is how one can check for errors.
+ [[nodiscard]] bool Exhausted() const;
+
+ /// Enum to return from an ``Parse`` handler.
+ ///
+ /// Since a callback has no way to know when to stop, the handler must return
+ /// a value indicating to the ``Parse`` function whether to stop or continue.
+ enum class ControlFlow {
+ Continue,
+ Break,
+ };
+
+ /// A callback approach to parsing.
+ ///
+ /// This approach is used to reduce the number of dynamic lookups involved with using a
+ /// variant.
+ ///
+ /// The handler must be of the form
+ /// ```cpp`
+ /// struct Handler {
+ /// ControlFlow OnBitPackedRun(BitPackedRun run);
+ ///
+ /// ControlFlow OnRleRun(RleRun run);
+ /// };
+ /// ```
+ template <typename Handler>
+ void Parse(Handler&& handler);
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Size in bytes of the run.
+ raw_data_size_type data_size_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+
+ /// Run the handler on the run read and return the number of values read.
+ /// Does not advance the parser.
+ template <typename Handler>
+ std::pair<raw_data_size_type, ControlFlow> PeekImpl(Handler&&) const;
+};
+
/// Decoder class for RLE encoded data.
+template <typename T>
class RleDecoder {
public:
- /// Create a decoder object. buffer/buffer_len is the decoded data.
- /// bit_width is the width of each value (before encoding).
- RleDecoder(const uint8_t* buffer, int buffer_len, int bit_width)
- : bit_reader_(buffer, buffer_len),
- bit_width_(bit_width),
- current_value_(0),
- repeat_count_(0),
- literal_count_(0) {
- ARROW_DCHECK_GE(bit_width_, 0);
- ARROW_DCHECK_LE(bit_width_, 64);
- }
+ /// The type in which the data should be decoded.
+ using value_type = T;
+ /// The type of run that can be decoded.
+ using run_type = RleRun;
+ using values_count_type = run_type::values_count_type;
- RleDecoder() : bit_width_(-1) {}
+ constexpr RleDecoder() noexcept = default;
- void Reset(const uint8_t* buffer, int buffer_len, int bit_width) {
- ARROW_DCHECK_GE(bit_width, 0);
- ARROW_DCHECK_LE(bit_width, 64);
- bit_reader_.Reset(buffer, buffer_len);
- bit_width_ = bit_width;
- current_value_ = 0;
- repeat_count_ = 0;
- literal_count_ = 0;
- }
+ explicit RleDecoder(run_type const& run) noexcept;
+
+ void Reset(run_type const& run) noexcept;
+
+ /// Return the number of values that can be advanced.
+ [[nodiscard]] values_count_type Remaining() const;
+
+ /// Return the repeated value of this decoder.
+ [[nodiscard]] constexpr value_type Value() const;
+
+ /// Try to advance by as many values as provided.
+ /// Return the number of values skipped.
+ [[nodiscard]] values_count_type Advance(values_count_type batch_size);
+
+ /// Get the next value and return false if there are no more.
+ [[nodiscard]] constexpr bool Get(value_type* out_value);
+
+ /// Get a batch of values return the number of decoded elements.
+ [[nodiscard]] values_count_type GetBatch(value_type* out, values_count_type batch_size);
+
+ private:
+ value_type value_ = {};
+ values_count_type remaining_count_ = 0;
+
+ static_assert(std::is_integral_v<value_type>,
+ "This class makes assumptions about integer endianness and padding");
+};
+
+/// Decoder class for Bit packing encoded data.
+template <typename T>
+class BitPackedDecoder {
+ public:
+ /// The type in which the data should be decoded.
+ using value_type = T;
+ /// The type of run that can be decoded.
+ using run_type = BitPackedRun;
+ using values_count_type = run_type::values_count_type;
+ using bit_size_type = run_type::bit_size_type;
+
+ BitPackedDecoder() noexcept = default;
+
+ explicit BitPackedDecoder(run_type const& run) noexcept;
+
+ void Reset(run_type const& run) noexcept;
+
+ /// Return the number of values that can be advanced.
+ [[nodiscard]] constexpr values_count_type Remaining() const;
+
+ /// Return the size in bit in which each encoded value is written.
+ [[nodiscard]] constexpr bit_size_type ValueBitWidth() const;
+
+ /// Try to advance by as many values as provided.
+ /// Return the number of values skipped.
+ [[nodiscard]] values_count_type Advance(values_count_type batch_size);
+
+ /// Get the next value and return false if there are no more.
+ [[nodiscard]] bool Get(value_type* out_value);
+
+ /// Get a batch of values return the number of decoded elements.
+ [[nodiscard]] values_count_type GetBatch(value_type* out, values_count_type batch_size);
+
+ private:
+ ::arrow::bit_util::BitReader bit_reader_ = {};
+ bit_size_type value_bit_width_ = 0;
+ values_count_type remaining_count_ = 0;
+
+ static_assert(std::is_integral_v<value_type>,
+ "This class makes assumptions about integer endianness and padding");
+};
+
+/// Decoder class for RLE encoded data.
+template <typename T>
+class RleBitPackedDecoder {
+ public:
+ /// The type in which the data should be decoded.
+ using value_type = T;
+ using byte = RleBitPackedParser::byte;
+ using raw_data_const_pointer = RleBitPackedParser::raw_data_const_pointer;
+ using raw_data_size_type = RleBitPackedParser::raw_data_size_type;
+ using bit_size_type = RleBitPackedParser::bit_size_type;
+ using dynamic_run_type = RleBitPackedParser::dynamic_run_type;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+
+ RleBitPackedDecoder() noexcept = default;
+
+ /// Create a decoder object.
+ ///
+ /// data and data_size are the raw bytes to decode.
+ /// value_bit_width is the size in bits of each encoded value.
+ RleBitPackedDecoder(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept;
+
+ void Reset(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width_) noexcept;
+
+ /// Whether there is still runs to iterate over.
+ ///
+ /// WARN: Due to lack of proper error handling, iteration with Get methods could return
+ /// no data while the parser is not exhausted.
+ /// This is how one can check for errors.
+ [[nodiscard]] bool Exhausted() const;
Review Comment:
Just a ping on this :)
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -84,32 +85,278 @@ namespace util {
/// (total 26 bytes, 1 byte overhead)
//
+class RleRun {
+ public:
+ using byte = uint8_t;
+ /// Enough space to store a 64bit value
+ using raw_data_storage = std::array<byte, 8>;
+ using raw_data_const_pointer = const byte*;
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr RleRun() noexcept = default;
+ constexpr RleRun(RleRun const&) noexcept = default;
+ constexpr RleRun(RleRun&&) noexcept = default;
+
+ explicit RleRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr RleRun& operator=(RleRun const&) noexcept = default;
+ constexpr RleRun& operator=(RleRun&&) noexcept = default;
+
+ /// The number of repeated values in this run.
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ /// A pointer to the repeated value raw bytes.
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ /// The number of bytes used for the raw repeated value.
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The repeated value raw bytes stored inside the class
+ raw_data_storage data_ = {};
+ /// The number of time the value is repeated
+ values_count_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+class BitPackedRun {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// According to the Parquet thrift definition the page size can be written into an
+ /// int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr BitPackedRun() noexcept = default;
+ constexpr BitPackedRun(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun(BitPackedRun&&) noexcept = default;
+
+ constexpr BitPackedRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr BitPackedRun& operator=(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun& operator=(BitPackedRun&&) noexcept = default;
+
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Number of values in this run.
+ raw_data_size_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+/// A parser that emits either a ``BitPackedRun`` or a ``RleRun``.
+class RleBitPackedParser {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// By Parquet thrift definition the page size can be written into an int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+ /// The different types of runs emitted by the parser
+ using dynamic_run_type = std::variant<RleRun, BitPackedRun>;
+
+ constexpr RleBitPackedParser() noexcept = default;
+
+ constexpr RleBitPackedParser(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr void Reset(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width_) noexcept;
+
+ /// Get the current run with a small parsing cost without advancing the iteration.
+ [[nodiscard]] std::optional<dynamic_run_type> Peek() const;
+
+ /// Move to the next run.
+ [[nodiscard]] bool Advance();
+
+ /// Advance and return the current run.
+ [[nodiscard]] std::optional<dynamic_run_type> Next();
+
+ /// Whether there is still runs to iterate over.
+ ///
+ /// WARN: Due to lack of proper error handling, iteration with Next and Peek could
+ /// return not data while the parser is not exhausted.
+ /// This is how one can check for errors.
+ [[nodiscard]] bool Exhausted() const;
+
+ /// Enum to return from an ``Parse`` handler.
+ ///
+ /// Since a callback has no way to know when to stop, the handler must return
+ /// a value indicating to the ``Parse`` function whether to stop or continue.
+ enum class ControlFlow {
+ Continue,
+ Break,
+ };
+
+ /// A callback approach to parsing.
+ ///
+ /// This approach is used to reduce the number of dynamic lookups involved with using a
+ /// variant.
+ ///
+ /// The handler must be of the form
+ /// ```cpp`
+ /// struct Handler {
+ /// ControlFlow OnBitPackedRun(BitPackedRun run);
+ ///
+ /// ControlFlow OnRleRun(RleRun run);
+ /// };
+ /// ```
+ template <typename Handler>
+ void Parse(Handler&& handler);
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Size in bytes of the run.
+ raw_data_size_type data_size_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+
+ /// Run the handler on the run read and return the number of values read.
+ /// Does not advance the parser.
+ template <typename Handler>
+ std::pair<raw_data_size_type, ControlFlow> PeekImpl(Handler&&) const;
+};
+
/// Decoder class for RLE encoded data.
+template <typename T>
class RleDecoder {
public:
- /// Create a decoder object. buffer/buffer_len is the decoded data.
- /// bit_width is the width of each value (before encoding).
- RleDecoder(const uint8_t* buffer, int buffer_len, int bit_width)
- : bit_reader_(buffer, buffer_len),
- bit_width_(bit_width),
- current_value_(0),
- repeat_count_(0),
- literal_count_(0) {
- ARROW_DCHECK_GE(bit_width_, 0);
- ARROW_DCHECK_LE(bit_width_, 64);
- }
+ /// The type in which the data should be decoded.
+ using value_type = T;
+ /// The type of run that can be decoded.
+ using run_type = RleRun;
+ using values_count_type = run_type::values_count_type;
- RleDecoder() : bit_width_(-1) {}
+ constexpr RleDecoder() noexcept = default;
- void Reset(const uint8_t* buffer, int buffer_len, int bit_width) {
- ARROW_DCHECK_GE(bit_width, 0);
- ARROW_DCHECK_LE(bit_width, 64);
- bit_reader_.Reset(buffer, buffer_len);
- bit_width_ = bit_width;
- current_value_ = 0;
- repeat_count_ = 0;
- literal_count_ = 0;
- }
+ explicit RleDecoder(run_type const& run) noexcept;
+
+ void Reset(run_type const& run) noexcept;
+
+ /// Return the number of values that can be advanced.
+ [[nodiscard]] values_count_type Remaining() const;
Review Comment:
Note that if errors are signaled by a special value (e.g. zero), the docstring should make it clear.
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -84,32 +85,278 @@ namespace util {
/// (total 26 bytes, 1 byte overhead)
//
+class RleRun {
+ public:
+ using byte = uint8_t;
+ /// Enough space to store a 64bit value
+ using raw_data_storage = std::array<byte, 8>;
+ using raw_data_const_pointer = const byte*;
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr RleRun() noexcept = default;
+ constexpr RleRun(RleRun const&) noexcept = default;
+ constexpr RleRun(RleRun&&) noexcept = default;
+
+ explicit RleRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr RleRun& operator=(RleRun const&) noexcept = default;
+ constexpr RleRun& operator=(RleRun&&) noexcept = default;
+
+ /// The number of repeated values in this run.
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ /// A pointer to the repeated value raw bytes.
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ /// The number of bytes used for the raw repeated value.
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The repeated value raw bytes stored inside the class
+ raw_data_storage data_ = {};
+ /// The number of time the value is repeated
+ values_count_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+class BitPackedRun {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// According to the Parquet thrift definition the page size can be written into an
+ /// int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type of the size of either run, between 1 and 2^31-1 as per Parquet spec
+ using values_count_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+
+ constexpr BitPackedRun() noexcept = default;
+ constexpr BitPackedRun(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun(BitPackedRun&&) noexcept = default;
+
+ constexpr BitPackedRun(raw_data_const_pointer data, values_count_type values_count,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr BitPackedRun& operator=(BitPackedRun const&) noexcept = default;
+ constexpr BitPackedRun& operator=(BitPackedRun&&) noexcept = default;
+
+ [[nodiscard]] constexpr values_count_type ValuesCount() const noexcept;
+
+ /// The size in bits of each encoded value.
+ [[nodiscard]] constexpr bit_size_type ValuesBitWidth() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_const_pointer RawDataPtr() const noexcept;
+
+ [[nodiscard]] constexpr raw_data_size_type RawDataSize() const noexcept;
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Number of values in this run.
+ raw_data_size_type values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+};
+
+/// A parser that emits either a ``BitPackedRun`` or a ``RleRun``.
+class RleBitPackedParser {
+ public:
+ using byte = uint8_t;
+ using raw_data_const_pointer = const byte*;
+ /// By Parquet thrift definition the page size can be written into an int32_t.
+ using raw_data_size_type = int32_t;
+ /// The type to represent a size in bits
+ using bit_size_type = int32_t;
+ /// The different types of runs emitted by the parser
+ using dynamic_run_type = std::variant<RleRun, BitPackedRun>;
+
+ constexpr RleBitPackedParser() noexcept = default;
+
+ constexpr RleBitPackedParser(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept;
+
+ constexpr void Reset(raw_data_const_pointer data, raw_data_size_type data_size,
+ bit_size_type value_bit_width_) noexcept;
+
+ /// Get the current run with a small parsing cost without advancing the iteration.
+ [[nodiscard]] std::optional<dynamic_run_type> Peek() const;
+
+ /// Move to the next run.
+ [[nodiscard]] bool Advance();
+
+ /// Advance and return the current run.
+ [[nodiscard]] std::optional<dynamic_run_type> Next();
+
+ /// Whether there is still runs to iterate over.
+ ///
+ /// WARN: Due to lack of proper error handling, iteration with Next and Peek could
+ /// return not data while the parser is not exhausted.
+ /// This is how one can check for errors.
+ [[nodiscard]] bool Exhausted() const;
+
+ /// Enum to return from an ``Parse`` handler.
+ ///
+ /// Since a callback has no way to know when to stop, the handler must return
+ /// a value indicating to the ``Parse`` function whether to stop or continue.
+ enum class ControlFlow {
+ Continue,
+ Break,
+ };
+
+ /// A callback approach to parsing.
+ ///
+ /// This approach is used to reduce the number of dynamic lookups involved with using a
+ /// variant.
+ ///
+ /// The handler must be of the form
+ /// ```cpp`
+ /// struct Handler {
+ /// ControlFlow OnBitPackedRun(BitPackedRun run);
+ ///
+ /// ControlFlow OnRleRun(RleRun run);
+ /// };
+ /// ```
+ template <typename Handler>
+ void Parse(Handler&& handler);
+
+ private:
+ /// The pointer to the beginning of the run
+ raw_data_const_pointer data_ = nullptr;
+ /// Size in bytes of the run.
+ raw_data_size_type data_size_ = 0;
+ /// The size in bit of a packed value in the run
+ bit_size_type value_bit_width_ = 0;
+
+ /// Run the handler on the run read and return the number of values read.
+ /// Does not advance the parser.
+ template <typename Handler>
+ std::pair<raw_data_size_type, ControlFlow> PeekImpl(Handler&&) const;
+};
+
/// Decoder class for RLE encoded data.
+template <typename T>
class RleDecoder {
public:
- /// Create a decoder object. buffer/buffer_len is the decoded data.
- /// bit_width is the width of each value (before encoding).
- RleDecoder(const uint8_t* buffer, int buffer_len, int bit_width)
- : bit_reader_(buffer, buffer_len),
- bit_width_(bit_width),
- current_value_(0),
- repeat_count_(0),
- literal_count_(0) {
- ARROW_DCHECK_GE(bit_width_, 0);
- ARROW_DCHECK_LE(bit_width_, 64);
- }
+ /// The type in which the data should be decoded.
+ using value_type = T;
+ /// The type of run that can be decoded.
+ using run_type = RleRun;
+ using values_count_type = run_type::values_count_type;
- RleDecoder() : bit_width_(-1) {}
+ constexpr RleDecoder() noexcept = default;
- void Reset(const uint8_t* buffer, int buffer_len, int bit_width) {
- ARROW_DCHECK_GE(bit_width, 0);
- ARROW_DCHECK_LE(bit_width, 64);
- bit_reader_.Reset(buffer, buffer_len);
- bit_width_ = bit_width;
- current_value_ = 0;
- repeat_count_ = 0;
- literal_count_ = 0;
- }
+ explicit RleDecoder(run_type const& run) noexcept;
+
+ void Reset(run_type const& run) noexcept;
+
+ /// Return the number of values that can be advanced.
+ [[nodiscard]] values_count_type Remaining() const;
Review Comment:
> For all functions reading data, I'd consider it dangerous not to read the output (e.g. number of bytes read) since it carries error information (e.g. when it is zero). What do you think?
Ah! Well, in that case we can keep it indeed. But for simple accessors it seems superfluous.
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ ARROW_DCHECK_LT(values_read, batch_size);
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+ auto const read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
return values_read;
}
-template <typename T, typename RunType, typename Converter>
-inline int RleDecoder::GetSpaced(Converter converter, int batch_size, int null_count,
- const uint8_t* valid_bits, int64_t valid_bits_offset,
- T* out) {
- if (ARROW_PREDICT_FALSE(null_count == batch_size)) {
- converter.FillZero(out, out + batch_size);
- return batch_size;
+namespace internal {
+
+/// Utility class to safely handle values and null count without too error-prone
+/// verbosity.
+class BatchCounter {
+ public:
+ using size_type = int32_t;
+
+ [[nodiscard]] static constexpr BatchCounter FromBatchSizeAndNulls(
+ size_type batch_size, size_type null_count) {
+ ARROW_DCHECK_LE(null_count, batch_size);
+ return {batch_size - null_count, null_count};
}
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
- int values_remaining = batch_size - null_count;
+ constexpr BatchCounter(size_type values_count, size_type null_count) noexcept
+ : values_count_(values_count), null_count_(null_count) {}
- // Assume no bits to start.
- arrow::internal::BitRunReader bit_reader(valid_bits, valid_bits_offset,
- /*length=*/batch_size);
- arrow::internal::BitRun valid_run = bit_reader.NextRun();
- while (values_read < batch_size) {
- if (ARROW_PREDICT_FALSE(valid_run.length == 0)) {
- valid_run = bit_reader.NextRun();
+ [[nodiscard]] constexpr size_type ValuesCount() const noexcept { return values_count_; }
+
+ [[nodiscard]] constexpr size_type ValuesRead() const noexcept { return values_read_; }
+
+ [[nodiscard]] constexpr size_type ValuesRemaining() const noexcept {
+ ARROW_DCHECK_LE(values_read_, values_count_);
+ return values_count_ - values_read_;
+ }
+
+ constexpr void AccrueReadValues(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, ValuesRemaining());
+ values_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type NullCount() const noexcept { return null_count_; }
+
+ [[nodiscard]] constexpr size_type NullRead() const noexcept { return null_read_; }
+
+ [[nodiscard]] constexpr size_type NullRemaining() const noexcept {
+ ARROW_DCHECK_LE(null_read_, null_count_);
+ return null_count_ - null_read_;
+ }
+
+ constexpr void AccrueReadNulls(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, NullRemaining());
+ null_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type TotalRemaining() const noexcept {
+ return ValuesRemaining() + NullRemaining();
+ }
+
+ [[nodiscard]] constexpr size_type TotalRead() const noexcept {
+ return values_read_ + null_read_;
+ }
+
+ [[nodiscard]] constexpr bool IsFullyNull() const noexcept {
+ return ValuesRemaining() == 0;
+ }
+
+ [[nodiscard]] constexpr bool IsDone() const noexcept { return TotalRemaining() == 0; }
+
+ private:
+ size_type values_count_ = 0;
+ size_type values_read_ = 0;
+ size_type null_count_ = 0;
+ size_type null_read_ = 0;
+};
+
+// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+/// Overload for GetSpaced for a single run in a RleDecoder
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ RleDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto values_remaining_run = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // Consume as much as possible from the repeated run.
+ // We only need to count the number of nulls and non-nulls because we can fill in the
+ // same value for nulls and non-nulls.
+ // This proves to be a big efficiency win.
+ while (values_remaining_run() > 0 && !batch.IsDone()) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const& validity_run_size = static_cast<values_count_type>(validity_run.length);
+
+ if (validity_run.set) {
+ // We may end the current RLE run in the middle of the validity run
+ auto update_size = std::min(validity_run_size, values_remaining_run());
+ batch.AccrueReadValues(update_size);
+ validity_run.length -= update_size;
+ } else {
+ // We can consume all nulls here because it does not matter if we consume on this
+ // RLE run, or an a next encoded run. The value filled does not matter.
+ auto update_size = std::min(validity_run_size, batch.NullRemaining());
+ batch.AccrueReadNulls(update_size);
+ validity_run.length -= update_size;
+ }
+
+ if (ARROW_PREDICT_TRUE(validity_run.length == 0)) {
+ validity_run = validity_reader.NextRun();
}
+ }
+
+ value_type const value = decoder.Value();
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(value))) {
+ return {0, 0};
+ }
+ converter.WriteRepeated(out, out + batch.TotalRead(), value);
+ auto const actual_values_read = decoder.Advance(batch.ValuesRead());
+ // We always cropped the number of values_read by the remaining values in the run.
+ // What's more the RLE decoder should not encounter any errors.
+ ARROW_DCHECK_EQ(actual_values_read, batch.ValuesRead());
- ARROW_DCHECK_GT(batch_size, 0);
- ARROW_DCHECK_GT(valid_run.length, 0);
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+template <typename T, typename... Ts>
+[[nodiscard]] constexpr T min(T x, Ts... ys) {
+ ((x = std::min(x, ys)), ...);
+ return x;
+}
+
+static_assert(min(5) == 5);
+static_assert(min(5, 4, -1) == -1);
+static_assert(min(5, 41) == 5);
+
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ BitPackedDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto run_values_remaining = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ while (run_values_remaining() > 0 && batch.ValuesRemaining() > 0) {
+ // TODO should this size be tune depending on sizeof(value_size)? cpu cache size?
Review Comment:
Do you plan to investigate this later? Perhaps open an issue?
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -602,20 +1038,126 @@ struct GetBatchSpacedTestCase {
int bit_width;
};
-TEST(RleDecoder, GetBatchSpaced) {
- uint32_t kSeed = 1337;
- ::arrow::random::RandomArrayGenerator rand(kSeed);
-
- std::vector<GetBatchSpacedTestCase<int32_t>> int32_cases{
- {1, 100000, 0.01, 1}, {1, 100000, 0.1, 1}, {1, 100000, 0.5, 1},
- {4, 100000, 0.05, 3}, {100, 100000, 0.05, 7},
+template <typename T>
+void DoTestGetBatchSpacedRoundtrip() {
+ using Data = DataTestRleBitPacked<T>;
+ using ArrowType = typename Data::ArrowType;
+ using RandomPart = typename Data::RandomPart;
+ using NullPart = typename Data::NullPart;
+ using RepeatPart = typename Data::RepeatPart;
+
+ std::vector<Data> test_cases = {
+ {
+ {RandomPart{/* max=*/1, /* size=*/400, /* null_proba= */ 0.1}},
+ /* bit_width= */ 1,
+ },
+ {
+ {
+ RandomPart{/* max=*/7, /* size=*/10037, /* null_proba= */ 0.0},
+ NullPart{/* size= */ 1153},
+ RandomPart{/* max=*/7, /* size=*/800, /* null_proba= */ 0.5},
+ },
+ /* bit_width= */ 3,
+ },
+ {
+ {
+ NullPart{/* size= */ 80},
+ RandomPart{/* max=*/static_cast<T>(1023), /* size=*/800,
+ /* null_proba= */ 0.01},
+ NullPart{/* size= */ 1023},
+ },
+ /* bit_width= */ 11,
+ },
+ {
+ {RepeatPart{/* value=*/13, /* size=*/100000, /* null_proba= */ 0.01}},
+ /* bit_width= */ 10,
+ },
+ {
+ {
+ NullPart{/* size= */ 1024},
+ RepeatPart{/* value=*/static_cast<T>(10000), /* size=*/100000,
+ /* null_proba= */ 0.1},
+ NullPart{/* size= */ 77},
+ },
+ /* bit_width= */ 23,
+ },
+ {
+ {
+ RepeatPart{/* value=*/13, /* size=*/100000, /* null_proba= */ 0.0},
+ NullPart{/* size= */ 1153},
+ RepeatPart{/* value=*/72, /* size=*/100799, /* null_proba= */ 0.5},
+ },
+ /* bit_width= */ 10,
+ },
+ {
+ {
+ RandomPart{/* max=*/1, /* size=*/1013, /* null_proba= */ 0.01},
+ NullPart{/* size=*/8},
+ RepeatPart{1, /* size= */ 256, /* null_proba= */ 0.1},
+ NullPart{/* size=*/128},
+ RepeatPart{0, /* size= */ 256, /* null_proba= */ 0.0},
+ NullPart{/* size=*/15},
+ RandomPart{/* max=*/1, /* size=*/8 * 1024, /* null_proba= */ 0.01},
+ },
+ /* bit_width= */ 1,
+ },
};
- for (auto case_ : int32_cases) {
- auto arr = rand.Int32(case_.size, /*min=*/0, case_.max_value, case_.null_probability);
- CheckRoundTripSpaced<Int32Type>(*arr, case_.bit_width);
- CheckRoundTripSpaced<Int32Type>(*arr->Slice(1), case_.bit_width);
+
+ ::arrow::random::RandomArrayGenerator rand(/* seed= */ 12);
+ // FRAGILE: we create a dictionary large enough so that any encoded value from the
+ // previous test cases can be used as an index in the dictionary.
+ // Its size must be increased accordingly if larger values are encoded in the test
+ // cases.
+ auto dict = std::static_pointer_cast<arrow::FloatArray>(rand.Float32(20000, -1.0, 1.0));
+
+ // Number of bits available in T to write a positive integer.
+ constexpr int kBitsAvailable = 8 * sizeof(T) - (std::is_signed_v<T> ? 1 : 0);
Review Comment:
Most actual data (whether dictionary indices or definition levels) will opportunistically use a smaller bit-width, for the record.
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -207,12 +209,310 @@ TEST(BitUtil, RoundTripIntValues) {
}
}
+/// A Rle run is a simple class owning some data and a repetition count.
+/// It does not know how to read such data.
+TEST(Rle, RleRun) {
+ const std::array<uint8_t, 4> value = {21, 2, 0, 0};
+
+ RleRun::values_count_type value_count = 12;
+
+ // 12 times the value 21 fitting over 5 bits
+ const auto run_5 = RleRun(value.data(), value_count, /* value_bit_width= */ 5);
+ EXPECT_EQ(run_5.ValuesCount(), value_count);
+ EXPECT_EQ(run_5.ValuesBitWidth(), 5);
+ EXPECT_EQ(run_5.RawDataSize(), 1); // 5 bits fit in one byte
+ EXPECT_EQ(*run_5.RawDataPtr(), 21);
+
+ // 12 times the value 21 fitting over 16 bits
Review Comment:
```suggestion
// 12 times the value 21 fitting over 8 bits
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2363120215
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +612,663 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/************************
+ * RleBitPackedParser *
+ ************************/
+
+template <typename Handler>
+void RleBitPackedParser::Parse(Handler&& handler) {
+ while (!exhausted()) {
+ auto [read, control] = PeekImpl(handler);
+ data_ += read;
+ data_size_ -= read;
+ if (ARROW_PREDICT_FALSE(control == ControlFlow::Break)) {
+ break;
+ }
+ }
+}
+
+namespace internal {
+/// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+} // namespace internal
+
+template <typename Handler>
+auto RleBitPackedParser::PeekImpl(Handler&& handler) const
+ -> std::pair<rle_size_t, ControlFlow> {
+ ARROW_DCHECK(!exhausted());
+
+ constexpr auto kMaxSize = bit_util::kMaxLEB128ByteLenFor<uint32_t>;
+ uint32_t run_len_type = 0;
+ const auto header_bytes = bit_util::ParseLeadingLEB128(data_, kMaxSize, &run_len_type);
+
+ if (ARROW_PREDICT_FALSE(header_bytes == 0)) {
+ // Malfomrmed LEB128 data
+ return {};
+ }
+
+ const bool is_bit_packed = run_len_type & 1;
+ const uint32_t count = run_len_type >> 1;
+ if (is_bit_packed) {
+ constexpr auto kMaxCount = bit_util::CeilDiv(internal::max_size_for_v<rle_size_t>, 8);
+ if (ARROW_PREDICT_FALSE(count == 0 || count > kMaxCount)) {
+ // Illegal number of encoded values
+ return {0, ControlFlow::Break};
+ }
+
+ const auto values_count = static_cast<rle_size_t>(count * 8);
+ ARROW_DCHECK_LT(count, internal::max_size_for_v<rle_size_t>);
Review Comment:
I changed for a check useful before the `static_cast`.
```
ARROW_DCHECK_LT(static_cast<uint64_t>(count) * 8, internal::max_size_for_v<rle_size_t>);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2368427502
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -82,86 +81,400 @@ namespace util {
/// 200 ints = 25 groups of 8
/// <varint((25 << 1) | 1)> <25 bytes of values, bitpacked>
/// (total 26 bytes, 1 byte overhead)
-//
-/// Decoder class for RLE encoded data.
-class RleDecoder {
+/// The type for an encoded Rle of BitPacked run size, between 1 and 2^31-1 as per Parquet
+/// spec.
+/// This is also pragmatically used for other integer used in the Rle and BitPacked runs
+/// and decoder to avoid conversions.
+/// It can therefore be referred to as a "typical" size for Rle and BitPacked logic.
+using rle_size_t = int32_t;
+
+template <typename T>
+class RleRunDecoder;
+
+class RleRun {
public:
- /// Create a decoder object. buffer/buffer_len is the decoded data.
- /// bit_width is the width of each value (before encoding).
- RleDecoder(const uint8_t* buffer, int buffer_len, int bit_width)
- : bit_reader_(buffer, buffer_len),
- bit_width_(bit_width),
- current_value_(0),
- repeat_count_(0),
- literal_count_(0) {
- ARROW_DCHECK_GE(bit_width_, 0);
- ARROW_DCHECK_LE(bit_width_, 64);
+ /// The decoder class used to decode a single run in the given type.
+ template <typename T>
+ using DecoderType = RleRunDecoder<T>;
+
+ constexpr RleRun() noexcept = default;
+
+ explicit RleRun(const uint8_t* data, rle_size_t values_count,
+ rle_size_t value_bit_width) noexcept
+ : values_count_(values_count), value_bit_width_(value_bit_width) {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_GE(values_count, 0);
+ std::copy(data, data + raw_data_size(), data_.begin());
}
- RleDecoder() : bit_width_(-1) {}
+ /// The number of repeated values in this run.
+ constexpr rle_size_t values_count() const noexcept { return values_count_; }
- void Reset(const uint8_t* buffer, int buffer_len, int bit_width) {
- ARROW_DCHECK_GE(bit_width, 0);
- ARROW_DCHECK_LE(bit_width, 64);
- bit_reader_.Reset(buffer, buffer_len);
- bit_width_ = bit_width;
- current_value_ = 0;
- repeat_count_ = 0;
- literal_count_ = 0;
+ /// The size in bits of each encoded value.
+ constexpr rle_size_t values_bit_width() const noexcept { return value_bit_width_; }
+
+ /// A pointer to the repeated value raw bytes.
+ constexpr const uint8_t* raw_data_ptr() const noexcept { return data_.data(); }
+
+ /// The number of bytes used for the raw repeated value.
+ constexpr rle_size_t raw_data_size() const noexcept {
+ auto out = bit_util::BytesForBits(value_bit_width_);
+ ARROW_DCHECK_LE(out, std::numeric_limits<rle_size_t>::max());
+ return static_cast<rle_size_t>(out);
}
- /// Gets the next value. Returns false if there are no more.
+ private:
+ /// The repeated value raw bytes stored inside the class with enough space to store
+ /// up to a 64 bit value.
+ std::array<uint8_t, 8> data_ = {};
+ /// The number of time the value is repeated.
+ rle_size_t values_count_ = 0;
+ /// The size in bit of a packed value in the run.
+ rle_size_t value_bit_width_ = 0;
+};
+
+template <typename T>
+class BitPackedRunDecoder;
+
+class BitPackedRun {
+ public:
+ /// The decoder class used to decode a single run in the given type.
+ template <typename T>
+ using DecoderType = BitPackedRunDecoder<T>;
+
+ constexpr BitPackedRun() noexcept = default;
+
+ constexpr BitPackedRun(const uint8_t* data, rle_size_t values_count,
+ rle_size_t value_bit_width) noexcept
+ : data_(data), values_count_(values_count), value_bit_width_(value_bit_width) {
+ ARROW_CHECK_GE(value_bit_width_, 0);
+ ARROW_CHECK_GE(values_count_, 0);
+ }
+
+ constexpr rle_size_t values_count() const noexcept { return values_count_; }
+
+ /// The size in bits of each encoded value.
+ constexpr rle_size_t values_bit_width() const noexcept { return value_bit_width_; }
+
+ constexpr const uint8_t* raw_data_ptr() const noexcept { return data_; }
+
+ constexpr rle_size_t raw_data_size() const noexcept {
+ auto out = bit_util::BytesForBits(static_cast<int64_t>(value_bit_width_) *
+ static_cast<int64_t>(values_count_));
+ ARROW_CHECK_LE(out, std::numeric_limits<rle_size_t>::max());
+ return static_cast<rle_size_t>(out);
+ }
+
+ private:
+ /// The pointer to the beginning of the run
+ const uint8_t* data_ = nullptr;
+ /// Number of values in this run.
+ rle_size_t values_count_ = 0;
+ /// The size in bit of a packed value in the run
+ rle_size_t value_bit_width_ = 0;
+};
+
+/// A parser that emits either a ``BitPackedRun`` or a ``RleRun``.
+class RleBitPackedParser {
+ public:
+ /// The different types of runs emitted by the parser
+ using dynamic_run_type = std::variant<RleRun, BitPackedRun>;
+
+ constexpr RleBitPackedParser() noexcept = default;
+
+ constexpr RleBitPackedParser(const uint8_t* data, rle_size_t data_size,
+ rle_size_t value_bit_width) noexcept
+ : data_(data), data_size_(data_size), value_bit_width_(value_bit_width) {}
+
+ constexpr void Reset(const uint8_t* data, rle_size_t data_size,
+ rle_size_t value_bit_width) noexcept {
+ *this = {data, data_size, value_bit_width};
+ }
+
+ /// Whether there is still runs to iterate over.
+ ///
+ /// WARN: Due to simplistic error handling, iteration with Next and Peek could
+ /// fail to return data while the parser is not exhausted.
+ /// This is how one can check for errors.
+ bool exhausted() const { return data_size_ == 0; }
+
+ /// Enum to return from an ``Parse`` handler.
+ ///
+ /// Since a callback has no way to know when to stop, the handler must return
+ /// a value indicating to the ``Parse`` function whether to stop or continue.
+ enum class ControlFlow {
+ Continue,
+ Break,
+ };
+
+ /// A callback approach to parsing.
+ ///
+ /// This approach is used to reduce the number of dynamic lookups involved with using a
+ /// variant.
+ ///
+ /// The handler must be of the form
+ /// ```cpp
+ /// struct Handler {
+ /// ControlFlow OnBitPackedRun(BitPackedRun run);
+ ///
+ /// ControlFlow OnRleRun(RleRun run);
+ /// };
+ /// ```
+ template <typename Handler>
+ void Parse(Handler&& handler);
+
+ private:
+ /// The pointer to the beginning of the run
+ const uint8_t* data_ = nullptr;
+ /// Size in bytes of the run.
+ rle_size_t data_size_ = 0;
+ /// The size in bit of a packed value in the run
+ rle_size_t value_bit_width_ = 0;
Review Comment:
Actually we cannot "reconstruct a parser" through assignment if one of the member is const, so this is not easily doable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2363057652
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -82,86 +81,400 @@ namespace util {
/// 200 ints = 25 groups of 8
/// <varint((25 << 1) | 1)> <25 bytes of values, bitpacked>
/// (total 26 bytes, 1 byte overhead)
-//
-/// Decoder class for RLE encoded data.
-class RleDecoder {
+/// The type for an encoded Rle of BitPacked run size, between 1 and 2^31-1 as per Parquet
+/// spec.
+/// This is also pragmatically used for other integer used in the Rle and BitPacked runs
+/// and decoder to avoid conversions.
+/// It can therefore be referred to as a "typical" size for Rle and BitPacked logic.
+using rle_size_t = int32_t;
+
+template <typename T>
+class RleRunDecoder;
+
+class RleRun {
public:
- /// Create a decoder object. buffer/buffer_len is the decoded data.
- /// bit_width is the width of each value (before encoding).
- RleDecoder(const uint8_t* buffer, int buffer_len, int bit_width)
- : bit_reader_(buffer, buffer_len),
- bit_width_(bit_width),
- current_value_(0),
- repeat_count_(0),
- literal_count_(0) {
- ARROW_DCHECK_GE(bit_width_, 0);
- ARROW_DCHECK_LE(bit_width_, 64);
+ /// The decoder class used to decode a single run in the given type.
+ template <typename T>
+ using DecoderType = RleRunDecoder<T>;
+
+ constexpr RleRun() noexcept = default;
+
+ explicit RleRun(const uint8_t* data, rle_size_t values_count,
+ rle_size_t value_bit_width) noexcept
+ : values_count_(values_count), value_bit_width_(value_bit_width) {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_GE(values_count, 0);
+ std::copy(data, data + raw_data_size(), data_.begin());
}
- RleDecoder() : bit_width_(-1) {}
+ /// The number of repeated values in this run.
+ constexpr rle_size_t values_count() const noexcept { return values_count_; }
- void Reset(const uint8_t* buffer, int buffer_len, int bit_width) {
- ARROW_DCHECK_GE(bit_width, 0);
- ARROW_DCHECK_LE(bit_width, 64);
- bit_reader_.Reset(buffer, buffer_len);
- bit_width_ = bit_width;
- current_value_ = 0;
- repeat_count_ = 0;
- literal_count_ = 0;
+ /// The size in bits of each encoded value.
+ constexpr rle_size_t values_bit_width() const noexcept { return value_bit_width_; }
+
+ /// A pointer to the repeated value raw bytes.
+ constexpr const uint8_t* raw_data_ptr() const noexcept { return data_.data(); }
+
+ /// The number of bytes used for the raw repeated value.
+ constexpr rle_size_t raw_data_size() const noexcept {
+ auto out = bit_util::BytesForBits(value_bit_width_);
+ ARROW_DCHECK_LE(out, std::numeric_limits<rle_size_t>::max());
+ return static_cast<rle_size_t>(out);
}
- /// Gets the next value. Returns false if there are no more.
+ private:
+ /// The repeated value raw bytes stored inside the class with enough space to store
+ /// up to a 64 bit value.
+ std::array<uint8_t, 8> data_ = {};
+ /// The number of time the value is repeated.
+ rle_size_t values_count_ = 0;
+ /// The size in bit of a packed value in the run.
+ rle_size_t value_bit_width_ = 0;
Review Comment:
For both type of run they serve to know how many bytes are represented in the run (`raw_data_size`), and more generally to make the runs self sufficient/descriptive, whether that is in the `RleBitPackedParser::Parse` callback, or for future use of the runs without a decoder.
Note that they basically come for free because the runs would otherwise be 96 bits and padded to 128.
I understand that it conflicts with making it clear that this does not change for the whole Mixed RLE/BitPacked data.
What do you think? Should we keep them or remove them?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2363176222
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +612,663 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/************************
+ * RleBitPackedParser *
+ ************************/
+
+template <typename Handler>
+void RleBitPackedParser::Parse(Handler&& handler) {
+ while (!exhausted()) {
+ auto [read, control] = PeekImpl(handler);
+ data_ += read;
+ data_size_ -= read;
+ if (ARROW_PREDICT_FALSE(control == ControlFlow::Break)) {
+ break;
+ }
+ }
+}
+
+namespace internal {
+/// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+} // namespace internal
+
+template <typename Handler>
+auto RleBitPackedParser::PeekImpl(Handler&& handler) const
+ -> std::pair<rle_size_t, ControlFlow> {
+ ARROW_DCHECK(!exhausted());
+
+ constexpr auto kMaxSize = bit_util::kMaxLEB128ByteLenFor<uint32_t>;
+ uint32_t run_len_type = 0;
+ const auto header_bytes = bit_util::ParseLeadingLEB128(data_, kMaxSize, &run_len_type);
+
+ if (ARROW_PREDICT_FALSE(header_bytes == 0)) {
+ // Malfomrmed LEB128 data
+ return {};
+ }
+
+ const bool is_bit_packed = run_len_type & 1;
+ const uint32_t count = run_len_type >> 1;
+ if (is_bit_packed) {
+ constexpr auto kMaxCount = bit_util::CeilDiv(internal::max_size_for_v<rle_size_t>, 8);
+ if (ARROW_PREDICT_FALSE(count == 0 || count > kMaxCount)) {
+ // Illegal number of encoded values
+ return {0, ControlFlow::Break};
+ }
+
+ const auto values_count = static_cast<rle_size_t>(count * 8);
+ ARROW_DCHECK_LT(count, internal::max_size_for_v<rle_size_t>);
+ // Count Already divided by 8
+ const auto bytes_read =
+ header_bytes + static_cast<rle_size_t>(count) * value_bit_width_;
+
+ auto control = handler.OnBitPackedRun(
+ BitPackedRun(data_ + header_bytes, values_count, value_bit_width_));
+
+ return {bytes_read, control};
+ }
+
+ if (ARROW_PREDICT_FALSE(
+ count == 0 ||
+ count > static_cast<uint32_t>(std::numeric_limits<rle_size_t>::max()))) {
+ // Illegal number of encoded values
+ return {0, ControlFlow::Break};
+ }
+
+ const auto values_count = static_cast<rle_size_t>(count);
+ const auto value_bytes = bit_util::BytesForBits(value_bit_width_);
+ ARROW_DCHECK_LT(value_bytes, internal::max_size_for_v<rle_size_t>);
+ const auto bytes_read = header_bytes + static_cast<rle_size_t>(value_bytes);
+
+ auto control =
+ handler.OnRleRun(RleRun(data_ + header_bytes, values_count, value_bit_width_));
+
+ return {bytes_read, control};
+}
+
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+template <typename Callable>
+void RleBitPackedDecoder<T>::ParseWithCallable(Callable&& func) {
+ struct {
+ Callable func;
+ auto OnBitPackedRun(BitPackedRun run) { return func(std::move(run)); }
+ auto OnRleRun(RleRun run) { return func(std::move(run)); }
+ } handler{std::move(func)};
+
+ parser_.Parse(std::move(handler));
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, rle_size_t batch_size)
+ -> rle_size_t {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ rle_size_t values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(run_remaining() > 0)) {
+ const auto read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(run_remaining() == 0);
}
+ ParseWithCallable([&](auto run) {
+ using RunDecoder = typename decltype(run)::template DecoderType<value_type>;
+
+ ARROW_DCHECK_LT(values_read, batch_size);
+ RunDecoder decoder(run);
+ const auto read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
Review Comment:
No, here `values_read == batch_size` implies that we have read as many values as requested *but* there may be remaining values in the to decode in the run.
The run is immutable, and the decoder keeps the state within a run between `GetBatch` calls.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2337010039
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
Review Comment:
Yes (move assignment), but more precisely to clear the decoder (make sure we don't hold on to old pointers).
The actual initializing is done lazily in `GetXXX` functions as we parse new runs.
The decoder mostly keep a state if we stop in a middle of a run in `GetXXX`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2337148078
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ ARROW_DCHECK_LT(values_read, batch_size);
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+ auto const read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
return values_read;
}
-template <typename T, typename RunType, typename Converter>
-inline int RleDecoder::GetSpaced(Converter converter, int batch_size, int null_count,
- const uint8_t* valid_bits, int64_t valid_bits_offset,
- T* out) {
- if (ARROW_PREDICT_FALSE(null_count == batch_size)) {
- converter.FillZero(out, out + batch_size);
- return batch_size;
+namespace internal {
+
+/// Utility class to safely handle values and null count without too error-prone
+/// verbosity.
+class BatchCounter {
+ public:
+ using size_type = int32_t;
+
+ [[nodiscard]] static constexpr BatchCounter FromBatchSizeAndNulls(
+ size_type batch_size, size_type null_count) {
+ ARROW_DCHECK_LE(null_count, batch_size);
+ return {batch_size - null_count, null_count};
}
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
- int values_remaining = batch_size - null_count;
+ constexpr BatchCounter(size_type values_count, size_type null_count) noexcept
+ : values_count_(values_count), null_count_(null_count) {}
- // Assume no bits to start.
- arrow::internal::BitRunReader bit_reader(valid_bits, valid_bits_offset,
- /*length=*/batch_size);
- arrow::internal::BitRun valid_run = bit_reader.NextRun();
- while (values_read < batch_size) {
- if (ARROW_PREDICT_FALSE(valid_run.length == 0)) {
- valid_run = bit_reader.NextRun();
+ [[nodiscard]] constexpr size_type ValuesCount() const noexcept { return values_count_; }
+
+ [[nodiscard]] constexpr size_type ValuesRead() const noexcept { return values_read_; }
+
+ [[nodiscard]] constexpr size_type ValuesRemaining() const noexcept {
+ ARROW_DCHECK_LE(values_read_, values_count_);
+ return values_count_ - values_read_;
+ }
+
+ constexpr void AccrueReadValues(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, ValuesRemaining());
+ values_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type NullCount() const noexcept { return null_count_; }
+
+ [[nodiscard]] constexpr size_type NullRead() const noexcept { return null_read_; }
+
+ [[nodiscard]] constexpr size_type NullRemaining() const noexcept {
+ ARROW_DCHECK_LE(null_read_, null_count_);
+ return null_count_ - null_read_;
+ }
+
+ constexpr void AccrueReadNulls(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, NullRemaining());
+ null_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type TotalRemaining() const noexcept {
+ return ValuesRemaining() + NullRemaining();
+ }
+
+ [[nodiscard]] constexpr size_type TotalRead() const noexcept {
+ return values_read_ + null_read_;
+ }
+
+ [[nodiscard]] constexpr bool IsFullyNull() const noexcept {
+ return ValuesRemaining() == 0;
+ }
+
+ [[nodiscard]] constexpr bool IsDone() const noexcept { return TotalRemaining() == 0; }
+
+ private:
+ size_type values_count_ = 0;
+ size_type values_read_ = 0;
+ size_type null_count_ = 0;
+ size_type null_read_ = 0;
+};
+
+// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+/// Overload for GetSpaced for a single run in a RleDecoder
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ RleDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto values_remaining_run = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // Consume as much as possible from the repeated run.
+ // We only need to count the number of nulls and non-nulls because we can fill in the
+ // same value for nulls and non-nulls.
+ // This proves to be a big efficiency win.
+ while (values_remaining_run() > 0 && !batch.IsDone()) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const& validity_run_size = static_cast<values_count_type>(validity_run.length);
+
+ if (validity_run.set) {
+ // We may end the current RLE run in the middle of the validity run
+ auto update_size = std::min(validity_run_size, values_remaining_run());
+ batch.AccrueReadValues(update_size);
+ validity_run.length -= update_size;
+ } else {
+ // We can consume all nulls here because it does not matter if we consume on this
+ // RLE run, or an a next encoded run. The value filled does not matter.
+ auto update_size = std::min(validity_run_size, batch.NullRemaining());
+ batch.AccrueReadNulls(update_size);
+ validity_run.length -= update_size;
+ }
+
+ if (ARROW_PREDICT_TRUE(validity_run.length == 0)) {
+ validity_run = validity_reader.NextRun();
}
+ }
+
+ value_type const value = decoder.Value();
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(value))) {
+ return {0, 0};
+ }
+ converter.WriteRepeated(out, out + batch.TotalRead(), value);
+ auto const actual_values_read = decoder.Advance(batch.ValuesRead());
+ // We always cropped the number of values_read by the remaining values in the run.
+ // What's more the RLE decoder should not encounter any errors.
+ ARROW_DCHECK_EQ(actual_values_read, batch.ValuesRead());
- ARROW_DCHECK_GT(batch_size, 0);
- ARROW_DCHECK_GT(valid_run.length, 0);
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+template <typename T, typename... Ts>
+[[nodiscard]] constexpr T min(T x, Ts... ys) {
+ ((x = std::min(x, ys)), ...);
+ return x;
+}
+
+static_assert(min(5) == 5);
+static_assert(min(5, 4, -1) == -1);
+static_assert(min(5, 41) == 5);
Review Comment:
Removed, it was only used once.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2337190377
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -207,12 +209,310 @@ TEST(BitUtil, RoundTripIntValues) {
}
}
+/// A Rle run is a simple class owning some data and a repetition count.
+/// It does not know how to read such data.
+TEST(Rle, RleRun) {
+ const std::array<RleRun::byte, 4> value = {21, 2, 0, 0};
+
+ RleRun::values_count_type value_count = 12;
+
+ // 12 times the value 21 fitting over 5 bits
+ auto const run_5 = RleRun(value.data(), value_count, /* value_bit_width= */ 5);
+ EXPECT_EQ(run_5.ValuesCount(), value_count);
+ EXPECT_EQ(run_5.ValuesBitWidth(), 5);
+ EXPECT_EQ(run_5.RawDataSize(), 1); // 5 bits fit in one byte
+ EXPECT_EQ(*run_5.RawDataPtr(), 21);
+
+ // 12 times the value 21 fitting over 16 bits
+ auto const run_8 = RleRun(value.data(), value_count, /* value_bit_width= */ 8);
+ EXPECT_EQ(run_8.ValuesCount(), value_count);
+ EXPECT_EQ(run_8.ValuesBitWidth(), 8);
+ EXPECT_EQ(run_8.RawDataSize(), 1); // 8 bits fit in 1 byte
+ EXPECT_EQ(*run_8.RawDataPtr(), 21);
+
+ // 12 times the value {21, 2} fitting over 10 bits
+ auto const run_10 = RleRun(value.data(), value_count, /* value_bit_width= */ 10);
+
+ EXPECT_EQ(run_10.ValuesCount(), value_count);
+ EXPECT_EQ(run_10.ValuesBitWidth(), 10);
+ EXPECT_EQ(run_10.RawDataSize(), 2); // 10 bits fit in 2 bytes
+ EXPECT_EQ(*(run_10.RawDataPtr() + 0), 21);
+ EXPECT_EQ(*(run_10.RawDataPtr() + 1), 2);
+
+ // 12 times the value {21, 2} fitting over 32 bits
+ auto const run_32 = RleRun(value.data(), value_count, /* value_bit_width= */ 32);
+ EXPECT_EQ(run_32.ValuesCount(), value_count);
+ EXPECT_EQ(run_32.ValuesBitWidth(), 32);
+ EXPECT_EQ(run_32.RawDataSize(), 4); // 32 bits fit in 4 bytes
+ EXPECT_EQ(*(run_32.RawDataPtr() + 0), 21);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 1), 2);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 2), 0);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 3), 0);
+}
+
+/// A BitPacked run is a simple class owning some data and its size.
+/// It does not know how to read such data.
+TEST(BitPacked, BitPackedRun) {
+ const std::array<BitPackedRun::byte, 4> value = {0b10101010, 0, 0, 0b1111111};
+
+ /// 16 values of 1 bit for a total of 16 bits
+ BitPackedRun::values_count_type value_count_1 = 16;
+ auto const run_1 = BitPackedRun(value.data(), value_count_1, /* value_bit_width= */ 1);
+ EXPECT_EQ(run_1.ValuesCount(), value_count_1);
+ EXPECT_EQ(run_1.ValuesBitWidth(), 1);
+ EXPECT_EQ(run_1.RawDataSize(), 2); // 16 bits fit in 2 bytes
+ for (BitPackedRun::raw_data_size_type i = 0; i < run_1.RawDataSize(); ++i) {
+ EXPECT_EQ(*(run_1.RawDataPtr() + i), value[i]);
+ }
+
+ /// 8 values of 3 bits for a total of 24 bits
+ BitPackedRun::values_count_type value_count_3 = 8;
+ auto const run_3 = BitPackedRun(value.data(), value_count_3, /* value_bit_width= */ 3);
+ EXPECT_EQ(run_3.ValuesCount(), value_count_3);
+ EXPECT_EQ(run_3.ValuesBitWidth(), 3);
+ EXPECT_EQ(run_3.RawDataSize(), 3); // 24 bits fit in 3 bytes
+ for (BitPackedRun::raw_data_size_type i = 0; i < run_3.RawDataSize(); ++i) {
+ EXPECT_EQ(*(run_3.RawDataPtr() + i), value[i]);
+ }
+}
+
+template <typename T>
+void TestRleDecoder(std::vector<RleRun::byte> bytes,
+ RleRun::values_count_type value_count,
+ RleRun::bit_size_type bit_width) {
+ // Pre-requisite for this test
+ EXPECT_GT(value_count, 6);
+
+ // Compute value associated with bytes encoded as little endian
+ T value = 0;
+ for (std::size_t i = 0; i < bytes.size(); ++i) {
+ value += static_cast<T>(bytes.at(i)) << (8 * i);
+ }
+
+ auto const run = RleRun(bytes.data(), value_count, bit_width);
+
+ auto decoder = RleDecoder<T>(run);
+ std::vector<T> vals = {0, 0};
+
+ EXPECT_EQ(decoder.Remaining(), value_count);
+
+ typename decltype(decoder)::values_count_type read = 0;
+ EXPECT_EQ(decoder.Get(vals.data()), 1);
+ read += 1;
+ EXPECT_EQ(vals.at(0), value);
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ EXPECT_EQ(decoder.Advance(3), 3);
+ read += 3;
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ vals = {0, 0};
+ EXPECT_EQ(decoder.GetBatch(vals.data(), 2), vals.size());
+ EXPECT_EQ(vals.at(0), value);
+ EXPECT_EQ(vals.at(1), value);
+ read += static_cast<decltype(read)>(vals.size());
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ // Exhaust iteration
+ EXPECT_EQ(decoder.Advance(value_count - read), value_count - read);
+ EXPECT_EQ(decoder.Remaining(), 0);
+ EXPECT_EQ(decoder.Advance(1), 0);
+ vals = {0, 0};
+ EXPECT_EQ(decoder.Get(vals.data()), 0);
+ EXPECT_EQ(vals.at(0), 0);
+
+ // Reset the decoder
+ decoder.Reset(run);
+ EXPECT_EQ(decoder.Remaining(), value_count);
+ vals = {0, 0};
+ EXPECT_EQ(decoder.GetBatch(vals.data(), 2), vals.size());
+ EXPECT_EQ(vals.at(0), value);
+ EXPECT_EQ(vals.at(1), value);
+}
+
+TEST(Rle, RleDecoder) {
+ TestRleDecoder<uint32_t>({21, 0, 0}, /* value_count= */ 21, /* bit_width= */ 5);
+ TestRleDecoder<uint16_t>({1, 0}, /* value_count= */ 13, /* bit_width= */ 1);
+ TestRleDecoder<uint64_t>({21, 2, 0, 1}, /* value_count= */ 20, /* bit_width= */ 30);
+}
+
+template <typename T>
+void TestBitPackedDecoder(std::vector<RleRun::byte> bytes,
+ BitPackedRun::values_count_type value_count,
+ BitPackedRun::bit_size_type bit_width,
+ std::vector<T> expected) {
+ // Pre-requisite for this test
+ EXPECT_GT(value_count, 6);
+
+ auto const run = BitPackedRun(bytes.data(), value_count, bit_width);
+
+ auto decoder = BitPackedDecoder<T>(run);
+ std::vector<T> vals = {0, 0};
+
+ EXPECT_EQ(decoder.Remaining(), value_count);
+
+ typename decltype(decoder)::values_count_type read = 0;
+ EXPECT_EQ(decoder.Get(vals.data()), 1);
+ EXPECT_EQ(vals.at(0), expected.at(0 + read));
+ read += 1;
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ EXPECT_EQ(decoder.Advance(3), 3);
+ read += 3;
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ vals = {0, 0};
+ EXPECT_EQ(decoder.GetBatch(vals.data(), 2), vals.size());
+ EXPECT_EQ(vals.at(0), expected.at(0 + read));
+ EXPECT_EQ(vals.at(1), expected.at(1 + read));
+ read += static_cast<decltype(read)>(vals.size());
+ EXPECT_EQ(decoder.Remaining(), value_count - read);
+
+ // Exhaust iteration
+ EXPECT_EQ(decoder.Advance(value_count - read), value_count - read);
+ EXPECT_EQ(decoder.Remaining(), 0);
+ EXPECT_EQ(decoder.Advance(1), 0);
+ vals = {0, 0};
+ EXPECT_EQ(decoder.Get(vals.data()), 0);
+ EXPECT_EQ(vals.at(0), 0);
+
+ // Reset the decoder
+ decoder.Reset(run);
+ read = 0;
+ EXPECT_EQ(decoder.Remaining(), value_count);
+ vals = {0, 0};
+ EXPECT_EQ(decoder.GetBatch(vals.data(), 2), vals.size());
+ EXPECT_EQ(vals.at(0), expected.at(0 + read));
+ EXPECT_EQ(vals.at(1), expected.at(1 + read));
+}
+
+TEST(BitPacked, BitPackedDecoder) {
+ /// See parquet encoding for bytes layout
+ TestBitPackedDecoder<uint16_t>(
+ /* bytes= */ {0x88, 0xc6, 0xfa},
+ /* values_count= */ 8,
+ /* bit_width= */ 3,
+ /* expected= */ {0, 1, 2, 3, 4, 5, 6, 7});
+ TestBitPackedDecoder<uint8_t>(
+ /* bytes= */ {0x0, 0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7},
+ /* values_count= */ 8,
+ /* bit_width= */ 8,
+ /* expected= */ {0, 1, 2, 3, 4, 5, 6, 7});
+ TestBitPackedDecoder<uint32_t>(
+ /* bytes= */ {0x47, 0xc, 0x10, 0x35},
+ /* values_count= */ 8,
+ /* bit_width= */ 4,
+ /* expected= */ {7, 4, 12, 0, 0, 1, 5, 3});
+ TestBitPackedDecoder<uint64_t>(
+ /* bytes= */ {0xe8, 0x7, 0x20, 0xc0, 0x0, 0x4, 0x14, 0x60, 0xc0, 0x1},
+ /* values_count= */ 8,
+ /* bit_width= */ 10,
+ /* expected= */ {1000, 1, 2, 3, 4, 5, 6, 7});
+}
+
+template <typename T>
+void TestRleBitPackedParser(std::vector<RleRun::byte> bytes,
+ RleBitPackedParser::bit_size_type bit_width,
+ std::vector<T> expected) {
+ auto parser = RleBitPackedParser(
+ bytes.data(), static_cast<BitPackedRun::raw_data_size_type>(bytes.size()),
+ bit_width);
+ EXPECT_FALSE(parser.Exhausted());
+
+ // Peek return the same data
+ auto run1 = parser.Peek();
+ EXPECT_TRUE(run1.has_value());
+ auto run2 = parser.Peek();
+ EXPECT_TRUE(run2.has_value());
+ auto ptr1 = std::visit([](auto const& r) { return r.RawDataPtr(); }, run1.value());
+ auto size1 = std::visit([](auto const& r) { return r.RawDataSize(); }, run1.value());
+ auto ptr2 = std::visit([](auto const& r) { return r.RawDataPtr(); }, run2.value());
+ auto size2 = std::visit([](auto const& r) { return r.RawDataSize(); }, run2.value());
+ EXPECT_TRUE(std::equal(ptr1, ptr1 + size1, ptr2, ptr2 + size2));
Review Comment:
This code was actually removed moving only to the callback API.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #47294:
URL: https://github.com/apache/arrow/pull/47294#issuecomment-3324862090
The benchmarks above are bit of a mixed bag. They show some regressions on select RLE micro-benchmarks (mostly on Arm64 machines), but also a couple improvements on other RLE micro-benchmarks.
However, the slightly higher-level benchmarks in `parquet-arrow-reader-writer-benchmark` don't seem to regress, which is reassuring.
All in all, it seems the future benefits of this outweigh the potential temporary regressions on select workloads.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2319572219
##########
cpp/src/arrow/util/rle_encoding_internal.h:
##########
@@ -299,385 +552,988 @@ class RleEncoder {
uint8_t* literal_indicator_byte_;
};
+/*************************
+ * RleBitPackedDecoder *
+ *************************/
+
+template <typename T>
+RleBitPackedDecoder<T>::RleBitPackedDecoder(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ Reset(data, data_size, value_bit_width);
+}
+
+template <typename T>
+void RleBitPackedDecoder<T>::Reset(raw_data_const_pointer data,
+ raw_data_size_type data_size,
+ bit_size_type value_bit_width) noexcept {
+ ARROW_DCHECK_GE(value_bit_width, 0);
+ ARROW_DCHECK_LE(value_bit_width, 64);
+ parser_.Reset(data, data_size, value_bit_width);
+ decoder_ = {};
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunRemaining() const -> values_count_type {
+ return std::visit([](auto const& dec) { return dec.Remaining(); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Exhausted() const {
+ return (RunRemaining() == 0) && parser_.Exhausted();
+}
+
template <typename T>
-inline bool RleDecoder::Get(T* val) {
+bool RleBitPackedDecoder<T>::ParseAndResetDecoder() {
+ auto dyn_run = parser_.Next();
+ if (!dyn_run.has_value()) {
+ return false;
+ }
+
+ if (auto* rle_run = std::get_if<BitPackedRun>(dyn_run.operator->())) {
+ decoder_ = {BitPackedDecoder<value_type>(*rle_run)};
+ return true;
+ }
+
+ auto* bit_packed_run = std::get_if<RleRun>(dyn_run.operator->());
+ ARROW_DCHECK(bit_packed_run); // Only two possibilities in the variant
+ decoder_ = {RleDecoder<value_type>(*bit_packed_run)};
+ return true;
+}
+
+template <typename T>
+auto RleBitPackedDecoder<T>::RunGetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ return std::visit([&](auto& dec) { return dec.GetBatch(out, batch_size); }, decoder_);
+}
+
+template <typename T>
+bool RleBitPackedDecoder<T>::Get(value_type* val) {
return GetBatch(val, 1) == 1;
}
+namespace internal {
+
+/// A ``Parse`` handler that calls a single lambda.
+///
+/// This lambda would typically take the input run as ``auto run`` (i.e. the lambda is
+/// templated) and deduce other types from it.
+template <typename Lambda>
+struct LambdaHandler {
+ Lambda handlder_;
+
+ auto OnBitPackedRun(BitPackedRun run) { return handlder_(std::move(run)); }
+
+ auto OnRleRun(RleRun run) { return handlder_(std::move(run)); }
+};
+
+template <typename Lambda>
+LambdaHandler(Lambda) -> LambdaHandler<Lambda>;
+
+template <typename value_type, typename Run>
+struct decoder_for;
+
+template <typename value_type>
+struct decoder_for<value_type, BitPackedRun> {
+ using type = BitPackedDecoder<value_type>;
+};
+
+template <typename value_type>
+struct decoder_for<value_type, RleRun> {
+ using type = RleDecoder<value_type>;
+};
+
+template <typename value_type, typename Run>
+using decoder_for_t = typename decoder_for<value_type, Run>::type;
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatch(T* values, int batch_size) {
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) { // Repeated value case.
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, static_cast<T>(current_value_));
-
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(remaining, literal_count_);
- int actual_read = bit_reader_.GetBatch(bit_width_, out, literal_batch);
- if (actual_read != literal_batch) {
- return values_read;
- }
+auto RleBitPackedDecoder<T>::GetBatch(value_type* out, values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
- literal_count_ -= literal_batch;
- values_read += literal_batch;
- out += literal_batch;
- } else {
- if (!NextCounts<T>()) return values_read;
+ values_count_type values_read = 0;
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const read = RunGetBatch(out, batch_size);
+ values_read += read;
+ out += read;
+
+ // Either we fulfilled all the batch to be read or we finished remaining run.
+ if (ARROW_PREDICT_FALSE(values_read == batch_size)) {
+ return values_read;
}
+ ARROW_DCHECK(RunRemaining() == 0);
}
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ ARROW_DCHECK_LT(values_read, batch_size);
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+ auto const read = decoder.GetBatch(out, batch_size - values_read);
+ ARROW_DCHECK_LE(read, batch_size - values_read);
+ values_read += read;
+ out += read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == batch_size || read == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
return values_read;
}
-template <typename T, typename RunType, typename Converter>
-inline int RleDecoder::GetSpaced(Converter converter, int batch_size, int null_count,
- const uint8_t* valid_bits, int64_t valid_bits_offset,
- T* out) {
- if (ARROW_PREDICT_FALSE(null_count == batch_size)) {
- converter.FillZero(out, out + batch_size);
- return batch_size;
+namespace internal {
+
+/// Utility class to safely handle values and null count without too error-prone
+/// verbosity.
+class BatchCounter {
+ public:
+ using size_type = int32_t;
+
+ [[nodiscard]] static constexpr BatchCounter FromBatchSizeAndNulls(
+ size_type batch_size, size_type null_count) {
+ ARROW_DCHECK_LE(null_count, batch_size);
+ return {batch_size - null_count, null_count};
}
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
- int values_remaining = batch_size - null_count;
+ constexpr BatchCounter(size_type values_count, size_type null_count) noexcept
+ : values_count_(values_count), null_count_(null_count) {}
- // Assume no bits to start.
- arrow::internal::BitRunReader bit_reader(valid_bits, valid_bits_offset,
- /*length=*/batch_size);
- arrow::internal::BitRun valid_run = bit_reader.NextRun();
- while (values_read < batch_size) {
- if (ARROW_PREDICT_FALSE(valid_run.length == 0)) {
- valid_run = bit_reader.NextRun();
+ [[nodiscard]] constexpr size_type ValuesCount() const noexcept { return values_count_; }
+
+ [[nodiscard]] constexpr size_type ValuesRead() const noexcept { return values_read_; }
+
+ [[nodiscard]] constexpr size_type ValuesRemaining() const noexcept {
+ ARROW_DCHECK_LE(values_read_, values_count_);
+ return values_count_ - values_read_;
+ }
+
+ constexpr void AccrueReadValues(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, ValuesRemaining());
+ values_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type NullCount() const noexcept { return null_count_; }
+
+ [[nodiscard]] constexpr size_type NullRead() const noexcept { return null_read_; }
+
+ [[nodiscard]] constexpr size_type NullRemaining() const noexcept {
+ ARROW_DCHECK_LE(null_read_, null_count_);
+ return null_count_ - null_read_;
+ }
+
+ constexpr void AccrueReadNulls(size_type to_read) noexcept {
+ ARROW_DCHECK_LE(to_read, NullRemaining());
+ null_read_ += to_read;
+ }
+
+ [[nodiscard]] constexpr size_type TotalRemaining() const noexcept {
+ return ValuesRemaining() + NullRemaining();
+ }
+
+ [[nodiscard]] constexpr size_type TotalRead() const noexcept {
+ return values_read_ + null_read_;
+ }
+
+ [[nodiscard]] constexpr bool IsFullyNull() const noexcept {
+ return ValuesRemaining() == 0;
+ }
+
+ [[nodiscard]] constexpr bool IsDone() const noexcept { return TotalRemaining() == 0; }
+
+ private:
+ size_type values_count_ = 0;
+ size_type values_read_ = 0;
+ size_type null_count_ = 0;
+ size_type null_read_ = 0;
+};
+
+// The maximal unsigned size that a variable can fit.
+template <typename T>
+constexpr auto max_size_for_v =
+ static_cast<std::make_unsigned_t<T>>(std::numeric_limits<T>::max());
+
+/// Overload for GetSpaced for a single run in a RleDecoder
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ RleDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto values_remaining_run = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // Consume as much as possible from the repeated run.
+ // We only need to count the number of nulls and non-nulls because we can fill in the
+ // same value for nulls and non-nulls.
+ // This proves to be a big efficiency win.
+ while (values_remaining_run() > 0 && !batch.IsDone()) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const& validity_run_size = static_cast<values_count_type>(validity_run.length);
+
+ if (validity_run.set) {
+ // We may end the current RLE run in the middle of the validity run
+ auto update_size = std::min(validity_run_size, values_remaining_run());
+ batch.AccrueReadValues(update_size);
+ validity_run.length -= update_size;
+ } else {
+ // We can consume all nulls here because it does not matter if we consume on this
+ // RLE run, or an a next encoded run. The value filled does not matter.
+ auto update_size = std::min(validity_run_size, batch.NullRemaining());
+ batch.AccrueReadNulls(update_size);
+ validity_run.length -= update_size;
+ }
+
+ if (ARROW_PREDICT_TRUE(validity_run.length == 0)) {
+ validity_run = validity_reader.NextRun();
}
+ }
+
+ value_type const value = decoder.Value();
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(value))) {
+ return {0, 0};
+ }
+ converter.WriteRepeated(out, out + batch.TotalRead(), value);
+ auto const actual_values_read = decoder.Advance(batch.ValuesRead());
+ // We always cropped the number of values_read by the remaining values in the run.
+ // What's more the RLE decoder should not encounter any errors.
+ ARROW_DCHECK_EQ(actual_values_read, batch.ValuesRead());
- ARROW_DCHECK_GT(batch_size, 0);
- ARROW_DCHECK_GT(valid_run.length, 0);
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+template <typename T, typename... Ts>
+[[nodiscard]] constexpr T min(T x, Ts... ys) {
+ ((x = std::min(x, ys)), ...);
+ return x;
+}
+
+static_assert(min(5) == 5);
+static_assert(min(5, 4, -1) == -1);
+static_assert(min(5, 41) == 5);
+
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(Converter& converter, typename Converter::out_type* out,
+ values_count_type batch_size, values_count_type null_count,
+ BitRunReader&& validity_reader, BitRun&& validity_run,
+ BitPackedDecoder<value_type>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ ARROW_DCHECK_GT(batch_size, 0);
+ // The equality case is handled in the main loop in GetSpaced
+ ARROW_DCHECK_LT(null_count, batch_size);
+
+ auto batch = BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ values_count_type const values_available = decoder.Remaining();
+ ARROW_DCHECK_GT(values_available, 0);
+ auto run_values_remaining = [&]() {
+ auto out = values_available - batch.ValuesRead();
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ while (run_values_remaining() > 0 && batch.ValuesRemaining() > 0) {
+ // TODO should this size be tune depending on sizeof(value_size)? cpu cache size?
+ // Pull a batch of values from the bit packed encoded data and store it in a local
+ // buffer to benefit from unpacking intrinsics and data locality.
+ static constexpr values_count_type kBufferCapacity = 1024;
+ std::array<value_type, kBufferCapacity> buffer = {};
+
+ values_count_type buffer_start = 0;
+ values_count_type buffer_end = 0;
+ auto buffer_size = [&]() {
+ auto out = buffer_end - buffer_start;
+ ARROW_DCHECK_GE(out, 0);
+ return out;
+ };
+
+ // buffer_start is 0 at this point so size is end
+ buffer_end = min(run_values_remaining(), batch.ValuesRemaining(), kBufferCapacity);
+ buffer_end = decoder.GetBatch(buffer.data(), buffer_size());
+ ARROW_DCHECK_LE(buffer_size(), kBufferCapacity);
+
+ if (ARROW_PREDICT_FALSE(!converter.InputIsValid(buffer.data(), buffer_size()))) {
+ return {batch.ValuesRead(), batch.NullRead()};
+ }
- if (valid_run.set) {
- if ((repeat_count_ == 0) && (literal_count_ == 0)) {
- if (!NextCounts<RunType>()) return values_read;
- ARROW_DCHECK((repeat_count_ > 0) ^ (literal_count_ > 0));
+ // Copy chunks of valid values into the output, while adjusting spacing for null
+ // values.
+ while (buffer_size() > 0) {
+ ARROW_DCHECK_GE(validity_run.length, 0);
+ ARROW_DCHECK_LT(validity_run.length, max_size_for_v<values_count_type>);
+ ARROW_DCHECK_LE(validity_run.length, batch.TotalRemaining());
+ auto const validity_run_length =
+ static_cast<values_count_type>(validity_run.length);
+
+ // Copy as much as possible from the buffer into the output while not exceeding
+ // validity run
+ if (validity_run.set) {
+ auto const update_size = std::min(validity_run_length, buffer_size());
+ converter.WriteRange(out, buffer.data() + buffer_start, update_size);
+ buffer_start += update_size;
+ batch.AccrueReadValues(update_size);
+ out += update_size;
+ validity_run.length -= update_size;
+ // Simply write zeros in the output
+ } else {
+ auto const update_size = std::min(validity_run_length, batch.NullRemaining());
+ converter.WriteZero(out, out + update_size);
+ batch.AccrueReadNulls(update_size);
+ out += update_size;
+ validity_run.length -= update_size;
}
- if (repeat_count_ > 0) {
- int repeat_batch = 0;
- // Consume the entire repeat counts incrementing repeat_batch to
- // be the total of nulls + values consumed, we only need to
- // get the total count because we can fill in the same value for
- // nulls and non-nulls. This proves to be a big efficiency win.
- while (repeat_count_ > 0 && (values_read + repeat_batch) < batch_size) {
- ARROW_DCHECK_GT(valid_run.length, 0);
- if (valid_run.set) {
- int update_size = std::min(static_cast<int>(valid_run.length), repeat_count_);
- repeat_count_ -= update_size;
- repeat_batch += update_size;
- valid_run.length -= update_size;
- values_remaining -= update_size;
- } else {
- // We can consume all nulls here because we would do so on
- // the next loop anyways.
- repeat_batch += static_cast<int>(valid_run.length);
- valid_run.length = 0;
- }
- if (valid_run.length == 0) {
- valid_run = bit_reader.NextRun();
- }
- }
- RunType current_value = static_cast<RunType>(current_value_);
- if (ARROW_PREDICT_FALSE(!converter.IsValid(current_value))) {
- return values_read;
- }
- converter.Fill(out, out + repeat_batch, current_value);
- out += repeat_batch;
- values_read += repeat_batch;
- } else if (literal_count_ > 0) {
- int literal_batch = std::min(values_remaining, literal_count_);
- ARROW_DCHECK_GT(literal_batch, 0);
-
- // Decode the literals
- constexpr int kBufferSize = 1024;
- RunType indices[kBufferSize];
- literal_batch = std::min(literal_batch, kBufferSize);
- int actual_read = bit_reader_.GetBatch(bit_width_, indices, literal_batch);
- if (ARROW_PREDICT_FALSE(actual_read != literal_batch)) {
- return values_read;
- }
- if (!converter.IsValid(indices, /*length=*/actual_read)) {
- return values_read;
- }
- int skipped = 0;
- int literals_read = 0;
- while (literals_read < literal_batch) {
- if (valid_run.set) {
- int update_size = std::min(literal_batch - literals_read,
- static_cast<int>(valid_run.length));
- converter.Copy(out, indices + literals_read, update_size);
- literals_read += update_size;
- out += update_size;
- valid_run.length -= update_size;
- } else {
- converter.FillZero(out, out + valid_run.length);
- out += valid_run.length;
- skipped += static_cast<int>(valid_run.length);
- valid_run.length = 0;
- }
- if (valid_run.length == 0) {
- valid_run = bit_reader.NextRun();
- }
- }
- literal_count_ -= literal_batch;
- values_remaining -= literal_batch;
- values_read += literal_batch + skipped;
+ if (validity_run.length == 0) {
+ validity_run = validity_reader.NextRun();
}
- } else {
- converter.FillZero(out, out + valid_run.length);
- out += valid_run.length;
- values_read += static_cast<int>(valid_run.length);
- valid_run.length = 0;
}
+
+ ARROW_DCHECK_EQ(buffer_size(), 0);
}
- ARROW_DCHECK_EQ(valid_run.length, 0);
- ARROW_DCHECK_EQ(values_remaining, 0);
- return values_read;
+
+ ARROW_DCHECK_EQ(values_available - decoder.Remaining(), batch.ValuesRead());
+ ARROW_DCHECK_LE(batch.TotalRead(), batch_size);
+ ARROW_DCHECK_LE(batch.NullRead(), batch.NullCount());
+
+ return {batch.ValuesRead(), batch.NullRead()};
+}
+
+/// Overload for GetSpaced for a single run in a decoder variant
+template <typename Converter, typename BitRunReader, typename BitRun,
+ typename values_count_type, typename value_type>
+auto RunGetSpaced(
+ Converter& converter, typename Converter::out_type* out, values_count_type batch_size,
+ values_count_type null_count, BitRunReader&& validity_reader, BitRun&& validity_run,
+ std::variant<RleDecoder<value_type>, BitPackedDecoder<value_type>>& decoder)
+ -> std::pair<values_count_type, values_count_type> {
+ return std::visit(
+ [&](auto& dec) {
+ ARROW_DCHECK_GT(dec.Remaining(), 0);
+ return RunGetSpaced(converter, out, batch_size, null_count, validity_reader,
+ validity_run, dec);
+ },
+ decoder);
}
+} // namespace internal
+
+template <typename T>
+template <typename Converter>
+auto RleBitPackedDecoder<T>::GetSpaced(
+ Converter converter, typename Converter::out_type* out, values_count_type batch_size,
+ const byte* validity_bits, int64_t validity_bits_offset, values_count_type null_count)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
+
+ ARROW_DCHECK_GT(batch_size, 0);
+
+ auto batch = internal::BatchCounter::FromBatchSizeAndNulls(batch_size, null_count);
+
+ if (ARROW_PREDICT_FALSE(batch.IsFullyNull())) {
+ converter.WriteZero(out, out + batch.NullRemaining());
+ return batch.NullRemaining();
+ }
+
+ arrow::internal::BitRunReader validity_reader(validity_bits, validity_bits_offset,
+ /*length=*/batch.TotalRemaining());
+ arrow::internal::BitRun validity_run = validity_reader.NextRun();
+
+ auto const check_and_handle_fully_null_remaining = [&]() {
+ if (batch.IsFullyNull()) {
+ ARROW_DCHECK(validity_run.length == 0 || !validity_run.set);
+ ARROW_DCHECK_GE(validity_run.length, batch.NullRemaining());
+
+ converter.WriteZero(out, out + batch.NullRemaining());
+ out += batch.NullRemaining();
+ batch.AccrueReadNulls(batch.NullRemaining());
+ }
+ };
+
+ // Remaining from a previous call that would have left some unread data from a run.
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
+ auto const [values_read, null_read] =
+ RunGetSpaced(converter, out, batch.TotalRemaining(), batch.NullRemaining(),
+ validity_reader, validity_run, decoder_);
+
+ batch.AccrueReadNulls(null_read);
+ batch.AccrueReadValues(values_read);
+ out += values_read + null_read;
+
+ // Either we fulfilled all the batch values to be read
+ if (ARROW_PREDICT_FALSE(batch.ValuesRemaining() == 0)) {
+ // There may be remaining null if they are not greedily filled
+ check_and_handle_fully_null_remaining();
+ return batch.TotalRead();
+ }
+
+ /// We finished the remaining run
+ ARROW_DCHECK(RunRemaining() == 0);
+ }
+
+ auto handler = internal::LambdaHandler{
+ [&](auto run) {
+ internal::decoder_for_t<value_type, decltype(run)> decoder(run);
+
+ auto const [values_read, null_read] = internal::RunGetSpaced(
+ converter, out, batch.TotalRemaining(), batch.NullRemaining(),
+ validity_reader, validity_run, decoder);
+
+ batch.AccrueReadNulls(null_read);
+ batch.AccrueReadValues(values_read);
+ out += values_read + null_read;
+
+ // Stop reading and store remaining decoder
+ if (ARROW_PREDICT_FALSE(values_read == 0 || batch.ValuesRemaining() == 0)) {
+ decoder_ = std::move(decoder);
+ return ControlFlow::Break;
+ }
+
+ return ControlFlow::Continue;
+ },
+ };
+
+ parser_.Parse(handler);
+
+ // There may be remaining null if they are not greedily filled by either decoder calls
+ check_and_handle_fully_null_remaining();
+
+ ARROW_DCHECK(batch.IsDone() || Exhausted());
+ // batch.Done() => batch.NullRemaining() == 0
+ ARROW_DCHECK(!batch.IsDone() || (batch.NullRemaining() == 0));
+ return batch.TotalRead();
+}
+
+namespace internal {
+
// Converter for GetSpaced that handles runs that get returned
// directly as output.
template <typename T>
-struct PlainRleConverter {
- T kZero = {};
- inline bool IsValid(const T& values) const { return true; }
- inline bool IsValid(const T* values, int32_t length) const { return true; }
- inline void Fill(T* begin, T* end, const T& run_value) const {
+struct NoOpConverter {
+ using in_type = T;
+ using out_type = T;
+ using size_type = int32_t;
+
+ static constexpr bool kIsIdentity = true;
+
+ [[nodiscard]] static constexpr bool InputIsValid(const in_type& values) { return true; }
+
+ [[nodiscard]] static constexpr bool InputIsValid(const in_type* values,
+ size_type length) {
+ return true;
+ }
+
+ static void WriteRepeated(out_type* begin, out_type* end, in_type run_value) {
std::fill(begin, end, run_value);
}
- inline void FillZero(T* begin, T* end) { std::fill(begin, end, kZero); }
- inline void Copy(T* out, const T* values, int length) const {
- std::memcpy(out, values, length * sizeof(T));
+
+ static void WriteZero(out_type* begin, out_type* end) {
+ std::fill(begin, end, out_type{});
+ }
+
+ static void WriteRange(out_type* out, const in_type* values, size_type length) {
+ std::memcpy(out, values, length * sizeof(out_type));
}
};
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatchSpaced(int batch_size, int null_count,
- const uint8_t* valid_bits,
- int64_t valid_bits_offset, T* out) {
+auto RleBitPackedDecoder<T>::GetBatchSpaced(values_count_type batch_size,
+ values_count_type null_count,
+ const byte* valid_bits,
+ int64_t valid_bits_offset, value_type* out)
+ -> values_count_type {
if (null_count == 0) {
- return GetBatch<T>(out, batch_size);
+ return GetBatch(out, batch_size);
}
- PlainRleConverter<T> converter;
- arrow::internal::BitBlockCounter block_counter(valid_bits, valid_bits_offset,
- batch_size);
-
- int total_processed = 0;
- int processed = 0;
- arrow::internal::BitBlockCount block;
+ internal::NoOpConverter<value_type> converter;
- do {
- block = block_counter.NextFourWords();
- if (block.length == 0) {
- break;
- }
- if (block.AllSet()) {
- processed = GetBatch<T>(out, block.length);
- } else if (block.NoneSet()) {
- converter.FillZero(out, out + block.length);
- processed = block.length;
- } else {
- processed = GetSpaced<T, /*RunType=*/T, PlainRleConverter<T>>(
- converter, block.length, block.length - block.popcount, valid_bits,
- valid_bits_offset, out);
- }
- total_processed += processed;
- out += block.length;
- valid_bits_offset += block.length;
- } while (processed == block.length);
- return total_processed;
+ return GetSpaced(converter, out, batch_size, valid_bits, valid_bits_offset, null_count);
}
-static inline bool IndexInRange(int32_t idx, int32_t dictionary_length) {
- return idx >= 0 && idx < dictionary_length;
+namespace internal {
+
+template <typename I>
+bool IndexInRange(I idx, int32_t dictionary_length) {
+ ARROW_DCHECK_GT(dictionary_length, 0);
+ using T = std::common_type_t<decltype(idx), decltype(dictionary_length)>;
+ return idx >= 0 && static_cast<T>(idx) < static_cast<T>(dictionary_length);
}
// Converter for GetSpaced that handles runs of returned dictionary
// indices.
-template <typename T>
+template <typename V, typename I>
struct DictionaryConverter {
- T kZero = {};
- const T* dictionary;
- int32_t dictionary_length;
-
- inline bool IsValid(int32_t value) { return IndexInRange(value, dictionary_length); }
-
- inline bool IsValid(const int32_t* values, int32_t length) const {
- using IndexType = int32_t;
- IndexType min_index = std::numeric_limits<IndexType>::max();
- IndexType max_index = std::numeric_limits<IndexType>::min();
- for (int x = 0; x < length; x++) {
- min_index = std::min(values[x], min_index);
- max_index = std::max(values[x], max_index);
+ using out_type = V;
+ using in_type = I;
+ using size_type = int32_t;
+
+ static constexpr bool kIsIdentity = false;
+
+ const out_type* dictionary;
+ size_type dictionary_length;
+
+ [[nodiscard]] bool InputIsValid(in_type idx) const {
+ return IndexInRange(idx, dictionary_length);
+ }
+
+ [[nodiscard]] bool InputIsValid(const in_type* indices, size_type length) const {
+ in_type min_index = std::numeric_limits<in_type>::max();
+ in_type max_index = std::numeric_limits<in_type>::min();
+ for (size_type x = 0; x < length; x++) {
+ min_index = std::min(indices[x], min_index);
+ max_index = std::max(indices[x], max_index);
}
return IndexInRange(min_index, dictionary_length) &&
IndexInRange(max_index, dictionary_length);
}
- inline void Fill(T* begin, T* end, const int32_t& run_value) const {
+
+ void WriteRepeated(out_type* begin, out_type* end, in_type run_value) const {
std::fill(begin, end, dictionary[run_value]);
}
- inline void FillZero(T* begin, T* end) { std::fill(begin, end, kZero); }
- inline void Copy(T* out, const int32_t* values, int length) const {
- for (int x = 0; x < length; x++) {
+ static void WriteZero(out_type* begin, out_type* end) {
+ std::fill(begin, end, out_type{});
+ }
+
+ void WriteRange(out_type* out, const in_type* values, size_type length) const {
+ for (size_type x = 0; x < length; x++) {
out[x] = dictionary[values[x]];
}
}
};
+/// Dummy imitation of BitRun that is all set.
+struct AllSetBitRun {
+ static constexpr bool set = true;
+ int64_t length = 0;
+};
+
+/// Dummy imitation of BitRunReader that should never be called.
+struct UnreachableBitRunReader {
+ constexpr static AllSetBitRun NextRun() { return {}; }
+};
+
+} // namespace internal
+
template <typename T>
-inline int RleDecoder::GetBatchWithDict(const T* dictionary, int32_t dictionary_length,
- T* values, int batch_size) {
- // Per https://github.com/apache/parquet-format/blob/master/Encodings.md,
- // the maximum dictionary index width in Parquet is 32 bits.
- using IndexType = int32_t;
- DictionaryConverter<T> converter;
- converter.dictionary = dictionary;
- converter.dictionary_length = dictionary_length;
-
- ARROW_DCHECK_GE(bit_width_, 0);
- int values_read = 0;
-
- auto* out = values;
-
- while (values_read < batch_size) {
- int remaining = batch_size - values_read;
-
- if (repeat_count_ > 0) {
- auto idx = static_cast<IndexType>(current_value_);
- if (ARROW_PREDICT_FALSE(!IndexInRange(idx, dictionary_length))) {
- return values_read;
- }
- T val = dictionary[idx];
+template <typename V>
+auto RleBitPackedDecoder<T>::GetBatchWithDict(const V* dictionary,
+ int32_t dictionary_length, V* out,
+ values_count_type batch_size)
+ -> values_count_type {
+ using ControlFlow = RleBitPackedParser::ControlFlow;
+
+ if (ARROW_PREDICT_FALSE(batch_size <= 0)) {
+ return 0;
+ }
- int repeat_batch = std::min(remaining, repeat_count_);
- std::fill(out, out + repeat_batch, val);
+ internal::DictionaryConverter<V, value_type> converter{dictionary, dictionary_length};
- /* Upkeep counters */
- repeat_count_ -= repeat_batch;
- values_read += repeat_batch;
- out += repeat_batch;
- } else if (literal_count_ > 0) {
- constexpr int kBufferSize = 1024;
- IndexType indices[kBufferSize];
+ // Make lightweight BitRun class to reuse previous methods.
+ constexpr internal::UnreachableBitRunReader validity_reader{};
+ internal::AllSetBitRun validity_run = {batch_size};
- int literal_batch = std::min(remaining, literal_count_);
- literal_batch = std::min(literal_batch, kBufferSize);
+ values_count_type values_read = 0;
+ auto batch_values_remaining = [&]() {
+ ARROW_DCHECK_LE(values_read, batch_size);
+ return batch_size - values_read;
+ };
- int actual_read = bit_reader_.GetBatch(bit_width_, indices, literal_batch);
- if (ARROW_PREDICT_FALSE(actual_read != literal_batch)) {
- return values_read;
- }
- if (ARROW_PREDICT_FALSE(!converter.IsValid(indices, /*length=*/literal_batch))) {
- return values_read;
- }
- converter.Copy(out, indices, literal_batch);
+ if (ARROW_PREDICT_FALSE(RunRemaining() > 0)) {
Review Comment:
Same question re ARROW_PREDICT_FALSE
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2322029416
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -207,12 +209,310 @@ TEST(BitUtil, RoundTripIntValues) {
}
}
+/// A Rle run is a simple class owning some data and a repetition count.
+/// It does not know how to read such data.
+TEST(Rle, RleRun) {
+ const std::array<RleRun::byte, 4> value = {21, 2, 0, 0};
+
+ RleRun::values_count_type value_count = 12;
+
+ // 12 times the value 21 fitting over 5 bits
+ auto const run_5 = RleRun(value.data(), value_count, /* value_bit_width= */ 5);
+ EXPECT_EQ(run_5.ValuesCount(), value_count);
+ EXPECT_EQ(run_5.ValuesBitWidth(), 5);
+ EXPECT_EQ(run_5.RawDataSize(), 1); // 5 bits fit in one byte
+ EXPECT_EQ(*run_5.RawDataPtr(), 21);
+
+ // 12 times the value 21 fitting over 16 bits
+ auto const run_8 = RleRun(value.data(), value_count, /* value_bit_width= */ 8);
+ EXPECT_EQ(run_8.ValuesCount(), value_count);
+ EXPECT_EQ(run_8.ValuesBitWidth(), 8);
+ EXPECT_EQ(run_8.RawDataSize(), 1); // 8 bits fit in 1 byte
+ EXPECT_EQ(*run_8.RawDataPtr(), 21);
+
+ // 12 times the value {21, 2} fitting over 10 bits
+ auto const run_10 = RleRun(value.data(), value_count, /* value_bit_width= */ 10);
+
+ EXPECT_EQ(run_10.ValuesCount(), value_count);
+ EXPECT_EQ(run_10.ValuesBitWidth(), 10);
+ EXPECT_EQ(run_10.RawDataSize(), 2); // 10 bits fit in 2 bytes
+ EXPECT_EQ(*(run_10.RawDataPtr() + 0), 21);
+ EXPECT_EQ(*(run_10.RawDataPtr() + 1), 2);
+
+ // 12 times the value {21, 2} fitting over 32 bits
+ auto const run_32 = RleRun(value.data(), value_count, /* value_bit_width= */ 32);
+ EXPECT_EQ(run_32.ValuesCount(), value_count);
+ EXPECT_EQ(run_32.ValuesBitWidth(), 32);
+ EXPECT_EQ(run_32.RawDataSize(), 4); // 32 bits fit in 4 bytes
+ EXPECT_EQ(*(run_32.RawDataPtr() + 0), 21);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 1), 2);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 2), 0);
+ EXPECT_EQ(*(run_32.RawDataPtr() + 3), 0);
+}
+
+/// A BitPacked run is a simple class owning some data and its size.
+/// It does not know how to read such data.
+TEST(BitPacked, BitPackedRun) {
+ const std::array<BitPackedRun::byte, 4> value = {0b10101010, 0, 0, 0b1111111};
+
+ /// 16 values of 1 bit for a total of 16 bits
+ BitPackedRun::values_count_type value_count_1 = 16;
+ auto const run_1 = BitPackedRun(value.data(), value_count_1, /* value_bit_width= */ 1);
+ EXPECT_EQ(run_1.ValuesCount(), value_count_1);
+ EXPECT_EQ(run_1.ValuesBitWidth(), 1);
+ EXPECT_EQ(run_1.RawDataSize(), 2); // 16 bits fit in 2 bytes
+ for (BitPackedRun::raw_data_size_type i = 0; i < run_1.RawDataSize(); ++i) {
+ EXPECT_EQ(*(run_1.RawDataPtr() + i), value[i]);
Review Comment:
Is there a point in testing this? You could just compare `run_1.RawDataPtr()` with `value.data()` (pointer comparison).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #47294:
URL: https://github.com/apache/arrow/pull/47294#discussion_r2322150615
##########
cpp/src/arrow/util/rle_encoding_test.cc:
##########
@@ -553,47 +853,178 @@ TEST(BitRle, Overflow) {
}
}
-template <typename Type>
-void CheckRoundTripSpaced(const Array& data, int bit_width) {
+/// Check RleBitPacked encoding/decoding round trip.
+///
+/// \tparam kSpaced If set to false, treat Nulls in the input array as regular data.
+/// \tparam kParts The number of parts in which the data will be decoded.
+/// For number greater than one, this ensure that the decoder intermediary state
+/// is valid.
+template <typename Type, bool kSpaced, int32_t kParts>
+void CheckRoundTrip(const Array& data, int bit_width,
+ std::shared_ptr<FloatArray> dict = {}) {
using ArrayType = typename TypeTraits<Type>::ArrayType;
- using T = typename Type::c_type;
+ using value_type = typename Type::c_type;
- int num_values = static_cast<int>(data.length());
- int buffer_size = RleEncoder::MaxBufferSize(bit_width, num_values);
+ int const data_size = static_cast<int>(data.length());
+ int const data_values_count =
+ static_cast<int>(data.length() - kSpaced * data.null_count());
+ int const buffer_size = RleBitPackedEncoder::MaxBufferSize(bit_width, data_size);
+ ASSERT_GE(kParts, 1);
+ ASSERT_LE(kParts, data_size);
- const T* values = static_cast<const ArrayType&>(data).raw_values();
+ const value_type* data_values = static_cast<const ArrayType&>(data).raw_values();
+ // Encode the data into ``buffer`` using the encoder.
std::vector<uint8_t> buffer(buffer_size);
- RleEncoder encoder(buffer.data(), buffer_size, bit_width);
- for (int i = 0; i < num_values; ++i) {
- if (data.IsValid(i)) {
- if (!encoder.Put(static_cast<uint64_t>(values[i]))) {
- FAIL() << "Encoding failed";
- }
+ RleBitPackedEncoder encoder(buffer.data(), buffer_size, bit_width);
+ int32_t encoded_values_size = 0;
+ for (int i = 0; i < data_size; ++i) {
+ // Depending on kSpaced we treat nulls as regular values.
+ if (data.IsValid(i) || !kSpaced) {
+ bool success = encoder.Put(static_cast<uint64_t>(data_values[i]));
+ ASSERT_TRUE(success) << "Encoding failed in pos " << i;
+ ++encoded_values_size;
}
}
- int encoded_size = encoder.Flush();
+ int encoded_byte_size = encoder.Flush();
+ ASSERT_EQ(encoded_values_size, data_values_count)
+ << "All values input were not encoded successfully by the encoder";
+
+ // On to verify batch read
+ RleBitPackedDecoder<value_type> decoder(buffer.data(), encoded_byte_size, bit_width);
+ // We will only use one of them depending on whether this is a dictonnary tests
+ std::vector<float> dict_read;
+ std::vector<value_type> values_read;
+ if (dict) {
+ dict_read.resize(data_size);
+ } else {
+ values_read.resize(data_size);
+ }
- // Verify batch read
- RleDecoder decoder(buffer.data(), encoded_size, bit_width);
- std::vector<T> values_read(num_values);
+ // We will read the data in kParts calls to make sure intermediate states are valid
+ int32_t actual_read_count = 0;
+ int32_t requested_read_count = 0;
+ while (requested_read_count < data_size) {
+ auto const remaining = data_size - requested_read_count;
+ auto to_read = data_size / kParts;
+ if (remaining / to_read == 1) {
+ to_read = remaining;
+ }
- if (num_values != decoder.GetBatchSpaced(
- num_values, static_cast<int>(data.null_count()),
- data.null_bitmap_data(), data.offset(), values_read.data())) {
- FAIL();
- }
+ auto read = 0;
+ if constexpr (kSpaced) {
+ // We need to slice the input array get the proper null count and bitmap
+ auto data_remaining = data.Slice(requested_read_count, to_read);
+
+ if (dict) {
+ auto* out = dict_read.data() + requested_read_count;
+ read = decoder.GetBatchWithDictSpaced(
+ dict->raw_values(), static_cast<int32_t>(dict->length()), out, to_read,
+ static_cast<int32_t>(data_remaining->null_count()),
+ data_remaining->null_bitmap_data(), data_remaining->offset());
+ } else {
+ auto* out = values_read.data() + requested_read_count;
+ read = decoder.GetBatchSpaced(
+ to_read, static_cast<int32_t>(data_remaining->null_count()),
+ data_remaining->null_bitmap_data(), data_remaining->offset(), out);
+ }
+ } else {
+ if (dict) {
+ auto* out = dict_read.data() + requested_read_count;
+ read = decoder.GetBatchWithDict(
+ dict->raw_values(), static_cast<int32_t>(dict->length()), out, to_read);
+ } else {
+ auto* out = values_read.data() + requested_read_count;
+ read = decoder.GetBatch(out, to_read);
+ }
+ }
+ ASSERT_EQ(read, to_read) << "Decoder did not read as many values as requested";
- for (int64_t i = 0; i < num_values; ++i) {
- if (data.IsValid(i)) {
- if (values_read[i] != values[i]) {
- FAIL() << "Index " << i << " read " << values_read[i] << " but should be "
- << values[i];
+ actual_read_count += read;
+ requested_read_count += to_read;
+ }
+ EXPECT_EQ(requested_read_count, data_size) << "This test logic is wrong";
+ EXPECT_EQ(actual_read_count, data_size) << "Total number of values read is off";
+
+ // Verify the round trip: encoded-decoded values must equal the original one
+ for (int64_t i = 0; i < data_size; ++i) {
+ if (data.IsValid(i) || !kSpaced) {
+ if (dict) {
+ EXPECT_EQ(dict_read.at(i), dict->Value(data_values[i]))
+ << "Encoded then decoded and mapped value at position " << i << " ("
+ << values_read[i] << ") differs from original value (" << data_values[i]
+ << " mapped to " << dict->Value(data_values[i]) << ")";
+ } else {
+ EXPECT_EQ(values_read.at(i), data_values[i])
+ << "Encoded then decoded value at position " << i << " (" << values_read.at(i)
+ << ") differs from original value (" << data_values[i] << ")";
}
}
}
}
+template <typename T>
+struct DataTestRleBitPackedRandomPart {
+ using value_type = T;
+
+ value_type max;
+ int32_t size;
+ double null_probability;
+};
+
+template <typename T>
+struct DataTestRleBitPackedRepeatPart {
+ using value_type = T;
+
+ value_type value;
+ int32_t size;
+};
+
+template <typename T>
+struct DataTestRleBitPackedNullPart {
+ using value_type = T;
+
+ int32_t size;
+};
+
+template <typename T>
+struct DataTestRleBitPacked {
+ using value_type = T;
+ using ArrowType = typename arrow::CTypeTraits<value_type>::ArrowType;
+ using RandomPart = DataTestRleBitPackedRandomPart<value_type>;
+ using RepeatPart = DataTestRleBitPackedRepeatPart<value_type>;
+ using NullPart = DataTestRleBitPackedNullPart<value_type>;
+
+ std::vector<std::variant<RandomPart, RepeatPart, NullPart>> parts;
Review Comment:
Can we define a type alias for this `std::variant`? This will make things more readable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "AntoinePrv (via GitHub)" <gi...@apache.org>.
AntoinePrv commented on PR #47294:
URL: https://github.com/apache/arrow/pull/47294#issuecomment-4047835576
> The quick fix is to explicitly set `UnpackOptions::max_read_bytes` to not use its default value -1.
This should not be the case, where do you see it being left to `-1`? Or do you mean in the `unpack` benchmark themselves?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-47112: [Parquet][C++] Rle BitPacked parser [arrow]
Posted by "HuaHuaY (via GitHub)" <gi...@apache.org>.
HuaHuaY commented on PR #47294:
URL: https://github.com/apache/arrow/pull/47294#issuecomment-4004037226
We found that the less the number of distinct values, the more the performance regression.
`parquet-scan-main` is compiled from the latest commit of the main branch.
`parquet-scan-old` is compiled from `64f2055ffb68e5077420f4253e76d78952438cab` which is the previous commit of this PR on the main branch.
```sh
❯ env LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$LLVM_ROOT/lib/x86_64-unknown-linux-gnu ARROW_RUNTIME_SIMD_LEVEL=AVX2 hyperfine -w 5 -r 20 --sort mean-time "cpp/out/build/ninja-release/release/parquet-scan-main --columns=6 /dev/shm/TableSink0" "cpp/out/build/ninja-release/release/parquet-scan-old --columns=6 /dev/shm/TableSink0"
Benchmark 1: cpp/out/build/ninja-release/release/parquet-scan-main --columns=6 /dev/shm/TableSink0
Time (mean ± σ): 31.1 ms ± 0.3 ms [User: 27.2 ms, System: 3.7 ms]
Range (min … max): 30.7 ms … 31.8 ms 20 runs
Benchmark 2: cpp/out/build/ninja-release/release/parquet-scan-old --columns=6 /dev/shm/TableSink0
Time (mean ± σ): 22.8 ms ± 0.4 ms [User: 19.3 ms, System: 3.3 ms]
Range (min … max): 22.2 ms … 23.3 ms 20 runs
Summary
cpp/out/build/ninja-release/release/parquet-scan-old --columns=6 /dev/shm/TableSink0 ran
1.37 ± 0.02 times faster than cpp/out/build/ninja-release/release/parquet-scan-main --columns=6 /dev/shm/TableSink0
❯ env LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$LLVM_ROOT/lib/x86_64-unknown-linux-gnu cpp/out/build/ninja-release/release/parquet-reader --only-metadata /dev/shm/TableSink0
File Name: /dev/shm/TableSink0
Version: 2.6
Created By: cz-cpp version BuildInfo:GitBranch:release/20240820_rc8,GitVersion:73a4383,BuildTime:1725298762,CloudEnv:ALIYUN
Total rows: 6250733
Number of RowGroups: 1
Number of Real Columns: 40
Number of Columns: 40
Number of Selected Columns: 40
......
Column 6: lo_orderpriority (BYTE_ARRAY / String / UTF8)
......
--- Row Group: 0 ---
--- Total Bytes: 1016215218 ---
--- Total Compressed Bytes: 552911018 ---
--- Sort Columns:
column_idx: 5, descending: 0, nulls_first: 1
column_idx: 0, descending: 0, nulls_first: 1
--- Rows: 6250733 ---
......
Column 6
Values: 6250733, Null Values: 0, Distinct Values: 5
Max (exact: unknown): 5-LOW, Min (exact: unknown): 1-URGENT
Compression: LZ4_RAW, Encodings: PLAIN(DICT_PAGE) RLE_DICTIONARY
Uncompressed Size: 2267694, Compressed Size: 2092132
......
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org