[GitHub] [arrow-rs] Ted-Jiang commented on a diff in pull request #2111: Support skip_def_levels (only max_def_levels=1) for ColumnLevelDecoder

[GitBox <gi...@apache.org>]

Ted-Jiang commented on code in PR #2111:
URL: https://github.com/apache/arrow-rs/pull/2111#discussion_r925716839


##########
parquet/src/arrow/record_reader/definition_levels.rs:
##########
@@ -399,6 +442,55 @@ mod tests {
         assert_eq!(decoded.as_slice(), expected.as_slice());
     }
 
+    #[test]
+    fn test_packed_decoder_skip() {
+        let mut rng = thread_rng();
+        let len: usize = rng.gen_range(512..1024) * 8;
+
+        let mut expected = BooleanBufferBuilder::new(len);
+        let mut encoder = RleEncoder::new(1, 1024 * 8);
+
+        for _ in 0..len {
+            let bool = rng.gen_bool(0.8);
+            assert!(encoder.put(bool as u64).unwrap());
+            expected.append(bool);
+        }
+        assert_eq!(expected.len(), len);
+
+        let encoded = encoder.consume().unwrap();
+        let mut decoder = PackedDecoder::new(Encoding::RLE, ByteBufferPtr::new(encoded));
+
+        let mut skip_value = 0;
+        let mut read_value = 0;
+        let mut read_data = vec![];
+
+        loop {
+            let remaining = len - read_value - skip_value;
+            if remaining == 0 {
+                break;
+            }
+            let to_read = (rng.gen_range(1..=remaining)) / 8 * 8;

Review Comment:
   a little hack here,  because i use `u8` to compare equal. But the decoder is one bit. So i think if want to compare bit equal in `u8`, should read or skip in 8 bit 🤣  



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org