You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "rambleraptor (via GitHub)" <gi...@apache.org> on 2025/11/20 02:43:03 UTC

[PR] Check level byte length to avoid panic [arrow-rs]

rambleraptor opened a new pull request, #8889:
URL: https://github.com/apache/arrow-rs/pull/8889

   # fix: Check level byte length to avoid panic
   
   We've found a panic in our testing in these certain areas. I've added tests to help show when the errors would be thrown.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Check level byte length to avoid panic [arrow-rs]

Posted by "etseidl (via GitHub)" <gi...@apache.org>.
etseidl commented on code in PR #8889:
URL: https://github.com/apache/arrow-rs/pull/8889#discussion_r2578973043


##########
parquet/src/column/reader.rs:
##########
@@ -466,9 +472,12 @@ where
                                 self.def_level_decoder
                                     .as_mut()
                                     .unwrap()
-                                    .set_data(def_level_encoding, level_data)?;
+                                    .set_data(def_level_encoding, level_data);
                             }
 
+                            if offset > buf.len() {

Review Comment:
   Looking at it more, I think this error checking needs to move into `parse_v1_levels` which can still panic. If that doesn't return an error, then we don't need to check again here.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Check level byte length to avoid panic [arrow-rs]

Posted by "etseidl (via GitHub)" <gi...@apache.org>.
etseidl commented on code in PR #8889:
URL: https://github.com/apache/arrow-rs/pull/8889#discussion_r2546477854


##########
parquet/src/column/reader.rs:
##########
@@ -437,6 +437,9 @@ where
                             let mut offset = 0;
 
                             if max_rep_level > 0 {
+                                if offset > buf.len() {

Review Comment:
   This one should never evaluate `true`?



##########
parquet/src/column/reader.rs:
##########
@@ -466,9 +472,12 @@ where
                                 self.def_level_decoder
                                     .as_mut()
                                     .unwrap()
-                                    .set_data(def_level_encoding, level_data)?;
+                                    .set_data(def_level_encoding, level_data);
                             }
 
+                            if offset > buf.len() {

Review Comment:
   If the checks above are moved to after `offset` is incremented, then I think this check can be eliminated as it will either always be true (`offset` is still 0), or the error condition was already caught.



##########
parquet/src/column/reader.rs:
##########
@@ -509,27 +518,33 @@ where
                                 self.has_record_delimiter =
                                     self.page_reader.at_record_boundary()?;
 
+                                if rep_levels_byte_len as usize > buf.len() {
+                                    return Err(general_err!("not enough data to read levels"));
+                                }
                                 self.rep_level_decoder.as_mut().unwrap().set_data(
                                     Encoding::RLE,
                                     buf.slice(..rep_levels_byte_len as usize),
-                                )?;
+                                );
                             }
 
+                            let def_levels_end = rep_levels_byte_len + def_levels_byte_len;

Review Comment:
   Why not move this check higher up and check all levels in one shot?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Check level byte length to avoid panic [arrow-rs]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8889:
URL: https://github.com/apache/arrow-rs/pull/8889#issuecomment-4795288605

   Thank you for your contribution. Unfortunately, this pull request is stale because it has been open 60 days with no activity. Please remove the stale label or comment or this will be closed in 7 days.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Check level byte length to avoid panic [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #8889:
URL: https://github.com/apache/arrow-rs/pull/8889#issuecomment-3617747113

   I think my reading of @etseidl 's comments is that this PR needs a little more work, so marking it as draft to make that clear


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Check level byte length to avoid panic [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #8889:
URL: https://github.com/apache/arrow-rs/pull/8889#issuecomment-3575191294

   Hi @rambleraptor , will you have time to address @etseidl 's comments for this PR or shall we plan to do it as a follow on PR?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Check level byte length to avoid panic [arrow-rs]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #8889: Check level byte length to avoid panic
URL: https://github.com/apache/arrow-rs/pull/8889


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org