You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/07/16 12:26:42 UTC

[GitHub] [arrow-rs] thinkharderdev opened a new pull request, #2089: Issue 2078

thinkharderdev opened a new pull request, #2089:
URL: https://github.com/apache/arrow-rs/pull/2089

   # Which issue does this PR close?
   
   <!---
   We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes #123` indicates that this PR will close issue #123.
   -->
   
   Closes #2078 
   
   # Rationale for this change
    
    <!---
    Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed.
    Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes.
   -->
   
   # What changes are included in this PR?
   
   <!---
   There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR.
   -->
   
   # Are there any user-facing changes?
   
   
   <!---
   If there are user-facing changes then we may require documentation to be updated before approving the PR.
   -->
   
   <!---
   If there are any breaking changes to public APIs, please add the `breaking change` label.
   -->
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-rs] thinkharderdev commented on a diff in pull request #2089: Issue 2078

Posted by GitBox <gi...@apache.org>.
thinkharderdev commented on code in PR #2089:
URL: https://github.com/apache/arrow-rs/pull/2089#discussion_r922675052


##########
parquet/src/encodings/decoding.rs:
##########
@@ -681,13 +707,20 @@ where
         Ok(to_read)
     }
 
+
+
     fn values_left(&self) -> usize {
         self.values_left
     }
 
     fn encoding(&self) -> Encoding {
         Encoding::DELTA_BINARY_PACKED
     }
+
+    fn skip(&mut self, num_values: usize) -> Result<usize> {
+        let mut buffer = vec![T::T::default(); num_values];

Review Comment:
   This seems to kind of defeat the purpose but not sure how to skip delta encoded values without it....



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-rs] thinkharderdev commented on a diff in pull request #2089: Issue 2078

Posted by GitBox <gi...@apache.org>.
thinkharderdev commented on code in PR #2089:
URL: https://github.com/apache/arrow-rs/pull/2089#discussion_r922675061


##########
parquet/src/encodings/decoding.rs:
##########
@@ -922,6 +974,11 @@ impl<T: DataType> Decoder<T> for DeltaByteArrayDecoder<T> {
     fn encoding(&self) -> Encoding {
         Encoding::DELTA_BYTE_ARRAY
     }
+
+    fn skip(&mut self, num_values: usize) -> Result<usize> {
+        let mut buffer = vec![T::T::default(); num_values];

Review Comment:
   Same here



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-rs] codecov-commenter commented on pull request #2089: Support skip_values in ColumnValueDecoderImpl

Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #2089:
URL: https://github.com/apache/arrow-rs/pull/2089#issuecomment-1186181858

   # [Codecov](https://codecov.io/gh/apache/arrow-rs/pull/2089?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#2089](https://codecov.io/gh/apache/arrow-rs/pull/2089?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (154982e) into [master](https://codecov.io/gh/apache/arrow-rs/commit/86543a4b835f898308bd28a33d0da9d2ea7a9b35?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (86543a4) will **increase** coverage by `0.10%`.
   > The diff coverage is `86.04%`.
   
   ```diff
   @@            Coverage Diff             @@
   ##           master    #2089      +/-   ##
   ==========================================
   + Coverage   83.61%   83.72%   +0.10%     
   ==========================================
     Files         223      224       +1     
     Lines       58539    59350     +811     
   ==========================================
   + Hits        48947    49688     +741     
   - Misses       9592     9662      +70     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/arrow-rs/pull/2089?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [arrow-flight/src/arrow.flight.protocol.rs](https://codecov.io/gh/apache/arrow-rs/pull/2089/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YXJyb3ctZmxpZ2h0L3NyYy9hcnJvdy5mbGlnaHQucHJvdG9jb2wucnM=) | `0.00% <ø> (ø)` | |
   | [parquet/src/column/reader/decoder.rs](https://codecov.io/gh/apache/arrow-rs/pull/2089/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGFycXVldC9zcmMvY29sdW1uL3JlYWRlci9kZWNvZGVyLnJz) | `63.38% <0.00%> (-5.86%)` | :arrow_down: |
   | [parquet/src/data\_type.rs](https://codecov.io/gh/apache/arrow-rs/pull/2089/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGFycXVldC9zcmMvZGF0YV90eXBlLnJz) | `75.79% <84.31%> (+1.03%)` | :arrow_up: |
   | [parquet/src/encodings/decoding.rs](https://codecov.io/gh/apache/arrow-rs/pull/2089/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGFycXVldC9zcmMvZW5jb2RpbmdzL2RlY29kaW5nLnJz) | `88.70% <85.79%> (-0.84%)` | :arrow_down: |
   | [parquet/src/util/bit\_util.rs](https://codecov.io/gh/apache/arrow-rs/pull/2089/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGFycXVldC9zcmMvdXRpbC9iaXRfdXRpbC5ycw==) | `93.26% <89.47%> (-0.55%)` | :arrow_down: |
   | [parquet/src/encodings/rle.rs](https://codecov.io/gh/apache/arrow-rs/pull/2089/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGFycXVldC9zcmMvZW5jb2RpbmdzL3JsZS5ycw==) | `92.59% <92.77%> (+0.03%)` | :arrow_up: |
   | [arrow/src/array/builder/primitive\_builder.rs](https://codecov.io/gh/apache/arrow-rs/pull/2089/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YXJyb3cvc3JjL2FycmF5L2J1aWxkZXIvcHJpbWl0aXZlX2J1aWxkZXIucnM=) | `92.57% <0.00%> (-0.55%)` | :arrow_down: |
   | [arrow/src/array/array\_decimal.rs](https://codecov.io/gh/apache/arrow-rs/pull/2089/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YXJyb3cvc3JjL2FycmF5L2FycmF5X2RlY2ltYWwucnM=) | `90.59% <0.00%> (-0.53%)` | :arrow_down: |
   | [parquet\_derive/src/parquet\_field.rs](https://codecov.io/gh/apache/arrow-rs/pull/2089/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGFycXVldF9kZXJpdmUvc3JjL3BhcnF1ZXRfZmllbGQucnM=) | `65.75% <0.00%> (-0.23%)` | :arrow_down: |
   | [parquet/src/encodings/encoding.rs](https://codecov.io/gh/apache/arrow-rs/pull/2089/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGFycXVldC9zcmMvZW5jb2RpbmdzL2VuY29kaW5nLnJz) | `93.43% <0.00%> (-0.20%)` | :arrow_down: |
   | ... and [10 more](https://codecov.io/gh/apache/arrow-rs/pull/2089/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow-rs/pull/2089?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/arrow-rs/pull/2089?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [86543a4...154982e](https://codecov.io/gh/apache/arrow-rs/pull/2089?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-rs] Ted-Jiang commented on a diff in pull request #2089: Support skip_values in ColumnValueDecoderImpl

Posted by GitBox <gi...@apache.org>.
Ted-Jiang commented on code in PR #2089:
URL: https://github.com/apache/arrow-rs/pull/2089#discussion_r922688898


##########
parquet/src/encodings/decoding.rs:
##########
@@ -791,6 +824,25 @@ impl<T: DataType> Decoder<T> for DeltaLengthByteArrayDecoder<T> {
     fn encoding(&self) -> Encoding {
         Encoding::DELTA_LENGTH_BYTE_ARRAY
     }
+
+    fn skip(&mut self, num_values: usize) -> Result<usize> {
+        match T::get_physical_type() {
+            Type::BYTE_ARRAY => {
+                let num_values = cmp::min(num_values, self.num_values);
+
+                let next_offset: i32 =  self.lengths[self.current_idx..self.current_idx + num_values].iter().sum();
+
+                self.current_idx += num_values;
+                self.offset += next_offset as usize;
+
+                self.num_values -= num_values;
+                Ok(num_values)
+            }
+            _ => Err(general_err!(
+                "DeltaLengthByteArrayDecoder only support ByteArrayType"
+            )),

Review Comment:
   ```suggestion
              other_type => Err(general_err!(
                   "DeltaLengthByteArrayDecoder not support {}, only support byte array", other_type
               )),
   ```
   I think print the type is easy for debugging



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-rs] ursabot commented on pull request #2089: Support skip_values in ColumnValueDecoderImpl

Posted by GitBox <gi...@apache.org>.
ursabot commented on PR #2089:
URL: https://github.com/apache/arrow-rs/pull/2089#issuecomment-1186295821

   Benchmark runs are scheduled for baseline = 13adaa7c5a6179267124e79a79b1cc7b8976d9a5 and contender = c58554453bec8f4a78d503e79576b85108b210b5. c58554453bec8f4a78d503e79576b85108b210b5 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
   Conbench compare runs links:
   [Skipped :warning: Benchmarking of arrow-rs-commits is not supported on ec2-t3-xlarge-us-east-2] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/ba81095fcc6e4823a4b745343401fd7b...a1e47a523d3d4e648cb5ed06f2809d07/)
   [Skipped :warning: Benchmarking of arrow-rs-commits is not supported on test-mac-arm] [test-mac-arm](https://conbench.ursa.dev/compare/runs/e1ba31e189c8435bab70343475f588c5...62faad76066c43948eb253691b94fde3/)
   [Skipped :warning: Benchmarking of arrow-rs-commits is not supported on ursa-i9-9960x] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/a0867118a84e4592895602aa9ec4cdc4...80e45cdc76e446acb0ad5c6adc8b228d/)
   [Skipped :warning: Benchmarking of arrow-rs-commits is not supported on ursa-thinkcentre-m75q] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/af60a705f8e34582a2ebf71e3e3d4f31...ced12e5c621042058c3c7f3e76da96cb/)
   Buildkite builds:
   Supported benchmarks:
   ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
   test-mac-arm: Supported benchmark langs: C++, Python, R
   ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
   ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-rs] tustvold commented on a diff in pull request #2089: Support skip_values in ColumnValueDecoderImpl

Posted by GitBox <gi...@apache.org>.
tustvold commented on code in PR #2089:
URL: https://github.com/apache/arrow-rs/pull/2089#discussion_r922677184


##########
parquet/src/data_type.rs:
##########
@@ -936,6 +982,24 @@ pub(crate) mod private {
             Ok(num_values)
         }
 
+        fn skip(decoder: &mut PlainDecoderDetails, num_values: usize) -> Result<usize> {
+            let data = decoder
+                .data
+                .as_mut()
+                .expect("set_data should have been called");
+            let num_values = std::cmp::min(num_values, decoder.num_values);

Review Comment:
   ```suggestion
               let num_values = num_values.min(decoder.num_values);
   ```
   



##########
parquet/src/data_type.rs:
##########
@@ -690,6 +692,15 @@ pub(crate) mod private {
             Ok(values_read)
         }
 
+        fn skip(decoder: &mut PlainDecoderDetails, num_values: usize) -> Result<usize> {
+            let bit_reader = decoder.bit_reader.as_mut().unwrap();
+            let num_values = std::cmp::min(num_values, decoder.num_values);
+            let mut buffer = vec![false; num_values];
+            let values_read = bit_reader.get_batch(&mut buffer[..num_values], 1);

Review Comment:
   BitReader::skip?



##########
arrow/Cargo.toml:
##########
@@ -41,7 +41,7 @@ bench = false
 ahash = { version = "0.7", default-features = false }
 serde = { version = "1.0", default-features = false }
 serde_derive = { version = "1.0", default-features = false }
-serde_json = { version = "1.0", default-features = false, features = ["preserve_order"] }
+serde_json = { version = "1.0", default-features = false, features = ["preserve_order","std"] }

Review Comment:
   This shouldn't be necessary if you update your lockfile



##########
arrow-flight/src/arrow.flight.protocol.rs:
##########
@@ -279,9 +279,9 @@ pub mod flight_service_client {
             &mut self,
             request: impl tonic::IntoStreamingRequest<Message = super::HandshakeRequest>,
         ) -> Result<
-            tonic::Response<tonic::codec::Streaming<super::HandshakeResponse>>,
-            tonic::Status,
-        > {
+                tonic::Response<tonic::codec::Streaming<super::HandshakeResponse>>,

Review Comment:
   I wonder if you have an out of date lockfile, which is resulting in these changes??



##########
parquet/src/data_type.rs:
##########
@@ -764,6 +775,23 @@ pub(crate) mod private {
                     Ok(num_values)
                 }
 
+                #[inline]
+                fn skip(decoder: &mut PlainDecoderDetails, num_values: usize) -> Result<usize> {
+                    let data = decoder.data.as_ref().expect("set_data should have been called");
+                    let num_values = std::cmp::min(num_values, decoder.num_values);

Review Comment:
   ```suggestion
                       let num_values = num_values.min(decoder.num_values);
   ```
   



##########
parquet/src/encodings/rle.rs:
##########
@@ -631,6 +723,45 @@ mod tests {
         assert_eq!(buffer, expected);
     }
 
+    #[test]
+    fn test_rle_skip_dict() {
+        // Test RLE encoding: 3 0s followed by 4 1s followed by 5 2s
+        // 00000110 00000000 00001000 00000001 00001010 00000010
+        let dict = vec![10, 20, 30];
+        let data = ByteBufferPtr::new(vec![0x06, 0x00, 0x08, 0x01, 0x0A, 0x02]);
+        let mut decoder: RleDecoder = RleDecoder::new(3);
+        decoder.set_data(data);
+        let mut buffer = vec![0; 10];
+        let expected = vec![10, 20, 20, 20, 20, 30, 30, 30, 30, 30];
+        let skipped = decoder.skip(2).expect("skipping two values");
+        assert_eq!(skipped, 2);
+        let remainder = decoder.get_batch_with_dict::<i32>(&dict, &mut buffer, 10).expect("getting remainder");
+        assert_eq!(remainder, 10);
+        assert_eq!(buffer, expected);
+
+        // Test bit-pack encoding: 345345345455 (2 groups: 8 and 4)
+        // 011 100 101 011 100 101 011 100 101 100 101 101
+        // 00000011 01100011 11000111 10001110 00000011 01100101 00001011
+        let dict = vec!["aaa", "bbb", "ccc", "ddd", "eee", "fff"];
+        let data = ByteBufferPtr::new(vec![0x03, 0x63, 0xC7, 0x8E, 0x03, 0x65, 0x0B]);
+        let mut decoder: RleDecoder = RleDecoder::new(3);
+        decoder.set_data(data);
+        let mut buffer = vec![""; 8];
+        let expected = vec![
+            "eee", "fff", "ddd", "eee", "fff", "eee", "fff",
+            "fff",
+        ];
+        let skipped = decoder.skip(4).expect("skipping two values");

Review Comment:
   ```suggestion
           let skipped = decoder.skip(4).expect("skipping four values");
   ```
   



##########
parquet/src/util/bit_util.rs:
##########
@@ -605,6 +627,56 @@ impl BitReader {
         values_to_read
     }
 
+    /// Skip num_value values with num_bits bit width
+    ///
+    /// Return the number of values skipped (up to num_values)
+    pub fn skip(&mut self, num_values: usize, num_bits: usize) -> usize {
+        assert!(num_bits <= 64);
+
+        let mut num_values = num_values;
+        let needed_bits = num_bits * num_values;
+        let remaining_bits = (self.total_bytes - self.byte_offset) * 8 - self.bit_offset;
+        if remaining_bits < needed_bits {
+            num_values = remaining_bits / num_bits;
+        }
+
+        let mut values_skipped = 0;
+
+        if num_bits > 32 {

Review Comment:
   We don't need this special case for skip, this exists in get_batch because we only have unpack32 and not unpack64



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-rs] thinkharderdev commented on a diff in pull request #2089: Support skip_values in ColumnValueDecoderImpl

Posted by GitBox <gi...@apache.org>.
thinkharderdev commented on code in PR #2089:
URL: https://github.com/apache/arrow-rs/pull/2089#discussion_r922681539


##########
arrow-flight/src/arrow.flight.protocol.rs:
##########
@@ -279,9 +279,9 @@ pub mod flight_service_client {
             &mut self,
             request: impl tonic::IntoStreamingRequest<Message = super::HandshakeRequest>,
         ) -> Result<
-            tonic::Response<tonic::codec::Streaming<super::HandshakeResponse>>,
-            tonic::Status,
-        > {
+                tonic::Response<tonic::codec::Streaming<super::HandshakeResponse>>,

Review Comment:
   Yeah, that was it. Thanks!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-rs] Ted-Jiang commented on pull request #2089: Support skip_values in ColumnValueDecoderImpl

Posted by GitBox <gi...@apache.org>.
Ted-Jiang commented on PR #2089:
URL: https://github.com/apache/arrow-rs/pull/2089#issuecomment-1186214406

   Nice job! 🥳 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-rs] tustvold merged pull request #2089: Support skip_values in ColumnValueDecoderImpl

Posted by GitBox <gi...@apache.org>.
tustvold merged PR #2089:
URL: https://github.com/apache/arrow-rs/pull/2089


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-rs] Ted-Jiang commented on a diff in pull request #2089: Support skip_values in ColumnValueDecoderImpl

Posted by GitBox <gi...@apache.org>.
Ted-Jiang commented on code in PR #2089:
URL: https://github.com/apache/arrow-rs/pull/2089#discussion_r922688118


##########
parquet/src/encodings/decoding.rs:
##########
@@ -291,6 +294,10 @@ impl<T: DataType> Decoder<T> for PlainDecoder<T> {
     fn get(&mut self, buffer: &mut [T::T]) -> Result<usize> {
         T::T::decode(buffer, &mut self.inner)
     }
+

Review Comment:
   Should we set `#[inline]` get better perfomance ?🤔



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-rs] Ted-Jiang commented on a diff in pull request #2089: Support skip_values in ColumnValueDecoderImpl

Posted by GitBox <gi...@apache.org>.
Ted-Jiang commented on code in PR #2089:
URL: https://github.com/apache/arrow-rs/pull/2089#discussion_r922688208


##########
parquet/src/encodings/decoding.rs:
##########
@@ -291,6 +294,10 @@ impl<T: DataType> Decoder<T> for PlainDecoder<T> {
     fn get(&mut self, buffer: &mut [T::T]) -> Result<usize> {
         T::T::decode(buffer, &mut self.inner)
     }
+

Review Comment:
   because i fond `encoding`  `set_data` all with inline



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-rs] Ted-Jiang commented on a diff in pull request #2089: Support skip_values in ColumnValueDecoderImpl

Posted by GitBox <gi...@apache.org>.
Ted-Jiang commented on code in PR #2089:
URL: https://github.com/apache/arrow-rs/pull/2089#discussion_r922687835


##########
parquet/src/encodings/decoding.rs:
##########
@@ -355,6 +362,8 @@ impl<T: DataType> Decoder<T> for DictDecoder<T> {
         rle.get_batch_with_dict(&self.dictionary[..], buffer, num_values)
     }
 
+

Review Comment:
   nit: useless space.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-rs] thinkharderdev commented on pull request #2089: Issue 2078

Posted by GitBox <gi...@apache.org>.
thinkharderdev commented on PR #2089:
URL: https://github.com/apache/arrow-rs/pull/2089#issuecomment-1186172566

   @Ted-Jiang 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org