You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/10/26 20:03:49 UTC

[GitHub] [arrow] alamb opened a new pull request #8534: ARROW-10396: [Rust][Parquet] Publically export SliceableCursor and FileSource (DRAFT)

alamb opened a new pull request #8534:
URL: https://github.com/apache/arrow/pull/8534


   This PR builds on on https://github.com/apache/arrow/pull/8528, so waiting for that one to merge before this one
   
   https://github.com/apache/arrow/commit/7155cd5488310c15d864428252ca71dd9ebd3b48 / https://github.com/apache/arrow/pull/8300 Reworked how the parquet reader traits were implemented to be interms of a `ChunkReader` trait (for the better, in my opinion).
   
   That commit includes two helper classes, `SliceableCursor` and `FileSource`, which implement `ChunkReader` for a `Cursor` like thing and `File`s, respectively
   
   My project uses the parquet `SerializedFileWriter` with things that look like `File` and `Cursor` and thus I would like to re-use the logic in `SliceableCursor` and `FileSource` without having to copy/paste them.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] github-actions[bot] commented on pull request #8534: ARROW-10396: [Rust][Parquet] Publically export SliceableCursor and FileSource (DRAFT)

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8534:
URL: https://github.com/apache/arrow/pull/8534#issuecomment-716793275


   <!--
     Licensed to the Apache Software Foundation (ASF) under one
     or more contributor license agreements.  See the NOTICE file
     distributed with this work for additional information
     regarding copyright ownership.  The ASF licenses this file
     to you under the Apache License, Version 2.0 (the
     "License"); you may not use this file except in compliance
     with the License.  You may obtain a copy of the License at
   
       http://www.apache.org/licenses/LICENSE-2.0
   
     Unless required by applicable law or agreed to in writing,
     software distributed under the License is distributed on an
     "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
     KIND, either express or implied.  See the License for the
     specific language governing permissions and limitations
     under the License.
   -->
   
   Thanks for opening a pull request!
   
   Could you open an issue for this pull request on JIRA?
   https://issues.apache.org/jira/browse/ARROW
   
   Then could you also rename pull request title in the following format?
   
       ARROW-${JIRA_ID}: [${COMPONENT}] ${SUMMARY}
   
   See also:
   
     * [Other pull requests](https://github.com/apache/arrow/pulls/)
     * [Contribution Guidelines - How to contribute patches](https://arrow.apache.org/docs/developers/contributing.html#how-to-contribute-patches)
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] alamb commented on a change in pull request #8534: ARROW-10396: [Rust] [Parquet] Publically export SliceableCursor and FileSource (DRAFT)

Posted by GitBox <gi...@apache.org>.
alamb commented on a change in pull request #8534:
URL: https://github.com/apache/arrow/pull/8534#discussion_r512590881



##########
File path: rust/parquet/src/util/cursor.rs
##########
@@ -15,20 +15,31 @@
 // specific language governing permissions and limitations
 // under the License.
 
-use std::cmp;
-use std::io::{self, Error, ErrorKind, Read};
+use std::io::{self, Error, ErrorKind, Read, Seek, SeekFrom};
 use std::rc::Rc;
+use std::{cmp, fmt};
 
 /// This is object to use if your file is already in memory.
 /// The sliceable cursor is similar to std::io::Cursor, except that it makes it easy to create "cursor slices".
 /// To achieve this, it uses Rc instead of shared references. Indeed reference fields are painfull
 /// because the lack of Generic Associated Type implies that you would require complex lifetime propagation when
 /// returning such a cursor.
 pub struct SliceableCursor {
-    inner: Rc<Vec<u8>>,
     start: u64,
     length: usize,
     pos: u64,
+    inner: Rc<Vec<u8>>,
+}
+
+impl fmt::Debug for SliceableCursor {

Review comment:
       I used this formulation rather than `[#derive(Debug)]` to avoid getting the buffer contents dumped which can be lots of data to sort through. 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] alamb commented on pull request #8534: ARROW-10396: [Rust] [Parquet] Publically export SliceableCursor and FileSource

Posted by GitBox <gi...@apache.org>.
alamb commented on pull request #8534:
URL: https://github.com/apache/arrow/pull/8534#issuecomment-717160182


   fyi @rdettai  / @sunchao 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org