You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/06/10 08:55:19 UTC
[GitHub] [arrow-rs] Ted-Jiang opened a new issue, #1834: Sperate get_next_page_header from get_next_page in PageReader
Ted-Jiang opened a new issue, #1834:
URL: https://github.com/apache/arrow-rs/issues/1834
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
```
pub trait PageReader: Iterator<Item = Result<Page>> + Send {
/// Gets the next page in the column chunk associated with this reader.
/// Returns `None` if there are no pages left.
fn get_next_page(&mut self) -> Result<Option<Page>>;
}
```
change to
```
pub trait PageReader: Iterator<Item = Result<Page>> + Send {
/// Gets the next page in the column chunk associated with this reader.
/// Returns `None` if there are no pages left.
fn get_next_page(&mut self) -> Result<Option<Page>>;
/// Gets the next page in the column chunk associated with this reader.
/// Returns `None` if there are no pages left.
fn get_next_page_header(&mut self) -> Result<Option<PageHeader>>;
}
```
i want to add `skip_page` in PageReader (for page index) need check page header before call `skip_page` (
can not skip `DictionaryPage` `IndexPage`),
but now read header is in `get_next_page`, so extract it to a new func.
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
put `skip_page` in `get_next_page`
**Additional context**
Add any other context or screenshots about the feature request here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] Ted-Jiang commented on issue #1834: Sperate get_next_page_header from get_next_page in PageReader
Posted by GitBox <gi...@apache.org>.
Ted-Jiang commented on issue #1834:
URL: https://github.com/apache/arrow-rs/issues/1834#issuecomment-1168260387
@tustvold Thanks a lot, i miss the dictionary pate`MUST be the first one in the column chunk` part.
Last two weeks, i busy about my personal things. I will go back and work on page index!
Thanks again for your info ❤️
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] tustvold commented on issue #1834: Sperate get_next_page_header from get_next_page in PageReader
Posted by GitBox <gi...@apache.org>.
tustvold commented on issue #1834:
URL: https://github.com/apache/arrow-rs/issues/1834#issuecomment-1152140590
I could be missing something, but I was under the impression the intention of the ColumnIndex, as opposed to using page-level statistics, was precisely to avoid needing to read any page data for those being skipped? Is this information not encoded in the `PageLocation` index or something?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] Ted-Jiang closed issue #1834: Sperate get_next_page_header from get_next_page in PageReader
Posted by GitBox <gi...@apache.org>.
Ted-Jiang closed issue #1834: Sperate get_next_page_header from get_next_page in PageReader
URL: https://github.com/apache/arrow-rs/issues/1834
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] Ted-Jiang commented on issue #1834: Sperate get_next_page_header from get_next_page in PageReader
Posted by GitBox <gi...@apache.org>.
Ted-Jiang commented on issue #1834:
URL: https://github.com/apache/arrow-rs/issues/1834#issuecomment-1152136096
@tustvold PTAL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org