You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/09/08 19:04:30 UTC
[GitHub] [arrow] marsupialtail opened a new pull request, #14078: ARROW-17313: [C++][Python] Add fragment.slice() for CSV file format
marsupialtail opened a new pull request, #14078:
URL: https://github.com/apache/arrow/pull/14078
Carry over from: https://github.com/apache/arrow/pull/13830.
I rebased on the new 10.0, and really messed up my git environment. I think it's just easier for me to make a new pull request. This is unfortunate and not best practice, but I hope will not happen again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] marsupialtail commented on pull request #14078: ARROW-17313: [C++][Python] Add fragment.slice() for CSV file format
Posted by GitBox <gi...@apache.org>.
marsupialtail commented on PR #14078:
URL: https://github.com/apache/arrow/pull/14078#issuecomment-1242032677
Looks like https://github.com/apache/arrow/pull/13931/ will be merged before this one at this rate. I am going to have to do another rebase
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] marsupialtail commented on pull request #14078: ARROW-17313: [C++][Python] Add fragment.slice() for CSV file format
Posted by GitBox <gi...@apache.org>.
marsupialtail commented on PR #14078:
URL: https://github.com/apache/arrow/pull/14078#issuecomment-1241120506
@westonpace @pitrou @zhztheplayer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #14078: ARROW-17313: [C++][Python] Add fragment.slice() for CSV file format
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #14078:
URL: https://github.com/apache/arrow/pull/14078#issuecomment-1241119342
https://issues.apache.org/jira/browse/ARROW-17313
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] amol- closed pull request #14078: ARROW-17313: [C++][Python] Add fragment.slice() for CSV file format
Posted by "amol- (via GitHub)" <gi...@apache.org>.
amol- closed pull request #14078: ARROW-17313: [C++][Python] Add fragment.slice() for CSV file format
URL: https://github.com/apache/arrow/pull/14078
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] amol- commented on pull request #14078: ARROW-17313: [C++][Python] Add fragment.slice() for CSV file format
Posted by "amol- (via GitHub)" <gi...@apache.org>.
amol- commented on PR #14078:
URL: https://github.com/apache/arrow/pull/14078#issuecomment-1490649138
Closing because it has been untouched for a while, in case it's still relevant feel free to reopen and move it forward 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] marsupialtail commented on pull request #14078: ARROW-17313: [C++][Python] Add fragment.slice() for CSV file format
Posted by GitBox <gi...@apache.org>.
marsupialtail commented on PR #14078:
URL: https://github.com/apache/arrow/pull/14078#issuecomment-1241119871
Recap the discussion: we want to add slice to the FileFragment class so that we can open a Reader for just a partial byte range. Implemented a Slice method for FileFragment that makes a new FileFragment with a specified byte range.
Currently this assumes that the byte ranges supplied respect line breaks for the CSV file format. If the byte range starts/ends in the middle of a line, then an error will be thrown when the Reader parses the start / end block. As a result of this this PR doesn't support Parquet since you should just use the subset API already available to get this functionality.
In the future, we want to incorporate @zhztheplayer 's work to allow slicing in the middle of a row group for Parquet and possibly slicing in the middle of a linebreak for CSV.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org