You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@orc.apache.org by "cxzl25 (via GitHub)" <gi...@apache.org> on 2024/02/26 04:53:38 UTC
[PR] ORC-1631: Supports summary output in sizes command [orc]
cxzl25 opened a new pull request, #1816:
URL: https://github.com/apache/orc/pull/1816
### What changes were proposed in this pull request?
Add support for summarizing the number of files, file sizes and file lines in the sizes command.
### Why are the changes needed?
When we count the size of each field, we only know the percentage and the average size of each row, but we do not know the overall value.
### How was this patch tested?
local test
```bash
java -jar orc-tools-2.1.0-SNAPSHOT-uber.jar sizes -h
usage: sizes
-h,--help Print help message
-i,--ignoreExtension Ignore ORC file extension
-s,--summary Summarize the number of files, file size, and
number of file lines
```
```
java -jar orc-tools-2.1.0-SNAPSHOT-uber.jar sizes -s
```
```
Total Files: 5
Total Sizes: 4803687270
Total Rows: 39820045
Percent Bytes/Row Name
26.41 31.86
```
### Was this patch authored or co-authored using generative AI tooling?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@orc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ORC-1631: Support `summary` output in `sizes` command [orc]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun closed pull request #1816: ORC-1631: Support `summary` output in `sizes` command
URL: https://github.com/apache/orc/pull/1816
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@orc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ORC-1631: Support `summary` output in `sizes` command [orc]
Posted by "cxzl25 (via GitHub)" <gi...@apache.org>.
cxzl25 commented on code in PR #1816:
URL: https://github.com/apache/orc/pull/1816#discussion_r1502078352
##########
java/tools/src/java/org/apache/orc/tools/ColumnSizes.java:
##########
@@ -217,6 +227,11 @@ private static Options createOptions() {
.desc("Ignore ORC file extension")
.build());
+ result.addOption(Option.builder("s")
+ .longOpt("summary")
+ .desc("Summarize the number of files, file size, and number of file lines")
Review Comment:
Thanks. Updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@orc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ORC-1631: Support `summary` output in `sizes` command [orc]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on code in PR #1816:
URL: https://github.com/apache/orc/pull/1816#discussion_r1502045007
##########
java/tools/src/java/org/apache/orc/tools/ColumnSizes.java:
##########
@@ -217,6 +227,11 @@ private static Options createOptions() {
.desc("Ignore ORC file extension")
.build());
+ result.addOption(Option.builder("s")
+ .longOpt("summary")
+ .desc("Summarize the number of files, file size, and number of file lines")
Review Comment:
`file lines`? `Line` is not Apache ORC format. Maybe, did you want to say `Rows`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@orc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org