You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Hive QA (Jira)" <ji...@apache.org> on 2020/02/03 20:41:00 UTC
[jira] [Commented] (HIVE-22819) Refactor
Hive::listFilesCreatedByQuery to make it faster for object stores
[ https://issues.apache.org/jira/browse/HIVE-22819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17029251#comment-17029251 ]
Hive QA commented on HIVE-22819:
--------------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 37s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 50s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 38s{color} | {color:blue} ql in master has 1533 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 30s{color} | {color:blue} hcatalog/streaming in master has 11 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 27s{color} | {color:blue} streaming in master has 2 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 44s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 46s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 25s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m 9s{color} | {color:red} ql: The patch generated 47 new + 2707 unchanged - 24 fixed = 2754 total (was 2731) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 21s{color} | {color:red} itests/hive-unit: The patch generated 22 new + 177 unchanged - 21 fixed = 199 total (was 198) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m 6s{color} | {color:red} ql generated 1 new + 1532 unchanged - 1 fixed = 1533 total (was 1533) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 46s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 14s{color} | {color:green} The patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 37m 33s{color} | {color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Dead store to filename in org.apache.hadoop.hive.ql.io.AcidUtils.parseBucketId(Path) At AcidUtils.java:org.apache.hadoop.hive.ql.io.AcidUtils.parseBucketId(Path) At AcidUtils.java:[line 434] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-20447/dev-support/hive-personality.sh |
| git revision | master / f83bfbe |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-20447/yetus/diff-checkstyle-ql.txt |
| checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-20447/yetus/diff-checkstyle-itests_hive-unit.txt |
| findbugs | http://104.198.109.242/logs//PreCommit-HIVE-Build-20447/yetus/new-findbugs-ql.html |
| modules | C: ql hcatalog/streaming streaming itests/hive-unit U: . |
| Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-20447/yetus.txt |
| Powered by | Apache Yetus http://yetus.apache.org |
This message was automatically generated.
> Refactor Hive::listFilesCreatedByQuery to make it faster for object stores
> --------------------------------------------------------------------------
>
> Key: HIVE-22819
> URL: https://issues.apache.org/jira/browse/HIVE-22819
> Project: Hive
> Issue Type: Improvement
> Reporter: Marton Bod
> Assignee: Marton Bod
> Priority: Major
> Attachments: HIVE-22819.1.patch
>
>
> {color:#0000ff}Hive::listFilesCreatedByQuery{color} does an exists(), an isDir() and then a listing call. This can be expensive in object stores. We should instead directly list the files in the directory (we'd have to handle an exception if the directory does not exists, but issuing a single call to the object store would most likely still end up being more performant).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)