You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Lars Volker (Code Review)" <ge...@cloudera.org> on 2018/06/04 20:09:08 UTC
[Impala-ASF-CR] IMPALA-7008/IMPALA-7070: Increase log level for HDFS S3 code
Lars Volker has uploaded this change for review. ( http://gerrit.cloudera.org:8080/10596
Change subject: IMPALA-7008/IMPALA-7070: Increase log level for HDFS S3 code
......................................................................
IMPALA-7008/IMPALA-7070: Increase log level for HDFS S3 code
This change increases the log level for the HDFS S3 code to DEBUG to
get more information on both Jiras.
Change-Id: If5f9650a322f9c987f5f4e322681ab290cf487ee
---
M fe/src/test/resources/log4j.properties.template
1 file changed, 3 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/96/10596/1
--
To view, visit http://gerrit.cloudera.org:8080/10596
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: If5f9650a322f9c987f5f4e322681ab290cf487ee
Gerrit-Change-Number: 10596
Gerrit-PatchSet: 1
Gerrit-Owner: Lars Volker <lv...@cloudera.com>
[Impala-ASF-CR] IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10596 )
Change subject: IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code
......................................................................
IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code
This change increases the log level for the HDFS S3 code to DEBUG to
get more information on both Jiras.
Change-Id: If5f9650a322f9c987f5f4e322681ab290cf487ee
Reviewed-on: http://gerrit.cloudera.org:8080/10596
Reviewed-by: Lars Volker <lv...@cloudera.com>
Reviewed-by: Sailesh Mukil <sa...@cloudera.com>
Reviewed-by: Tim Armstrong <ta...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M fe/src/test/resources/log4j.properties.template
1 file changed, 3 insertions(+), 0 deletions(-)
Approvals:
Lars Volker: Looks good to me, but someone else must approve
Sailesh Mukil: Looks good to me, but someone else must approve
Tim Armstrong: Looks good to me, approved
Impala Public Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/10596
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: If5f9650a322f9c987f5f4e322681ab290cf487ee
Gerrit-Change-Number: 10596
Gerrit-PatchSet: 3
Gerrit-Owner: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Sailesh Mukil <sa...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
[Impala-ASF-CR] IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10596 )
Change subject: IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code
......................................................................
Patch Set 2: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/10596
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If5f9650a322f9c987f5f4e322681ab290cf487ee
Gerrit-Change-Number: 10596
Gerrit-PatchSet: 2
Gerrit-Owner: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Sailesh Mukil <sa...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Wed, 06 Jun 2018 20:21:15 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code
Posted by "Sailesh Mukil (Code Review)" <ge...@cloudera.org>.
Sailesh Mukil has posted comments on this change. ( http://gerrit.cloudera.org:8080/10596 )
Change subject: IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code
......................................................................
Patch Set 2: Code-Review+1
> My private build hit IMPALA-7070 when loading the testdata and
> didn't even get to test execution. It printed DEBUG statements as
> expected though and I reached out to some HDFS folks. Should we
> proceed with merging this and accept the risk that it might slow
> down some tests?
>
> I'm in favor.
I'm in favor too. But we need to remember to disable once we've fixed these issues.
--
To view, visit http://gerrit.cloudera.org:8080/10596
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If5f9650a322f9c987f5f4e322681ab290cf487ee
Gerrit-Change-Number: 10596
Gerrit-PatchSet: 2
Gerrit-Owner: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Sailesh Mukil <sa...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Tue, 05 Jun 2018 17:31:32 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/10596 )
Change subject: IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code
......................................................................
Patch Set 2: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/10596
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If5f9650a322f9c987f5f4e322681ab290cf487ee
Gerrit-Change-Number: 10596
Gerrit-PatchSet: 2
Gerrit-Owner: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Sailesh Mukil <sa...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Wed, 06 Jun 2018 06:11:39 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7008/IMPALA-7070: Increase log level for HDFS S3 code
Posted by "Philip Zeyliger (Code Review)" <ge...@cloudera.org>.
Philip Zeyliger has posted comments on this change. ( http://gerrit.cloudera.org:8080/10596 )
Change subject: IMPALA-7008/IMPALA-7070: Increase log level for HDFS S3 code
......................................................................
Patch Set 1:
Surely, we're not shipping fe/src/test/resources, so it doesn't matter vis-a-vis releases? I didn't double-check that, but we should have a way to specify log4j for our tests independently of what we recommend in clusters.
--
To view, visit http://gerrit.cloudera.org:8080/10596
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If5f9650a322f9c987f5f4e322681ab290cf487ee
Gerrit-Change-Number: 10596
Gerrit-PatchSet: 1
Gerrit-Owner: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Mon, 04 Jun 2018 20:37:59 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code
Posted by "Lars Volker (Code Review)" <ge...@cloudera.org>.
Hello Philip Zeyliger, Tim Armstrong,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/10596
to look at the new patch set (#2).
Change subject: IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code
......................................................................
IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code
This change increases the log level for the HDFS S3 code to DEBUG to
get more information on both Jiras.
Change-Id: If5f9650a322f9c987f5f4e322681ab290cf487ee
---
M fe/src/test/resources/log4j.properties.template
1 file changed, 3 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/96/10596/2
--
To view, visit http://gerrit.cloudera.org:8080/10596
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If5f9650a322f9c987f5f4e322681ab290cf487ee
Gerrit-Change-Number: 10596
Gerrit-PatchSet: 2
Gerrit-Owner: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
[Impala-ASF-CR] IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code
Posted by "Lars Volker (Code Review)" <ge...@cloudera.org>.
Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/10596 )
Change subject: IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code
......................................................................
Patch Set 2:
> > My private build hit IMPALA-7070 when loading the testdata and
> > didn't even get to test execution. It printed DEBUG statements as
> > expected though and I reached out to some HDFS folks. Should we
> > proceed with merging this and accept the risk that it might slow
> > down some tests?
> >
> > I'm in favor.
>
> I'm in favor too. But we need to remember to disable once we've
> fixed these issues.
I filed IMPALA-7117 to track removing it and made it a Blocker for future releases.
--
To view, visit http://gerrit.cloudera.org:8080/10596
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If5f9650a322f9c987f5f4e322681ab290cf487ee
Gerrit-Change-Number: 10596
Gerrit-PatchSet: 2
Gerrit-Owner: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Sailesh Mukil <sa...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Tue, 05 Jun 2018 18:14:08 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7008/IMPALA-7070: Increase log level for HDFS S3 code
Posted by "Philip Zeyliger (Code Review)" <ge...@cloudera.org>.
Philip Zeyliger has posted comments on this change. ( http://gerrit.cloudera.org:8080/10596 )
Change subject: IMPALA-7008/IMPALA-7070: Increase log level for HDFS S3 code
......................................................................
Patch Set 1: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/10596
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If5f9650a322f9c987f5f4e322681ab290cf487ee
Gerrit-Change-Number: 10596
Gerrit-PatchSet: 1
Gerrit-Owner: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Comment-Date: Mon, 04 Jun 2018 20:22:09 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7008/IMPALA-7070: Increase log level for HDFS S3 code
Posted by "Lars Volker (Code Review)" <ge...@cloudera.org>.
Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/10596 )
Change subject: IMPALA-7008/IMPALA-7070: Increase log level for HDFS S3 code
......................................................................
Patch Set 1:
I started a private build to check for additional test failures.
--
To view, visit http://gerrit.cloudera.org:8080/10596
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If5f9650a322f9c987f5f4e322681ab290cf487ee
Gerrit-Change-Number: 10596
Gerrit-PatchSet: 1
Gerrit-Owner: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Mon, 04 Jun 2018 20:50:33 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7008/IMPALA-7070: Increase log level for HDFS S3 code
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/10596 )
Change subject: IMPALA-7008/IMPALA-7070: Increase log level for HDFS S3 code
......................................................................
Patch Set 1:
Is there any chance this could slow down S3 builds enoguh to cause more test failures?
--
To view, visit http://gerrit.cloudera.org:8080/10596
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If5f9650a322f9c987f5f4e322681ab290cf487ee
Gerrit-Change-Number: 10596
Gerrit-PatchSet: 1
Gerrit-Owner: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Mon, 04 Jun 2018 20:32:58 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code
Posted by "Sailesh Mukil (Code Review)" <ge...@cloudera.org>.
Sailesh Mukil has posted comments on this change. ( http://gerrit.cloudera.org:8080/10596 )
Change subject: IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code
......................................................................
Patch Set 2:
> Surely, we're not shipping fe/src/test/resources, so it doesn't
> matter vis-a-vis releases? I didn't double-check that, but we
> should have a way to specify log4j for our tests independently of
> what we recommend in clusters.
+1 to Phil's comment, it can help with testing a lot of Hadoop features that aren't enabled by default.
Sharing Lars' sentiment, I've spent quite a lot of time trying to repro a different flaky S3 failure too, with no avail. This will definitely help catch the bug(s) when it happens on a nightly run. But we should also make sure that we turn it off later.
--
To view, visit http://gerrit.cloudera.org:8080/10596
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If5f9650a322f9c987f5f4e322681ab290cf487ee
Gerrit-Change-Number: 10596
Gerrit-PatchSet: 2
Gerrit-Owner: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Sailesh Mukil <sa...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Tue, 05 Jun 2018 16:08:24 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7008/IMPALA-7070: Increase log level for HDFS S3 code
Posted by "Lars Volker (Code Review)" <ge...@cloudera.org>.
Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/10596 )
Change subject: IMPALA-7008/IMPALA-7070: Increase log level for HDFS S3 code
......................................................................
Patch Set 1:
I would say there is a chance to slow things down and more tests could fail, but overall we haven't had a green build in a while and I'd be willing to give it a shot. Let me know if you prefer running this on a private branch in a loop.
I also created IMPALA-7117 to reduce the logging before the next release.
--
To view, visit http://gerrit.cloudera.org:8080/10596
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If5f9650a322f9c987f5f4e322681ab290cf487ee
Gerrit-Change-Number: 10596
Gerrit-PatchSet: 1
Gerrit-Owner: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Mon, 04 Jun 2018 20:35:04 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code
Posted by "Lars Volker (Code Review)" <ge...@cloudera.org>.
Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/10596 )
Change subject: IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code
......................................................................
Patch Set 2: Code-Review+1
My private build hit IMPALA-7070 when loading the testdata and didn't even get to test execution. It printed DEBUG statements as expected though and I reached out to some HDFS folks. Should we proceed with merging this and accept the risk that it might slow down some tests?
I'm in favor.
--
To view, visit http://gerrit.cloudera.org:8080/10596
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If5f9650a322f9c987f5f4e322681ab290cf487ee
Gerrit-Change-Number: 10596
Gerrit-PatchSet: 2
Gerrit-Owner: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Sailesh Mukil <sa...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Tue, 05 Jun 2018 17:29:51 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10596 )
Change subject: IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code
......................................................................
Patch Set 2:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2608/
--
To view, visit http://gerrit.cloudera.org:8080/10596
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If5f9650a322f9c987f5f4e322681ab290cf487ee
Gerrit-Change-Number: 10596
Gerrit-PatchSet: 2
Gerrit-Owner: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Sailesh Mukil <sa...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Wed, 06 Jun 2018 16:50:52 +0000
Gerrit-HasComments: No