You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Joe McDonnell (Code Review)" <ge...@cloudera.org> on 2019/02/07 23:18:50 UTC

[Impala-ASF-CR] IMPALA-8003: Improve logging for diskspace errors from dataload 

Joe McDonnell has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12399


Change subject: IMPALA-8003: Improve logging for diskspace errors from dataload
......................................................................

IMPALA-8003: Improve logging for diskspace errors from dataload

When dataload runs on a machine without enough diskspace, it
can fail with arcane errors that don't mentioned disk space.
This checks the HDFS NameNode log for common errors that indicate
disk space issues ("Failed to place enough replicas" and
"NOT_ENOUGH_STORAGE_SPACE") and prints a better error message.

Change-Id: If8766d819c68345cac7d0a32258415ef1a188f53
---
M testdata/bin/create-load-data.sh
1 file changed, 26 insertions(+), 0 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/99/12399/1
-- 
To view, visit http://gerrit.cloudera.org:8080/12399
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: If8766d819c68345cac7d0a32258415ef1a188f53
Gerrit-Change-Number: 12399
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>

[Impala-ASF-CR] IMPALA-8003: Improve logging for diskspace errors from dataload

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/12399 )

Change subject: IMPALA-8003: Improve logging for diskspace errors from dataload
......................................................................


Patch Set 3:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/12399/3//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/12399/3//COMMIT_MSG@9
PS3, Line 9: When dataload runs on a machine without enough diskspace
> Ah, never mind -- this clearly just applies to the minicluster.
I will tighten up the language. It definitely is meant only for the minicluster.


http://gerrit.cloudera.org:8080/#/c/12399/3/testdata/bin/create-load-data.sh
File testdata/bin/create-load-data.sh:

http://gerrit.cloudera.org:8080/#/c/12399/3/testdata/bin/create-load-data.sh@202
PS3, Line 202:   FIRST_MINICLUSTER_NODE="${IMPALA_CLUSTER_LOGS_DIR}/cdh${CDH_MAJOR_VERSION}-node-1"
> I need to check, but my concern is that this will fail when this script get
Good point, it has no ability to help when loading data to an external cluster, so I will disable this for that scenario.



-- 
To view, visit http://gerrit.cloudera.org:8080/12399
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If8766d819c68345cac7d0a32258415ef1a188f53
Gerrit-Change-Number: 12399
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: David Knupp <dk...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Paul Rogers <pr...@cloudera.com>
Gerrit-Comment-Date: Fri, 15 Mar 2019 22:38:17 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-8003: Improve logging for diskspace errors from dataload

Posted by "David Knupp (Code Review)" <ge...@cloudera.org>.
David Knupp has posted comments on this change. ( http://gerrit.cloudera.org:8080/12399 )

Change subject: IMPALA-8003: Improve logging for diskspace errors from dataload
......................................................................


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/12399/3/testdata/bin/create-load-data.sh
File testdata/bin/create-load-data.sh:

http://gerrit.cloudera.org:8080/#/c/12399/3/testdata/bin/create-load-data.sh@202
PS3, Line 202:   FIRST_MINICLUSTER_NODE="${IMPALA_CLUSTER_LOGS_DIR}/cdh${CDH_MAJOR_VERSION}-node-1"
I need to check, but my concern is that this will fail when this script gets run as part of a CDH cluster test run.



-- 
To view, visit http://gerrit.cloudera.org:8080/12399
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If8766d819c68345cac7d0a32258415ef1a188f53
Gerrit-Change-Number: 12399
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: David Knupp <dk...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Paul Rogers <pr...@cloudera.com>
Gerrit-Comment-Date: Fri, 15 Mar 2019 22:26:41 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-8003: Improve logging for diskspace errors from dataload

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12399 )

Change subject: IMPALA-8003: Improve logging for diskspace errors from dataload
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/2031/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/12399
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If8766d819c68345cac7d0a32258415ef1a188f53
Gerrit-Change-Number: 12399
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Thu, 07 Feb 2019 23:45:04 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8003: Improve logging for diskspace errors from dataload

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12399 )

Change subject: IMPALA-8003: Improve logging for diskspace errors from dataload
......................................................................


Patch Set 3:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/2418/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/12399
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If8766d819c68345cac7d0a32258415ef1a188f53
Gerrit-Change-Number: 12399
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Tue, 12 Mar 2019 18:46:58 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8003: Improve logging for diskspace errors from dataload

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12399 )

Change subject: IMPALA-8003: Improve logging for diskspace errors from dataload
......................................................................


Patch Set 4:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/5339/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/12399
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If8766d819c68345cac7d0a32258415ef1a188f53
Gerrit-Change-Number: 12399
Gerrit-PatchSet: 4
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: David Knupp <dk...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Paul Rogers <pr...@cloudera.com>
Gerrit-Comment-Date: Thu, 26 Dec 2019 21:27:28 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8003: Improve logging for diskspace errors from dataload

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/12399 )

Change subject: IMPALA-8003: Improve logging for diskspace errors from dataload
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/12399/1/testdata/bin/create-load-data.sh
File testdata/bin/create-load-data.sh:

http://gerrit.cloudera.org:8080/#/c/12399/1/testdata/bin/create-load-data.sh@134
PS1, Line 134: echo "Diskspace at start of dataload:"
             : df "${IMPALA_HOME}"
> Do you think it would be worth adding a comparison and a warning here if fr
Added a warning.



-- 
To view, visit http://gerrit.cloudera.org:8080/12399
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If8766d819c68345cac7d0a32258415ef1a188f53
Gerrit-Change-Number: 12399
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Fri, 01 Mar 2019 21:59:18 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-8003: Improve logging for diskspace errors from dataload

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Hello Laszlo Gaal, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/12399

to look at the new patch set (#3).

Change subject: IMPALA-8003: Improve logging for diskspace errors from dataload
......................................................................

IMPALA-8003: Improve logging for diskspace errors from dataload

When dataload runs on a machine without enough diskspace, it
can fail with arcane errors that don't mentioned disk space.
This checks the HDFS NameNode log for common errors that indicate
disk space issues ("Failed to place enough replicas" and
"NOT_ENOUGH_STORAGE_SPACE") and prints a better error message.

Change-Id: If8766d819c68345cac7d0a32258415ef1a188f53
---
M testdata/bin/create-load-data.sh
1 file changed, 26 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/99/12399/3
-- 
To view, visit http://gerrit.cloudera.org:8080/12399
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If8766d819c68345cac7d0a32258415ef1a188f53
Gerrit-Change-Number: 12399
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>

[Impala-ASF-CR] IMPALA-8003: Improve logging for diskspace errors from dataload

Posted by "Laszlo Gaal (Code Review)" <ge...@cloudera.org>.
Laszlo Gaal has posted comments on this change. ( http://gerrit.cloudera.org:8080/12399 )

Change subject: IMPALA-8003: Improve logging for diskspace errors from dataload
......................................................................


Patch Set 1: Code-Review+1

(1 comment)

http://gerrit.cloudera.org:8080/#/c/12399/1/testdata/bin/create-load-data.sh
File testdata/bin/create-load-data.sh:

http://gerrit.cloudera.org:8080/#/c/12399/1/testdata/bin/create-load-data.sh@134
PS1, Line 134: echo "Diskspace at start of dataload:"
             : df "${IMPALA_HOME}"
Do you think it would be worth adding a comparison and a warning here if free disk space is below 120 GBs?



-- 
To view, visit http://gerrit.cloudera.org:8080/12399
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If8766d819c68345cac7d0a32258415ef1a188f53
Gerrit-Change-Number: 12399
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Fri, 08 Feb 2019 13:17:50 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-8003: Improve logging for diskspace errors from dataload

Posted by "Laszlo Gaal (Code Review)" <ge...@cloudera.org>.
Laszlo Gaal has posted comments on this change. ( http://gerrit.cloudera.org:8080/12399 )

Change subject: IMPALA-8003: Improve logging for diskspace errors from dataload
......................................................................


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/12399/2/testdata/bin/create-load-data.sh
File testdata/bin/create-load-data.sh:

http://gerrit.cloudera.org:8080/#/c/12399/2/testdata/bin/create-load-data.sh@134
PS2, Line 134: df ${IMPALA_HOME} --output=avail
Unfortunately the version of df on older systems (I tested CentOS 6.4) does not understand the "--output=..." argument, and returns an error (1). If the logic to parse out the available bytes field becomes too complex, it may be just not worth doing it -- I leave it to your judgement.



-- 
To view, visit http://gerrit.cloudera.org:8080/12399
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If8766d819c68345cac7d0a32258415ef1a188f53
Gerrit-Change-Number: 12399
Gerrit-PatchSet: 2
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Mon, 11 Mar 2019 20:43:51 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-8003: Improve logging for diskspace errors from dataload

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Hello Laszlo Gaal, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/12399

to look at the new patch set (#2).

Change subject: IMPALA-8003: Improve logging for diskspace errors from dataload
......................................................................

IMPALA-8003: Improve logging for diskspace errors from dataload

When dataload runs on a machine without enough diskspace, it
can fail with arcane errors that don't mentioned disk space.
This checks the HDFS NameNode log for common errors that indicate
disk space issues ("Failed to place enough replicas" and
"NOT_ENOUGH_STORAGE_SPACE") and prints a better error message.

Change-Id: If8766d819c68345cac7d0a32258415ef1a188f53
---
M testdata/bin/create-load-data.sh
1 file changed, 32 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/99/12399/2
-- 
To view, visit http://gerrit.cloudera.org:8080/12399
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If8766d819c68345cac7d0a32258415ef1a188f53
Gerrit-Change-Number: 12399
Gerrit-PatchSet: 2
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>

[Impala-ASF-CR] IMPALA-8003: Improve logging for diskspace errors from dataload

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12399 )

Change subject: IMPALA-8003: Improve logging for diskspace errors from dataload
......................................................................


Patch Set 2:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/2323/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/12399
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If8766d819c68345cac7d0a32258415ef1a188f53
Gerrit-Change-Number: 12399
Gerrit-PatchSet: 2
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Fri, 01 Mar 2019 22:39:53 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8003: Improve logging for diskspace errors from dataload

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/12399 )

Change subject: IMPALA-8003: Improve logging for diskspace errors from dataload
......................................................................


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/12399/2/testdata/bin/create-load-data.sh
File testdata/bin/create-load-data.sh:

http://gerrit.cloudera.org:8080/#/c/12399/2/testdata/bin/create-load-data.sh@134
PS2, Line 134: df ${IMPALA_HOME} --output=avail
> Unfortunately the version of df on older systems (I tested CentOS 6.4) does
Good point. I went back to the previous version that doesn't do a special message.



-- 
To view, visit http://gerrit.cloudera.org:8080/12399
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If8766d819c68345cac7d0a32258415ef1a188f53
Gerrit-Change-Number: 12399
Gerrit-PatchSet: 2
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Comment-Date: Tue, 12 Mar 2019 18:15:40 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-8003: Improve logging for diskspace errors from dataload

Posted by "Laszlo Gaal (Code Review)" <ge...@cloudera.org>.
Laszlo Gaal has posted comments on this change. ( http://gerrit.cloudera.org:8080/12399 )

Change subject: IMPALA-8003: Improve logging for diskspace errors from dataload
......................................................................


Patch Set 3: Code-Review+1


-- 
To view, visit http://gerrit.cloudera.org:8080/12399
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If8766d819c68345cac7d0a32258415ef1a188f53
Gerrit-Change-Number: 12399
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: David Knupp <dk...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Paul Rogers <pr...@cloudera.com>
Gerrit-Comment-Date: Fri, 15 Mar 2019 16:28:07 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8003: Improve logging for diskspace errors from dataload

Posted by "David Knupp (Code Review)" <ge...@cloudera.org>.
David Knupp has posted comments on this change. ( http://gerrit.cloudera.org:8080/12399 )

Change subject: IMPALA-8003: Improve logging for diskspace errors from dataload
......................................................................


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/12399/3//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/12399/3//COMMIT_MSG@9
PS3, Line 9: When dataload runs on a machine without enough diskspace
By "machine", do you mean a local minicluster, or also when loading data onto an actual cluster?



-- 
To view, visit http://gerrit.cloudera.org:8080/12399
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If8766d819c68345cac7d0a32258415ef1a188f53
Gerrit-Change-Number: 12399
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: David Knupp <dk...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Paul Rogers <pr...@cloudera.com>
Gerrit-Comment-Date: Fri, 15 Mar 2019 21:59:46 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-8003: Improve logging for diskspace errors from dataload

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has abandoned this change. ( http://gerrit.cloudera.org:8080/12399 )

Change subject: IMPALA-8003: Improve logging for diskspace errors from dataload
......................................................................


Abandoned
-- 
To view, visit http://gerrit.cloudera.org:8080/12399
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: abandon
Gerrit-Change-Id: If8766d819c68345cac7d0a32258415ef1a188f53
Gerrit-Change-Number: 12399
Gerrit-PatchSet: 4
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Paul Rogers <pr...@cloudera.com>

[Impala-ASF-CR] IMPALA-8003: Improve logging for diskspace errors from dataload

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Hello Laszlo Gaal, Paul Rogers, David Knupp, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/12399

to look at the new patch set (#4).

Change subject: IMPALA-8003: Improve logging for diskspace errors from dataload
......................................................................

IMPALA-8003: Improve logging for diskspace errors from dataload

When running dataload for a local HDFS minicluster on a machine
without enough diskspace, it can fail with arcane errors that don't
mentioned disk space. This checks the HDFS NameNode log for common
errors that indicate disk space issues ("Failed to place enough replicas"
and "NOT_ENOUGH_STORAGE_SPACE") and prints a better error message.
This does not do any checks for non-HDFS or remote clusters.

Change-Id: If8766d819c68345cac7d0a32258415ef1a188f53
---
M testdata/bin/create-load-data.sh
1 file changed, 31 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/99/12399/4
-- 
To view, visit http://gerrit.cloudera.org:8080/12399
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If8766d819c68345cac7d0a32258415ef1a188f53
Gerrit-Change-Number: 12399
Gerrit-PatchSet: 4
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: David Knupp <dk...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Paul Rogers <pr...@cloudera.com>

[Impala-ASF-CR] IMPALA-8003: Improve logging for diskspace errors from dataload

Posted by "David Knupp (Code Review)" <ge...@cloudera.org>.
David Knupp has posted comments on this change. ( http://gerrit.cloudera.org:8080/12399 )

Change subject: IMPALA-8003: Improve logging for diskspace errors from dataload
......................................................................


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/12399/3//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/12399/3//COMMIT_MSG@9
PS3, Line 9: When dataload runs on a machine without enough diskspace
> By "machine", do you mean a local minicluster, or also when loading data on
Ah, never mind -- this clearly just applies to the minicluster.



-- 
To view, visit http://gerrit.cloudera.org:8080/12399
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If8766d819c68345cac7d0a32258415ef1a188f53
Gerrit-Change-Number: 12399
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: David Knupp <dk...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal <la...@cloudera.com>
Gerrit-Reviewer: Paul Rogers <pr...@cloudera.com>
Gerrit-Comment-Date: Fri, 15 Mar 2019 22:23:36 +0000
Gerrit-HasComments: Yes