You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Kengo Seki (JIRA)" <ji...@apache.org> on 2015/08/16 18:35:45 UTC
[jira] [Updated] (HADOOP-12303) test-patch pylint plugin fails
silently and votes +1 incorrectly
[ https://issues.apache.org/jira/browse/HADOOP-12303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kengo Seki updated HADOOP-12303:
--------------------------------
Attachment: HADOOP-12303.HADOOP-12111.00.patch
Attaching a patch. I confirmed the pylint plugin detects the above case correctly and passes some simple regression tests.
{code}
[sekikn@mobile hadoop]$ cat /tmp/test.patch
diff --git a/dev-support/releasedocmaker.py b/dev-support/releasedocmaker.py
index 3c398be..6c862ee 100755
--- a/dev-support/releasedocmaker.py
+++ b/dev-support/releasedocmaker.py
@@ -580,4 +580,4 @@ def main():
sys.exit(1)
if __name__ == "__main__":
- main()
+ main( )
[sekikn@mobile hadoop]$ locale
LANG=
LC_COLLATE="C"
LC_CTYPE="UTF-8"
LC_MESSAGES="C"
LC_MONETARY="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_ALL=
[sekikn@mobile hadoop]$ dev-support/test-patch.sh --basedir=/Users/sekikn/dev/hadoop --project=hadoop /tmp/test.patch
(snip)
| Vote | Subsystem | Runtime | Comment
============================================================================
| +1 | @author | 0m 00s | The patch does not contain any @author
| | | | tags.
| +1 | asflicense | 0m 31s | Patch does not generate ASF License
| | | | warnings.
| -1 | pylint | 0m 00s | Something bad seems to have happened in
| | | | running pylint. Please check pylint
| | | | stderr files.
| +1 | whitespace | 0m 00s | Patch has no whitespace issues.
| | | 0m 32s |
|| Subsystem || Report/Notes ||
============================================================================
| git revision | HADOOP-12111 / 565d9bf |
| Optional Tests | asflicense pylint |
| uname | Darwin mobile.local 14.4.0 Darwin Kernel Version 14.4.0: Thu May 28 11:35:04 PDT 2015; root:xnu-2782.30.5~1/RELEASE_X86_64 x86_64 |
| Build tool | maven |
| Personality | /Users/sekikn/hadoop/dev-support/personality/hadoop.sh |
| Default Java | 1.7.0_80 |
| pylint | prepatch stderr: /private/tmp/test-patch-hadoop/48440/pylint.48440.19203 |
| pylint | postpatch stderr: /private/tmp/test-patch-hadoop/48440/pylint.48440.3046 |
| Max memory used | 46MB |
============================================================================
============================================================================
Finished build.
============================================================================
============================================================================
[sekikn@mobile hadoop]$ cat /private/tmp/test-patch-hadoop/48440/pylint.48440.3046
No config file found, using default configuration
Traceback (most recent call last):
File "/usr/local/bin/pylint", line 11, in <module>
sys.exit(run_pylint())
File "/Library/Python/2.7/site-packages/pylint/__init__.py", line 23, in run_pylint
Run(sys.argv[1:])
File "/Library/Python/2.7/site-packages/pylint/lint.py", line 1332, in __init__
linter.check(args)
File "/Library/Python/2.7/site-packages/pylint/lint.py", line 747, in check
self._do_check(files_or_modules)
File "/Library/Python/2.7/site-packages/pylint/lint.py", line 869, in _do_check
self.check_astroid_module(ast_node, walker, rawcheckers, tokencheckers)
File "/Library/Python/2.7/site-packages/pylint/lint.py", line 944, in check_astroid_module
checker.process_tokens(tokens)
File "/Library/Python/2.7/site-packages/pylint/checkers/format.py", line 743, in process_tokens
self.check_indent_level(token, indents[-1]+1, line_num)
File "/Library/Python/2.7/site-packages/pylint/checkers/format.py", line 963, in check_indent_level
expected * unit_size))
File "/Library/Python/2.7/site-packages/pylint/checkers/__init__.py", line 101, in add_message
self.linter.add_message(msg_id, line, node, args, confidence)
File "/Library/Python/2.7/site-packages/pylint/utils.py", line 410, in add_message
(abspath, path, module, obj, line or 1, col_offset or 0), msg, confidence))
File "/Library/Python/2.7/site-packages/pylint/reporters/text.py", line 61, in handle_message
self.write_message(msg)
File "/Library/Python/2.7/site-packages/pylint/reporters/text.py", line 51, in write_message
self.writeln(msg.format(self._template))
File "/Library/Python/2.7/site-packages/pylint/reporters/__init__.py", line 94, in writeln
print(self.encode(string), file=self.out)
File "/Library/Python/2.7/site-packages/pylint/reporters/__init__.py", line 84, in encode
locale.getdefaultlocale()[1] or
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/locale.py", line 511, in getdefaultlocale
return _parse_localename(localename)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/locale.py", line 443, in _parse_localename
raise ValueError, 'unknown locale: %s' % localename
ValueError: unknown locale: UTF-8
{code}
regression (pylint fails):
{code}
(continued)
[sekikn@mobile hadoop]$ export LC_ALL=C
[sekikn@mobile hadoop]$ dev-support/test-patch.sh --basedir=/Users/sekikn/dev/hadoop --project=hadoop /tmp/test.patch
(snip)
| Vote | Subsystem | Runtime | Comment
============================================================================
| +1 | @author | 0m 00s | The patch does not contain any @author
| | | | tags.
| +1 | asflicense | 0m 24s | Patch does not generate ASF License
| | | | warnings.
| -1 | pylint | 0m 05s | The applied patch generated 2 new pylint
| | | | issues (total was 437, now 439).
| +1 | whitespace | 0m 00s | Patch has no whitespace issues.
| | | 0m 30s |
|| Subsystem || Report/Notes ||
============================================================================
| git revision | HADOOP-12111 / 565d9bf |
| Optional Tests | asflicense pylint |
| uname | Darwin mobile.local 14.4.0 Darwin Kernel Version 14.4.0: Thu May 28 11:35:04 PDT 2015; root:xnu-2782.30.5~1/RELEASE_X86_64 x86_64 |
| Build tool | maven |
| Personality | /Users/sekikn/hadoop/dev-support/personality/hadoop.sh |
| Default Java | 1.7.0_80 |
| pylint | v1.4.4 |
| pylint | /private/tmp/test-patch-hadoop/54107/diff-patch-pylint.txt |
| Max memory used | 45MB |
(snip)
[sekikn@mobile hadoop]$ cat /private/tmp/test-patch-hadoop/54107/diff-patch-pylint.txt
dev-support/releasedocmaker.py:583: [C0326(bad-whitespace), ] No space allowed after bracket
main( )
^
dev-support/releasedocmaker.py:583: [C0326(bad-whitespace), ] No space allowed before bracket
main( )
^
{code}
regression (pylint succeeds):
{code}
(continued)
[sekikn@mobile hadoop]$ cat /tmp/test.patch2
diff --git a/dev-support/releasedocmaker.py b/dev-support/releasedocmaker.py
index 3c398be..05aa347 100755
--- a/dev-support/releasedocmaker.py
+++ b/dev-support/releasedocmaker.py
@@ -581,3 +581,4 @@ def main():
if __name__ == "__main__":
main()
+
[sekikn@mobile hadoop]$ dev-support/test-patch.sh --basedir=/Users/sekikn/dev/hadoop --project=hadoop /tmp/test.patch2
(snip)
| Vote | Subsystem | Runtime | Comment
============================================================================
| +1 | @author | 0m 00s | The patch does not contain any @author
| | | | tags.
| +1 | asflicense | 0m 27s | Patch does not generate ASF License
| | | | warnings.
| +1 | pylint | 0m 05s | There were no new pylint issues.
| +1 | whitespace | 0m 00s | Patch has no whitespace issues.
| | | 0m 33s |
{code}
> test-patch pylint plugin fails silently and votes +1 incorrectly
> ----------------------------------------------------------------
>
> Key: HADOOP-12303
> URL: https://issues.apache.org/jira/browse/HADOOP-12303
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: yetus
> Affects Versions: HADOOP-12111
> Reporter: Kengo Seki
> Attachments: HADOOP-12303.HADOOP-12111.00.patch
>
>
> This patch
> {code}
> [sekikn@mobile hadoop]$ cat /tmp/test.patch
> diff --git a/dev-support/releasedocmaker.py b/dev-support/releasedocmaker.py
> index 37bd58a..7cd6dd3 100755
> --- a/dev-support/releasedocmaker.py
> +++ b/dev-support/releasedocmaker.py
> @@ -580,4 +580,4 @@ def main():
> sys.exit(1)
>
> if __name__ == "__main__":
> - main()
> + main( )
> {code}
> is supposed to cause the following pylint errors.
> {code}
> C:583, 0: No space allowed after bracket
> main( )
> ^ (bad-whitespace)
> C:583, 0: No space allowed before bracket
> main( )
> ^ (bad-whitespace)
> {code}
> But the system locale is set as follows, pylint check is passed, and there is no pylint output.
> {code}
> [sekikn@mobile hadoop]$ locale
> LANG=
> LC_COLLATE="C"
> LC_CTYPE="UTF-8"
> LC_MESSAGES="C"
> LC_MONETARY="C"
> LC_NUMERIC="C"
> LC_TIME="C"
> LC_ALL=
> [sekikn@mobile hadoop]$ dev-support/test-patch.sh --basedir=/Users/sekikn/dev/hadoop --project=hadoop /tmp/test.patch
> (snip)
> | Vote | Subsystem | Runtime | Comment
> ============================================================================
> | +1 | @author | 0m 00s | The patch does not contain any @author
> | | | | tags.
> | +1 | asflicense | 0m 21s | Patch does not generate ASF License
> | | | | warnings.
> | +1 | pylint | 0m 01s | There were no new pylint issues.
> | +1 | whitespace | 0m 00s | Patch has no whitespace issues.
> | | | 0m 24s |
> (snip)
> [sekikn@mobile hadoop]$ cat /private/tmp/test-patch-hadoop/8656/branch-pylint-result.txt
> [sekikn@mobile hadoop]$ cat /private/tmp/test-patch-hadoop/8656/patch-pylint-result.txt
> [sekikn@mobile hadoop]$ cat /private/tmp/test-patch-hadoop/8656/diff-patch-pylint.txt
> {code}
> Removing '2>/dev/null' from pylint.sh reveals the root cause. Setting LC_ALL or LC_CTYPE such like 'en_US.UTF-8' solves this problem.
> {code}
> ============================================================================
> ============================================================================
> pylint plugin: prepatch
> ============================================================================
> ============================================================================
> ============================================================================
> Running pylint against modified python scripts.
> No config file found, using default configuration
> Traceback (most recent call last):
> File "/usr/local/bin/pylint", line 11, in <module>
> sys.exit(run_pylint())
> File "/Library/Python/2.7/site-packages/pylint/__init__.py", line 23, in run_pylint
> Run(sys.argv[1:])
> File "/Library/Python/2.7/site-packages/pylint/lint.py", line 1332, in __init__
> linter.check(args)
> File "/Library/Python/2.7/site-packages/pylint/lint.py", line 747, in check
> self._do_check(files_or_modules)
> File "/Library/Python/2.7/site-packages/pylint/lint.py", line 869, in _do_check
> self.check_astroid_module(ast_node, walker, rawcheckers, tokencheckers)
> File "/Library/Python/2.7/site-packages/pylint/lint.py", line 944, in check_astroid_module
> checker.process_tokens(tokens)
> File "/Library/Python/2.7/site-packages/pylint/checkers/format.py", line 743, in process_tokens
> self.check_indent_level(token, indents[-1]+1, line_num)
> File "/Library/Python/2.7/site-packages/pylint/checkers/format.py", line 963, in check_indent_level
> expected * unit_size))
> File "/Library/Python/2.7/site-packages/pylint/checkers/__init__.py", line 101, in add_message
> self.linter.add_message(msg_id, line, node, args, confidence)
> File "/Library/Python/2.7/site-packages/pylint/utils.py", line 410, in add_message
> (abspath, path, module, obj, line or 1, col_offset or 0), msg, confidence))
> File "/Library/Python/2.7/site-packages/pylint/reporters/text.py", line 61, in handle_message
> self.write_message(msg)
> File "/Library/Python/2.7/site-packages/pylint/reporters/text.py", line 51, in write_message
> self.writeln(msg.format(self._template))
> File "/Library/Python/2.7/site-packages/pylint/reporters/__init__.py", line 94, in writeln
> print(self.encode(string), file=self.out)
> File "/Library/Python/2.7/site-packages/pylint/reporters/__init__.py", line 84, in encode
> locale.getdefaultlocale()[1] or
> File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/locale.py", line 511, in getdefaultlocale
> return _parse_localename(localename)
> File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/locale.py", line 443, in _parse_localename
> raise ValueError, 'unknown locale: %s' % localename
> ValueError: unknown locale: UTF-8
> {code}
> In such a case, pylint plugin should fail and vote -1.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)