You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2021/07/15 20:45:46 UTC
[spark] branch branch-3.2 updated: [SPARK-36164][INFRA][FOLLOWUP]
Add empty string check back
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-3.2
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.2 by this push:
new 0520497 [SPARK-36164][INFRA][FOLLOWUP] Add empty string check back
0520497 is described below
commit 0520497d600e77fe5bb431b50fc5b26fbbbed693
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Thu Jul 15 13:44:17 2021 -0700
[SPARK-36164][INFRA][FOLLOWUP] Add empty string check back
### What changes were proposed in this pull request?
This is a follow-up of #33371.
At the branch commit GitHub run, we have an empty environment variable.
This PR adds back the empty string check logic.
### Why are the changes needed?
Currently, the failure happens when we use `--modules` in GitHub Action.
```
$ GITHUB_ACTIONS=1 APACHE_SPARK_REF= dev/run-tests.py --modules core
[info] Using build tool sbt with Hadoop profile hadoop3.2 and Hive profile hive2.3 under environment github_actions
fatal: ambiguous argument '': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
Traceback (most recent call last):
File "/Users/dongjoon/APACHE/spark-merge/dev/run-tests.py", line 785, in <module>
main()
File "/Users/dongjoon/APACHE/spark-merge/dev/run-tests.py", line 663, in main
changed_files = identify_changed_files_from_git_commits(
File "/Users/dongjoon/APACHE/spark-merge/dev/run-tests.py", line 91, in identify_changed_files_from_git_commits
raw_output = subprocess.check_output(['git', 'diff', '--name-only', patch_sha, diff_target],
File "/Users/dongjoon/.pyenv/versions/3.9.5/lib/python3.9/subprocess.py", line 424, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/Users/dongjoon/.pyenv/versions/3.9.5/lib/python3.9/subprocess.py", line 528, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['git', 'diff', '--name-only', 'HEAD', '']' returned non-zero exit status 128.
```
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Manually. The following failure is correct in local environment because it passed `identify_changed_files_from_git_commits` already.
```
$ GITHUB_ACTIONS=1 APACHE_SPARK_REF= dev/run-tests.py --modules core
[info] Using build tool sbt with Hadoop profile hadoop3.2 and Hive profile hive2.3 under environment github_actions
Traceback (most recent call last):
File "/Users/dongjoon/APACHE/spark-merge/dev/run-tests.py", line 785, in <module>
main()
File "/Users/dongjoon/APACHE/spark-merge/dev/run-tests.py", line 668, in main
os.environ["GITHUB_SHA"], target_ref=os.environ["GITHUB_PREV_SHA"])
File "/Users/dongjoon/.pyenv/versions/3.9.5/lib/python3.9/os.py", line 679, in __getitem__
raise KeyError(key) from None
KeyError: 'GITHUB_SHA'
```
Closes #33374 from dongjoon-hyun/SPARK-36164.
Authored-by: Dongjoon Hyun <do...@apache.org>
Signed-off-by: Dongjoon Hyun <do...@apache.org>
(cherry picked from commit 5f41a2752fba617524c072ee4f0e3a4e5368be13)
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
dev/run-tests.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/dev/run-tests.py b/dev/run-tests.py
index b27bd57..5243dce 100755
--- a/dev/run-tests.py
+++ b/dev/run-tests.py
@@ -658,7 +658,7 @@ def main():
# If we're running the tests in GitHub Actions, attempt to detect and test
# only the affected modules.
if test_env == "github_actions":
- if "APACHE_SPARK_REF" in os.environ:
+ if "APACHE_SPARK_REF" in os.environ and os.environ["APACHE_SPARK_REF"] != "":
# Fork repository
changed_files = identify_changed_files_from_git_commits(
"HEAD", target_ref=os.environ["APACHE_SPARK_REF"])
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org