You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by wenlong88 <gi...@git.apache.org> on 2017/02/09 06:30:38 UTC
[GitHub] flink pull request #3288: [FLINK-5749]unset HADOOP_HOME and HADOOP_CONF_DIR ...
GitHub user wenlong88 opened a pull request:
https://github.com/apache/flink/pull/3288
[FLINK-5749]unset HADOOP_HOME and HADOOP_CONF_DIR to avoid env in build machine failing the UT and IT
Currently when we are trying to build flink on a machine with HADOOP_HOME environment variable set, Test data will be written to HDFS, instead of local tmp dir which is expected. This will cause tests failed.
This PR unsets HADOOP_HOME and HADOOP_CONF_DIR environment variable in pom, to make sure maven run the test cases in the exactly way we want.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/wenlong88/flink jira-5749
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/3288.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3288
----
commit d0e6464f37e2057c508a0e7858d3b42778610637
Author: wenlong.lwl <we...@alibaba-inc.com>
Date: 2017-01-13T06:12:17Z
[to #9896513] Unset HADOOP_HOME and HADOOP_CONF_DIR in pom
Summary:
unset HADOOP_HOME and HADOOP_CONF_DIR to avoid env in build machine failing the UT and IT
fix T9896513
Test Plan: UT/IT
Reviewers: \u9ece\u94a2
Reviewed By: \u9ece\u94a2
Subscribers: P577102
Differential Revision: https://aone.alibaba-inc.com/code/D96861
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #3288: [FLINK-5749]unset HADOOP_HOME and HADOOP_CONF_DIR to avoi...
Posted by StephanEwen <gi...@git.apache.org>.
Github user StephanEwen commented on the issue:
https://github.com/apache/flink/pull/3288
For the `HadoopUtils` in `flink-java` - I think those need to depend on the variables and are also not in the test scope.
The reason why I am asking also do make the change more to `filesystem-connector` and `fs-tests` is that we want to gradually make Flink depend less and less on Hadoop, so to move all Hadoop dependencies into separate optional modules. More targeted environment variable exclusion would fit that design better.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #3288: [FLINK-5749]unset HADOOP_HOME and HADOOP_CONF_DIR ...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/3288
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #3288: [FLINK-5749]unset HADOOP_HOME and HADOOP_CONF_DIR to avoi...
Posted by StephanEwen <gi...@git.apache.org>.
Github user StephanEwen commented on the issue:
https://github.com/apache/flink/pull/3288
Looks good, thank you!
Merging this...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #3288: [FLINK-5749]unset HADOOP_HOME and HADOOP_CONF_DIR to avoi...
Posted by wenlong88 <gi...@git.apache.org>.
Github user wenlong88 commented on the issue:
https://github.com/apache/flink/pull/3288
hi @StephanEwen, the cases in flink-filesystem-connector will fail first in our building machine. I have go through the cases in flink-fs-tests, I think they will fail too if I remove the change from the root pom. I think it is better to set in the root pom, because the HadoopUtils is in the package flink-java, and many modules have dependency on flink-java and users of the utils may have no idea that the HadoopUtils have dependency on the environment variables that will cause wired failures and it is hard to trace.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #3288: [FLINK-5749]unset HADOOP_HOME and HADOOP_CONF_DIR to avoi...
Posted by StephanEwen <gi...@git.apache.org>.
Github user StephanEwen commented on the issue:
https://github.com/apache/flink/pull/3288
I think this is a good addition.
We could try and make this change more targeted to the Maven project that has the affected tests. That would help us keeping the root pom.xml more compact. For example if this affects only tests in `flink-fs-tests` then we can for example make this change in `flink-fs-tests`.
Can you share what tests are affected by this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #3288: [FLINK-5749]unset HADOOP_HOME and HADOOP_CONF_DIR to avoi...
Posted by wenlong88 <gi...@git.apache.org>.
Github user wenlong88 commented on the issue:
https://github.com/apache/flink/pull/3288
hi @StephanEwen , I have move the changes to child module pom, `flink-tests` and `flink-hadoop-compatibility` need to be changed too, because the mapreduce related cases will load output compaction setting from the cluster configuration.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #3288: [FLINK-5749]unset HADOOP_HOME and HADOOP_CONF_DIR to avoi...
Posted by wenlong88 <gi...@git.apache.org>.
Github user wenlong88 commented on the issue:
https://github.com/apache/flink/pull/3288
ok, I will move the changes to filesystem-connector and fs-tests poms.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---