You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by mxm <gi...@git.apache.org> on 2015/10/07 12:43:16 UTC
[GitHub] flink pull request: [FLINK-2823][yarn] throw exception on file as ...
GitHub user mxm opened a pull request:
https://github.com/apache/flink/pull/1235
[FLINK-2823][yarn] throw exception on file as default file system
Without a distributed file system, Flink cannot distribute its files correctly. Thus, we can throw an exception in case we discover file is the default file system.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mxm/flink FLINK-2823
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/1235.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1235
----
commit eb3c8dd53864b764cbbac98fc44a384f3f9f3955
Author: Maximilian Michels <mx...@apache.org>
Date: 2015-10-07T10:05:21Z
[FLINK-2823][yarn] throw exception on file as default file system
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2823][yarn] throw exception on file as ...
Posted by mxm <gi...@git.apache.org>.
Github user mxm commented on the pull request:
https://github.com/apache/flink/pull/1235#issuecomment-146330037
The Hadoop configuration may also be set through resource files in the classpath. IMHO throwing an exception is only acceptable when we're sure no configuration has been loaded.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2823][yarn] throw exception on file as ...
Posted by mxm <gi...@git.apache.org>.
Github user mxm closed the pull request at:
https://github.com/apache/flink/pull/1235
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2823][yarn] throw exception on file as ...
Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger commented on the pull request:
https://github.com/apache/flink/pull/1235#issuecomment-146303645
I don't think that we can merge this change. We can not assume that file:/// is an invalid path.
Also, I don't think there is a good way to check whether the hadoop configuration has been loaded or whether the user is just running on the default configuration.
I think the only thing we can do is throw an exception if none of the common environment variable are set. But even then, users can put the path to the hadoop configuration into the flink-conf.yaml.
But I think its okay, a workaround for these users would be to just set one of the environment variables to make the exception go away.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2823][yarn] throw exception on file as ...
Posted by mxm <gi...@git.apache.org>.
Github user mxm commented on the pull request:
https://github.com/apache/flink/pull/1235#issuecomment-163220976
Closing this because of the raised objections.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2823][yarn] throw exception on file as ...
Posted by uce <gi...@git.apache.org>.
Github user uce commented on the pull request:
https://github.com/apache/flink/pull/1235#issuecomment-146201155
"file://" can also be a distributed file system in certain setups.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2823][yarn] throw exception on file as ...
Posted by uce <gi...@git.apache.org>.
Github user uce commented on the pull request:
https://github.com/apache/flink/pull/1235#issuecomment-146297878
PS Enforcing hdfs:// would also not allow local testing. I saw the warning log today when I was testing Till's HA PR.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2823][yarn] throw exception on file as ...
Posted by mxm <gi...@git.apache.org>.
Github user mxm commented on the pull request:
https://github.com/apache/flink/pull/1235#issuecomment-146330660
@uce This was not about enforcing HDFS, rather disallowing file.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2823][yarn] throw exception on file as ...
Posted by mxm <gi...@git.apache.org>.
Github user mxm commented on the pull request:
https://github.com/apache/flink/pull/1235#issuecomment-146204887
Outside of the Hadoop world. Yes.
Let's change this to check whether any kind of Hadoop configuration has been loaded previously.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---