You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by qmlmoon <gi...@git.apache.org> on 2014/10/17 13:33:35 UTC
[GitHub] incubator-flink pull request: [FLINK-1139] Fix HadoopOutputFormat ...
GitHub user qmlmoon opened a pull request:
https://github.com/apache/incubator-flink/pull/157
[FLINK-1139] Fix HadoopOutputFormat for DOP>1
This fix leaves the _temporary directory unremoved. It would be better if we could remove the temp directory after all tasks finished.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/qmlmoon/incubator-flink hadoop
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-flink/pull/157.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #157
----
commit 6512e3a2f5e02bf01e5839d36380b9db741e540d
Author: mingliang <qm...@gmail.com>
Date: 2014-10-17T11:17:45Z
Fix HadoopOutputFormat for DOP>1
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-flink pull request: [FLINK-1139] Fix HadoopOutputFormat ...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on the pull request:
https://github.com/apache/incubator-flink/pull/157#issuecomment-59501331
The problem with this issue is, that you don't know what is happening inside of the {{jobCommit()}} method.
For Hadoop FileOutputFormats it only removes the temp directory but other formats might have important logic in there. So removing that call is not a good solution.
I am currently working on another solution that adds a hook to the JobManager which is called after all output tasks finished. Will open a PR soon.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-flink pull request: [FLINK-1139] Fix HadoopOutputFormat ...
Posted by qmlmoon <gi...@git.apache.org>.
Github user qmlmoon closed the pull request at:
https://github.com/apache/incubator-flink/pull/157
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-flink pull request: [FLINK-1139] Fix HadoopOutputFormat ...
Posted by qmlmoon <gi...@git.apache.org>.
Github user qmlmoon commented on the pull request:
https://github.com/apache/incubator-flink/pull/157#issuecomment-59501445
ok. then that's better to have the hook.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-flink pull request: [FLINK-1139] Fix HadoopOutputFormat ...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on the pull request:
https://github.com/apache/incubator-flink/pull/157#issuecomment-59501414
And sorry that I forgot to assign the issue to myself :-/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-flink pull request: [FLINK-1139] Fix HadoopOutputFormat ...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on the pull request:
https://github.com/apache/incubator-flink/pull/157#issuecomment-59697722
Would you close this PR? You are the only one who can do that without merging it ;-)
Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---