You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by twalthr <gi...@git.apache.org> on 2014/07/21 12:18:27 UTC
[GitHub] incubator-flink pull request: Small fixes for running Hadoop Outpu...
GitHub user twalthr opened a pull request:
https://github.com/apache/incubator-flink/pull/75
Small fixes for running Hadoop OutputFormats with HadoopCompat layer
This PR includes some fixes necessary to run a large Hadoop OutputFormat such as the HCatOutputFormat on HDFS. Since HCatOutputFormat uses a custom output file name the following exception is thrown which is now fixed:
```java
eu.stratosphere.client.program.ProgramInvocationException: The program execution failed: java.lang.IllegalArgumentException: This method can only be called from within a Job
at org.apache.hadoop.mapred.FileOutputFormat.getUniqueName(FileOutputFormat.java:286)
at org.apache.hive.hcatalog.mapreduce.FileOutputFormatContainer.getRecordWriter(FileOutputFormatContainer.java:107)
at org.apache.hive.hcatalog.mapreduce.HCatOutputFormat.getRecordWriter(HCatOutputFormat.java:246)
at eu.stratosphere.hadoopcompatibility.mapreduce.HadoopOutputFormat.open(HadoopOutputFormat.java:122)
```
Furthermore there were some issues with HDFS that were also fixed with this PR.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/twalthr/incubator-flink HadoopCompatHCatReady
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-flink/pull/75.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #75
----
commit 82926a19e71ce03ec404413682a4ffbba4590666
Author: twalthr <in...@twalthr.com>
Date: 2014-07-21T10:11:16Z
Small bug fixes for running hadoop output formats
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-flink pull request: Small fixes for running Hadoop Outpu...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-flink/pull/75
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-flink pull request: Small fixes for running Hadoop Outpu...
Posted by StephanEwen <gi...@git.apache.org>.
Github user StephanEwen commented on the pull request:
https://github.com/apache/incubator-flink/pull/75#issuecomment-52226562
This looks good to me, I will merge it...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---