You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@crunch.apache.org by "Josh Wills (JIRA)" <ji...@apache.org> on 2015/03/03 22:56:05 UTC
[jira] [Resolved] (CRUNCH-502) OutputFormat has inconsistent
context state in interface functions
[ https://issues.apache.org/jira/browse/CRUNCH-502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Josh Wills resolved CRUNCH-502.
-------------------------------
Resolution: Fixed
Fix Version/s: 0.12.0
Pushed to master.
> OutputFormat has inconsistent context state in interface functions
> ------------------------------------------------------------------
>
> Key: CRUNCH-502
> URL: https://issues.apache.org/jira/browse/CRUNCH-502
> Project: Crunch
> Issue Type: Bug
> Components: IO
> Affects Versions: 0.12.0
> Reporter: Mārtiņš Kalvāns
> Assignee: Josh Wills
> Fix For: 0.12.0
>
> Attachments: CRUNCH-502.patch, CRUNCH-502b.patch, CRUNCH-502c.patch
>
>
> I created example project to demonstrate problematic behaviour:
> https://github.com/sisidra/crunch-ofb
> 1. FormatBundle config is not populated to Configuration in checkOutputSpecs:
> https://github.com/sisidra/crunch-ofb/blob/master/src/main/java/com/spotify/crunch/bugreport/MyOutputFormat.java#L39
> {code}
> 15/03/02 15:40:24 INFO bugreport.MyOutputFormat: my.config.key (checkOutputSpecs): null
> 15/03/02 15:40:24 ERROR bugreport.MyOutputFormat: Wrong my.config.key value in checkOutputSpecs!
> {code}
> 2. TaskAttemptContext. getTaskAttemptID().toString() is different in getRecordWriter and getOutputCommitter:
> {code}
> 2015-03-02 15:40:38,960 INFO [main] com.spotify.crunch.bugreport.MyOutputFormat: TaskAttemptID (getOutputCommitter): attempt_1422406067005_0121_m_000000_0
> ...
> 2015-03-02 15:40:39,789 INFO [main] com.spotify.crunch.bugreport.MyOutputFormat: TaskAttemptID (getRecordWriter): attempt_1422406067005_out0_0121_m_000000_0
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)