You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@crunch.apache.org by "Josh Wills (JIRA)" <ji...@apache.org> on 2014/09/24 23:51:34 UTC
[jira] [Resolved] (CRUNCH-471) Add synchronization checks to
DoFnIterator
[ https://issues.apache.org/jira/browse/CRUNCH-471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Josh Wills resolved CRUNCH-471.
-------------------------------
Resolution: Fixed
Pushed to master.
> Add synchronization checks to DoFnIterator
> ------------------------------------------
>
> Key: CRUNCH-471
> URL: https://issues.apache.org/jira/browse/CRUNCH-471
> Project: Crunch
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.11.0
> Reporter: Josh Wills
> Assignee: Josh Wills
> Fix For: 0.12.0
>
> Attachments: CRUNCH-471.patch
>
>
> I hit this one when I was working on a port of the Hadoop Streaming logic to Crunch: if a DoFn emits records via some sort of asynchronous process (e.g., a daemon thread), it can cause a problem for DoFnIterator, which doesn't have thread-safe controls over its underlying records list. Easy fix is to make the accessor methods for CacheEmitter inside of DoFnIterator synchronized.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)