You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gora.apache.org by "Hudson (JIRA)" <ji...@apache.org> on 2013/07/01 07:06:34 UTC
[jira] [Commented] (GORA-130) gora-accumulo caches tablet locations
between map reduce jobs
[ https://issues.apache.org/jira/browse/GORA-130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13696492#comment-13696492 ]
Hudson commented on GORA-130:
-----------------------------
Integrated in goraOracle #6 (See [https://builds.apache.org/job/goraOracle/6/])
GORA-130 cleare tablet location cache before computing partition queries for accumulo (Revision dfa409fc04425e0b86d55a2a62afb43c0f299baa)
Result = FAILURE
kturner :
Files :
* gora-accumulo/src/main/java/org/apache/gora/accumulo/store/AccumuloStore.java
> gora-accumulo caches tablet locations between map reduce jobs
> --------------------------------------------------------------
>
> Key: GORA-130
> URL: https://issues.apache.org/jira/browse/GORA-130
> Project: Apache Gora
> Issue Type: Bug
> Components: gora-accumulo
> Affects Versions: 0.2
> Reporter: Keith Turner
> Assignee: Keith Turner
> Fix For: 0.2.1
>
>
> Enis added a new Loop program to goraci that continually runs Generation and Verification map reduce jobs. So you have one process launching multiple map reduce jobs. I was running this I noticed an issue. After the first round of generation, the table had 16 tablets. So verification ran with 16 mappers, one per tablet. Then more data was inserted and the table split to 32 tablets. When verification ran again it started 16 mappers instead of 32. Turns out the gora-accumulo store was using stale cached information about the table to create the input splits for the map reduce job.
> This issues will not affect the simple usage pattern of a single java process launching one map reduce job that reads from accumulo.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira