You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gora.apache.org by "Keith Turner (JIRA)" <ji...@apache.org> on 2012/05/11 16:46:56 UTC
[jira] [Created] (GORA-130) gora-accumulo caches tablet locations
between map reduce jobs
Keith Turner created GORA-130:
---------------------------------
Summary: gora-accumulo caches tablet locations between map reduce jobs
Key: GORA-130
URL: https://issues.apache.org/jira/browse/GORA-130
Project: Apache Gora
Issue Type: Bug
Components: storage-accumulo
Affects Versions: 0.2
Reporter: Keith Turner
Fix For: 0.3
Enis added a new Loop program to goraci that continually runs Generation and Verification map reduce jobs. So you have one process launching multiple map reduce jobs. I was running this I noticed an issue. After the first round of generation, the table had 16 tablets. So verification ran with 16 mappers, one per tablet. Then more data was inserted and the table split to 32 tablets. When verification ran it started 16 mappers instead of 32. Turns out the gora-accumulo store was using stale cached information about the table to create the input splits for the map reduce job.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (GORA-130) gora-accumulo caches tablet locations
between map reduce jobs
Posted by "Keith Turner (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Keith Turner resolved GORA-130.
-------------------------------
Resolution: Fixed
> gora-accumulo caches tablet locations between map reduce jobs
> --------------------------------------------------------------
>
> Key: GORA-130
> URL: https://issues.apache.org/jira/browse/GORA-130
> Project: Apache Gora
> Issue Type: Bug
> Components: storage-accumulo
> Affects Versions: 0.2
> Reporter: Keith Turner
> Assignee: Keith Turner
> Fix For: 0.3
>
>
> Enis added a new Loop program to goraci that continually runs Generation and Verification map reduce jobs. So you have one process launching multiple map reduce jobs. I was running this I noticed an issue. After the first round of generation, the table had 16 tablets. So verification ran with 16 mappers, one per tablet. Then more data was inserted and the table split to 32 tablets. When verification ran again it started 16 mappers instead of 32. Turns out the gora-accumulo store was using stale cached information about the table to create the input splits for the map reduce job.
> This issues will not affect the simple usage pattern of a single java process launching one map reduce job that reads from accumulo.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (GORA-130) gora-accumulo caches tablet locations
between map reduce jobs
Posted by "Keith Turner (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Keith Turner reassigned GORA-130:
---------------------------------
Assignee: Keith Turner
> gora-accumulo caches tablet locations between map reduce jobs
> --------------------------------------------------------------
>
> Key: GORA-130
> URL: https://issues.apache.org/jira/browse/GORA-130
> Project: Apache Gora
> Issue Type: Bug
> Components: storage-accumulo
> Affects Versions: 0.2
> Reporter: Keith Turner
> Assignee: Keith Turner
> Fix For: 0.3
>
>
> Enis added a new Loop program to goraci that continually runs Generation and Verification map reduce jobs. So you have one process launching multiple map reduce jobs. I was running this I noticed an issue. After the first round of generation, the table had 16 tablets. So verification ran with 16 mappers, one per tablet. Then more data was inserted and the table split to 32 tablets. When verification ran again it started 16 mappers instead of 32. Turns out the gora-accumulo store was using stale cached information about the table to create the input splits for the map reduce job.
> This issues will not affect the simple usage pattern of a single java process launching one map reduce job that reads from accumulo.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GORA-130) gora-accumulo caches tablet locations
between map reduce jobs
Posted by "Lewis John McGibbney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lewis John McGibbney updated GORA-130:
--------------------------------------
Fix Version/s: (was: 0.3)
0.2.1
> gora-accumulo caches tablet locations between map reduce jobs
> --------------------------------------------------------------
>
> Key: GORA-130
> URL: https://issues.apache.org/jira/browse/GORA-130
> Project: Apache Gora
> Issue Type: Bug
> Components: storage-accumulo
> Affects Versions: 0.2
> Reporter: Keith Turner
> Assignee: Keith Turner
> Fix For: 0.2.1
>
>
> Enis added a new Loop program to goraci that continually runs Generation and Verification map reduce jobs. So you have one process launching multiple map reduce jobs. I was running this I noticed an issue. After the first round of generation, the table had 16 tablets. So verification ran with 16 mappers, one per tablet. Then more data was inserted and the table split to 32 tablets. When verification ran again it started 16 mappers instead of 32. Turns out the gora-accumulo store was using stale cached information about the table to create the input splits for the map reduce job.
> This issues will not affect the simple usage pattern of a single java process launching one map reduce job that reads from accumulo.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GORA-130) gora-accumulo caches tablet locations
between map reduce jobs
Posted by "Keith Turner (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Keith Turner updated GORA-130:
------------------------------
Description:
Enis added a new Loop program to goraci that continually runs Generation and Verification map reduce jobs. So you have one process launching multiple map reduce jobs. I was running this I noticed an issue. After the first round of generation, the table had 16 tablets. So verification ran with 16 mappers, one per tablet. Then more data was inserted and the table split to 32 tablets. When verification ran again it started 16 mappers instead of 32. Turns out the gora-accumulo store was using stale cached information about the table to create the input splits for the map reduce job.
This issues will not affect the simple usage pattern of a single java process launching one map reduce job that reads from accumulo.
was:Enis added a new Loop program to goraci that continually runs Generation and Verification map reduce jobs. So you have one process launching multiple map reduce jobs. I was running this I noticed an issue. After the first round of generation, the table had 16 tablets. So verification ran with 16 mappers, one per tablet. Then more data was inserted and the table split to 32 tablets. When verification ran it started 16 mappers instead of 32. Turns out the gora-accumulo store was using stale cached information about the table to create the input splits for the map reduce job.
> gora-accumulo caches tablet locations between map reduce jobs
> --------------------------------------------------------------
>
> Key: GORA-130
> URL: https://issues.apache.org/jira/browse/GORA-130
> Project: Apache Gora
> Issue Type: Bug
> Components: storage-accumulo
> Affects Versions: 0.2
> Reporter: Keith Turner
> Fix For: 0.3
>
>
> Enis added a new Loop program to goraci that continually runs Generation and Verification map reduce jobs. So you have one process launching multiple map reduce jobs. I was running this I noticed an issue. After the first round of generation, the table had 16 tablets. So verification ran with 16 mappers, one per tablet. Then more data was inserted and the table split to 32 tablets. When verification ran again it started 16 mappers instead of 32. Turns out the gora-accumulo store was using stale cached information about the table to create the input splits for the map reduce job.
> This issues will not affect the simple usage pattern of a single java process launching one map reduce job that reads from accumulo.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (GORA-130) gora-accumulo caches tablet locations
between map reduce jobs
Posted by "Lewis John McGibbney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lewis John McGibbney closed GORA-130.
-------------------------------------
> gora-accumulo caches tablet locations between map reduce jobs
> --------------------------------------------------------------
>
> Key: GORA-130
> URL: https://issues.apache.org/jira/browse/GORA-130
> Project: Apache Gora
> Issue Type: Bug
> Components: storage-accumulo
> Affects Versions: 0.2
> Reporter: Keith Turner
> Assignee: Keith Turner
> Fix For: 0.2.1
>
>
> Enis added a new Loop program to goraci that continually runs Generation and Verification map reduce jobs. So you have one process launching multiple map reduce jobs. I was running this I noticed an issue. After the first round of generation, the table had 16 tablets. So verification ran with 16 mappers, one per tablet. Then more data was inserted and the table split to 32 tablets. When verification ran again it started 16 mappers instead of 32. Turns out the gora-accumulo store was using stale cached information about the table to create the input splits for the map reduce job.
> This issues will not affect the simple usage pattern of a single java process launching one map reduce job that reads from accumulo.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-130) gora-accumulo caches tablet locations
between map reduce jobs
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13273332#comment-13273332 ]
Hudson commented on GORA-130:
-----------------------------
Integrated in gora-trunk #268 (See [https://builds.apache.org/job/gora-trunk/268/])
GORA-130 cleare tablet location cache before computing partition queries for accumulo (Revision 1337191)
Result = UNSTABLE
kturner :
Files :
* /gora/trunk/gora-accumulo/src/main/java/org/apache/gora/accumulo/store/AccumuloStore.java
> gora-accumulo caches tablet locations between map reduce jobs
> --------------------------------------------------------------
>
> Key: GORA-130
> URL: https://issues.apache.org/jira/browse/GORA-130
> Project: Apache Gora
> Issue Type: Bug
> Components: storage-accumulo
> Affects Versions: 0.2
> Reporter: Keith Turner
> Assignee: Keith Turner
> Fix For: 0.3
>
>
> Enis added a new Loop program to goraci that continually runs Generation and Verification map reduce jobs. So you have one process launching multiple map reduce jobs. I was running this I noticed an issue. After the first round of generation, the table had 16 tablets. So verification ran with 16 mappers, one per tablet. Then more data was inserted and the table split to 32 tablets. When verification ran again it started 16 mappers instead of 32. Turns out the gora-accumulo store was using stale cached information about the table to create the input splits for the map reduce job.
> This issues will not affect the simple usage pattern of a single java process launching one map reduce job that reads from accumulo.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira