You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@accumulo.apache.org by "John Vines (Created) (JIRA)" <ji...@apache.org> on 2011/11/16 15:48:51 UTC
[jira] [Created] (ACCUMULO-149) Remove MyMapFile dependency in
Isolation Image
Remove MyMapFile dependency in Isolation Image
----------------------------------------------
Key: ACCUMULO-149
URL: https://issues.apache.org/jira/browse/ACCUMULO-149
Project: Accumulo
Issue Type: Improvement
Components: tserver
Reporter: John Vines
Assignee: Keith Turner
Fix For: 1.5.0
MyMapFile and MySequenceFile are my mortal enemies and I wish for nothing more than to purge them from our codebase. The biggest standing issue keeping us from removing them entirely and using the compatible MapFile and SequenceFile is the isolation image we use to ensure Scanner consistency. It writes a temp MyMapFile to scan against until all scanners for that timeframe are complete. Because of the extra information stored in the file, an RFile is not quite compatible. So we need to come up with a method to use RFile to get done what we need to get done.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (ACCUMULO-149) Remove MyMapFile dependency in
Isolation Image
Posted by "John Vines (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Vines resolved ACCUMULO-149.
---------------------------------
Resolution: Fixed
> Remove MyMapFile dependency in Isolation Image
> ----------------------------------------------
>
> Key: ACCUMULO-149
> URL: https://issues.apache.org/jira/browse/ACCUMULO-149
> Project: Accumulo
> Issue Type: Improvement
> Components: tserver
> Reporter: John Vines
> Assignee: John Vines
> Labels: cleanup
> Fix For: 1.5.0
>
>
> MyMapFile and MySequenceFile are my mortal enemies and I wish for nothing more than to purge them from our codebase. The biggest standing issue keeping us from removing them entirely and using the compatible MapFile and SequenceFile is the isolation image we use to ensure Scanner consistency. It writes a temp MyMapFile to scan against until all scanners for that timeframe are complete. Because of the extra information stored in the file, an RFile is not quite compatible. So we need to come up with a method to use RFile to get done what we need to get done.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-149) Remove MyMapFile dependency in
Isolation Image
Posted by "Keith Turner (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13151267#comment-13151267 ]
Keith Turner commented on ACCUMULO-149:
---------------------------------------
goodluck and godspeed on your quest young master vines
> Remove MyMapFile dependency in Isolation Image
> ----------------------------------------------
>
> Key: ACCUMULO-149
> URL: https://issues.apache.org/jira/browse/ACCUMULO-149
> Project: Accumulo
> Issue Type: Improvement
> Components: tserver
> Reporter: John Vines
> Assignee: Keith Turner
> Labels: cleanup
> Fix For: 1.5.0
>
>
> MyMapFile and MySequenceFile are my mortal enemies and I wish for nothing more than to purge them from our codebase. The biggest standing issue keeping us from removing them entirely and using the compatible MapFile and SequenceFile is the isolation image we use to ensure Scanner consistency. It writes a temp MyMapFile to scan against until all scanners for that timeframe are complete. Because of the extra information stored in the file, an RFile is not quite compatible. So we need to come up with a method to use RFile to get done what we need to get done.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-149) Remove MyMapFile dependency in
Isolation Image
Posted by "Keith Turner (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175567#comment-13175567 ]
Keith Turner commented on ACCUMULO-149:
---------------------------------------
John, I think the changes broke seeking backwards in the in memory map. I was looking at the MemKeyConversionIterator.consume() method and was thinking it could prevent seeking backwards. I wrote the following test to confirm this. Add the following test to the InMemoryMapTest and it will fail.
{noformat}
public void testSeekBackWards() throws Exception {
InMemoryMap imm = new InMemoryMap(false, "/tmp");
mutate(imm, "r1", "foo:cq1", 3, "bar1");
mutate(imm, "r1", "foo:cq2", 3, "bar2");
mutate(imm, "r1", "foo:cq3", 3, "bar3");
mutate(imm, "r1", "foo:cq4", 3, "bar4");
MemoryIterator skvi1 = imm.skvIterator();
skvi1.seek(new Range(nk("r1", "foo:cq3", 3), null), LocalityGroupUtil.EMPTY_CF_SET, false);
ae(skvi1, "r1", "foo:cq3", 3, "bar3");
skvi1.seek(new Range(nk("r1", "foo:cq1", 3), null), LocalityGroupUtil.EMPTY_CF_SET, false);
ae(skvi1, "r1", "foo:cq1", 3, "bar1");
}
{noformat}
> Remove MyMapFile dependency in Isolation Image
> ----------------------------------------------
>
> Key: ACCUMULO-149
> URL: https://issues.apache.org/jira/browse/ACCUMULO-149
> Project: Accumulo
> Issue Type: Improvement
> Components: tserver
> Reporter: John Vines
> Assignee: John Vines
> Labels: cleanup
> Fix For: 1.5.0
>
>
> MyMapFile and MySequenceFile are my mortal enemies and I wish for nothing more than to purge them from our codebase. The biggest standing issue keeping us from removing them entirely and using the compatible MapFile and SequenceFile is the isolation image we use to ensure Scanner consistency. It writes a temp MyMapFile to scan against until all scanners for that timeframe are complete. Because of the extra information stored in the file, an RFile is not quite compatible. So we need to come up with a method to use RFile to get done what we need to get done.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-149) Remove MyMapFile dependency in
Isolation Image
Posted by "Keith Turner (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175576#comment-13175576 ]
Keith Turner commented on ACCUMULO-149:
---------------------------------------
I would propose putting MemKeyConversionIterator below the source switching iterator and changing its behavior. Remove the consume behavior and make it have a seek that handles MemKeys. The SourceSwitching iterator will reseek using a MemKey. To put it below the SourceSwitchingIterator make MemoryDataSource return it.
> Remove MyMapFile dependency in Isolation Image
> ----------------------------------------------
>
> Key: ACCUMULO-149
> URL: https://issues.apache.org/jira/browse/ACCUMULO-149
> Project: Accumulo
> Issue Type: Improvement
> Components: tserver
> Reporter: John Vines
> Assignee: John Vines
> Labels: cleanup
> Fix For: 1.5.0
>
>
> MyMapFile and MySequenceFile are my mortal enemies and I wish for nothing more than to purge them from our codebase. The biggest standing issue keeping us from removing them entirely and using the compatible MapFile and SequenceFile is the isolation image we use to ensure Scanner consistency. It writes a temp MyMapFile to scan against until all scanners for that timeframe are complete. Because of the extra information stored in the file, an RFile is not quite compatible. So we need to come up with a method to use RFile to get done what we need to get done.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (ACCUMULO-149) Remove MyMapFile dependency in
Isolation Image
Posted by "John Vines (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Vines reassigned ACCUMULO-149:
-----------------------------------
Assignee: John Vines (was: Keith Turner)
> Remove MyMapFile dependency in Isolation Image
> ----------------------------------------------
>
> Key: ACCUMULO-149
> URL: https://issues.apache.org/jira/browse/ACCUMULO-149
> Project: Accumulo
> Issue Type: Improvement
> Components: tserver
> Reporter: John Vines
> Assignee: John Vines
> Labels: cleanup
> Fix For: 1.5.0
>
>
> MyMapFile and MySequenceFile are my mortal enemies and I wish for nothing more than to purge them from our codebase. The biggest standing issue keeping us from removing them entirely and using the compatible MapFile and SequenceFile is the isolation image we use to ensure Scanner consistency. It writes a temp MyMapFile to scan against until all scanners for that timeframe are complete. Because of the extra information stored in the file, an RFile is not quite compatible. So we need to come up with a method to use RFile to get done what we need to get done.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira