You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@accumulo.apache.org by "Keith Turner (Issue Comment Edited) (JIRA)" <ji...@apache.org> on 2012/02/18 00:14:57 UTC
[jira] [Issue Comment Edited] (ACCUMULO-387) Support map reduce
directly over files
[ https://issues.apache.org/jira/browse/ACCUMULO-387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210658#comment-13210658 ]
Keith Turner edited comment on ACCUMULO-387 at 2/17/12 11:13 PM:
-----------------------------------------------------------------
This input format could run against offline tables. It does not care if you clone or not, but it will only start if the table is offline. This is easy to achieve, just clone the table and take it offline. This is simpler than trying to adjust settings to disable compactions and writes, setting that may change over time.
One draw back with this approach is that the current code to take a table offline is async. It starts a table going offline, but does not wait for it to happen. The inputformat could probably get around this pretty easily. It could check that the table states is offline and then wait for there to be no locations in the metadata table. Once there are no locations it could start computing input splits.
was (Author: kturner):
This input format could run against offline tables. It does not care if you clone or not, but it will only start if the table is offline. This is easy to achieve, just clone the table and take it offline. This is simpler than trying to adjust settings to disable compactions and reads, setting that may change over time.
One draw back with this approach is that the current code to take a table offline is async. It starts a table going offline, but does not wait for it to happen. The inputformat could probably get around this pretty easily. It could check that the table states is offline and then wait for there to be no locations in the metadata table. Once there are no locations it could start computing input splits.
> Support map reduce directly over files
> --------------------------------------
>
> Key: ACCUMULO-387
> URL: https://issues.apache.org/jira/browse/ACCUMULO-387
> Project: Accumulo
> Issue Type: New Feature
> Reporter: Keith Turner
> Assignee: Eric Newton
> Fix For: 1.4.1
>
>
> Support map reduce jobs that directly read Accumulo files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira