You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@accumulo.apache.org by "Keith Turner (Resolved) (JIRA)" <ji...@apache.org> on 2012/02/17 20:44:57 UTC
[jira] [Resolved] (ACCUMULO-395) Map reduce reading from Accumulo
is not running mapper locally
[ https://issues.apache.org/jira/browse/ACCUMULO-395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Keith Turner resolved ACCUMULO-395.
-----------------------------------
Resolution: Fixed
> Map reduce reading from Accumulo is not running mapper locally
> ---------------------------------------------------------------
>
> Key: ACCUMULO-395
> URL: https://issues.apache.org/jira/browse/ACCUMULO-395
> Project: Accumulo
> Issue Type: Bug
> Affects Versions: 1.3.5
> Reporter: Keith Turner
> Assignee: Keith Turner
> Priority: Critical
> Labels: 14_qa_bug
> Fix For: 1.4.0
>
>
> Noticed the continuous ingest verification map reduce job was running slow. Looking into I noticed that scans were not running locally. Not sure if this is map reduce issue or AccumuloInputFormat issue. Not only are the scans not local, sometimes a single tablet server will get a lot scans from many mappers overwhelming it.
> {noformat}
> $ accumulo shell -u root -p xxxx -e "listscans -np"
> TABLET SERVER | CLIENT | AGE | LAST | STATE | TYPE | USER | TABLE | COLUMNS | TABLET | ITERATORS | ITERATOR OPTIONS
> xxx.xxx.xxx.8:9997 | xxx.xxx.xxx.4:59715 | 2m42s | 8ms | IDLE |SINGLE | root | ci | [] |6;23a0cc;2380ca | [] | {}
> xxx.xxx.xxx.8:9997 | xxx.xxx.xxx.12:49054 | 5m3s | 7ms | IDLE |SINGLE | root | ci | [] |6;1ea0a4c;1e80ac | [] | {}
> xxx.xxx.xxx.8:9997 | xxx.xxx.xxx.9:57157 | 2m27s | 3ms |RUNNING |SINGLE | root | ci | [] |6;3a00b;39e0bb | [] | {}
> xxx.xxx.xxx.8:9997 | xxx.xxx.xxx.10:47880 | 2m58s | 3ms |RUNNING |SINGLE | root | ci | [] |6;0aa0ab;0a80aa | [] | {}
> xxx.xxx.xxx.8:9997 | xxx.xxx.xxx.9:57101 | 3m12s | 157ms |RUNNING |SINGLE | root | ci | [] |6;04a05f;04805a | [] | {}
> xxx.xxx.xxx.8:9997 | xxx.xxx.xxx.13:34053 | 6m20s | 180ms | IDLE |SINGLE | root | ci | [] |6;34a0cd;3480bd | [] | {}
> xxx.xxx.xxx.8:9997 | xxx.xxx.xxx.10:47862 | 3m18s | 48ms | IDLE |SINGLE | root | ci | [] |6;3ea08a;3e808f | [] | {}
> xxx.xxx.xxx.8:9997 | xxx.xxx.xxx.7:42593 | 2m35s | 2ms |RUNNING |SINGLE | root | ci | [] |6;46a0c5;4680c5 | [] | {}
> xxx.xxx.xxx.13:9997 | xxx.xxx.xxx.5:34764 | 2m11s | 177ms | IDLE |SINGLE | root | ci | [] |6;78007;77e07c | [] | {}
> xxx.xxx.xxx.13:9997 | xxx.xxx.xxx.8:43458 | 1m51s | 139ms | IDLE |SINGLE | root | ci | [] |6;3f803b;3f604e | [] | {}
> xxx.xxx.xxx.13:9997 | xxx.xxx.xxx.4:40674 | 3m0s | 1ms |RUNNING |SINGLE | root | ci | [] |6;0ea08b;0e808f | [] | {}
> xxx.xxx.xxx.13:9997 | xxx.xxx.xxx.13:49562 | 2m31s | 111ms |RUNNING |SINGLE | root | ci | [] |6;0ba086;0b8086 | [] | {}
> xxx.xxx.xxx.13:9997 | xxx.xxx.xxx.8:43274 | 2m38s | 35ms | IDLE |SINGLE | root | ci | [] |6;57a024;578025 | [] | {}
> xxx.xxx.xxx.13:9997 | xxx.xxx.xxx.6:39526 | 5m11s | 133ms | IDLE |SINGLE | root | ci | [] |6;16a0c9;1680cc | [] | {}
> xxx.xxx.xxx.13:9997 | xxx.xxx.xxx.9:40769 | 5m0s | 11ms | IDLE |SINGLE | root | ci | [] |6;3fa034;3f803b | [] | {}
> xxx.xxx.xxx.13:9997 | xxx.xxx.xxx.6:60691 | 2m59s | 11ms | IDLE |SINGLE | root | ci | [] |6;1ba0dbe;1b80da | [] | {}
> xxx.xxx.xxx.13:9997 | xxx.xxx.xxx.10:35987 | 5m4s | 31ms | IDLE |SINGLE | root | ci | [] |6;72a066;72806a | [] | {}
> xxx.xxx.xxx.13:9997 | xxx.xxx.xxx.9:40839 | 3m18s | 8ms | IDLE |SINGLE | root | ci | [] |6;71a05e;71805b | [] | {}
> xxx.xxx.xxx.13:9997 | xxx.xxx.xxx.8:43189 | 4m36s | 135ms | IDLE |SINGLE | root | ci | [] |6;75a07d;758075 | [] | {}
> xxx.xxx.xxx.13:9997 | xxx.xxx.xxx.12:45476 | 2m5s | 112ms | IDLE |SINGLE | root | ci | [] |6;48007;47e08 | [] | {}
> xxx.xxx.xxx.13:9997 | xxx.xxx.xxx.7:33791 | 4s981ms | 85ms |RUNNING |SINGLE | root | ci | [] |6;5f8036;5f6044c | [] | {}
> xxx.xxx.xxx.10:9997 | xxx.xxx.xxx.6:38304 | 2m38s | 18ms | IDLE |SINGLE | root | ci | [] |6;7ea059;7e8058 | [] | {}
> xxx.xxx.xxx.10:9997 | xxx.xxx.xxx.5:44152 | 2m3s | 7ms | IDLE |SINGLE | root | ci | [] |6;6e809f;6e60a8b | [] | {}
> xxx.xxx.xxx.10:9997 | xxx.xxx.xxx.5:44058 | 2m23s | 7ms | IDLE |SINGLE | root | ci | [] |6;7a00d;79e0d7f | [] | {}
> xxx.xxx.xxx.10:9997 | xxx.xxx.xxx.13:52463 | 2m35s | 6ms | IDLE |SINGLE | root | ci | [] |6;77a089;778089 | [] | {}
> xxx.xxx.xxx.10:9997 | xxx.xxx.xxx.5:44086 | 2m19s | 1ms |RUNNING |SINGLE | root | ci | [] |6;4a00c;49e0c3 | [] | {}
> xxx.xxx.xxx.7:9997 | xxx.xxx.xxx.13:43610 | 2m12s | 179ms | IDLE |SINGLE | root | ci | [] |6;3800b;37e0bf | [] | {}
> xxx.xxx.xxx.7:9997 | xxx.xxx.xxx.13:55025 | 4m59s | 152ms | IDLE |SINGLE | root | ci | [] |6;2ba1568;2b815c | [] | {}
> xxx.xxx.xxx.7:9997 | xxx.xxx.xxx.6:44613 | 1s520ms | 518ms |RUNNING |SINGLE | root | ci | [] |6;0f802f;0f6047 | [] | {}
> xxx.xxx.xxx.7:9997 | xxx.xxx.xxx.9:33954 | 3m23s | 3ms |RUNNING |SINGLE | root | ci | [] |6;4da0b4;4d80b9 | [] | {}
> xxx.xxx.xxx.6:9997 | xxx.xxx.xxx.13:57942 | 2m18s | 224ms | IDLE |SINGLE | root | ci | [] |6;0a00a;09e0a2 | [] | {}
> xxx.xxx.xxx.6:9997 | xxx.xxx.xxx.13:58004 | 2m9s | 164ms | IDLE |SINGLE | root | ci | [] |6;68016;67e16c | [] | {}
> xxx.xxx.xxx.6:9997 | xxx.xxx.xxx.10:57480 | 3m3s | 80ms |RUNNING |SINGLE | root | ci | [] |6;17a0bc;1780bf | [] | {}
> xxx.xxx.xxx.6:9997 | xxx.xxx.xxx.12:59552 | 1m59s | 145ms | IDLE |SINGLE | root | ci | [] |6;1880d8;1860cc | [] | {}
> xxx.xxx.xxx.6:9997 | xxx.xxx.xxx.5:47196 | 2m29s | 15ms | IDLE |SINGLE | root | ci | [] |6;1a00d;19e0dd | [] | {}
> xxx.xxx.xxx.6:9997 | xxx.xxx.xxx.13:58101 | 1m39s | 43ms | IDLE |SINGLE | root | ci | [] |6;4f8035;4f6045 | [] | {}
> xxx.xxx.xxx.6:9997 | xxx.xxx.xxx.4:36684 | 3m3s | 97ms | IDLE |SINGLE | root | ci | [] |6;1ca0df;1c80dc | [] | {}
> xxx.xxx.xxx.6:9997 | xxx.xxx.xxx.9:45627 | 3m15s | 4ms |RUNNING |SINGLE | root | ci | [] |6;07a022;078025 | [] | {}
> xxx.xxx.xxx.5:9997 | xxx.xxx.xxx.6:37328 | 4m35s | 93ms | IDLE |SINGLE | root | ci | [] |6;2da0e5;2d80ed | [] | {}
> xxx.xxx.xxx.5:9997 | xxx.xxx.xxx.4:46787 | 4m24s | 204ms | IDLE |SINGLE | root | ci | [] |6;39a0c7d;3980c8 | [] | {}
> xxx.xxx.xxx.5:9997 | xxx.xxx.xxx.12:39133 | 9m18s | 451ms | IDLE |SINGLE | root | ci | [] |6;25a147;25813c | [] | {}
> xxx.xxx.xxx.5:9997 | xxx.xxx.xxx.4:46827 | 3m21s | 3ms |RUNNING |SINGLE | root | ci | [] |6;52a069;528062 | [] | {}
> xxx.xxx.xxx.5:9997 | xxx.xxx.xxx.8:49713 | 2m29s | 4ms | IDLE |SINGLE | root | ci | [] |6;2aa18b;2a818b | [] | {}
> xxx.xxx.xxx.12:9997 | xxx.xxx.xxx.12:55427 | 2m39s | 1ms |RUNNING |SINGLE | root | ci | [] |6;45a0c6;4580c7 | [] | {}
> xxx.xxx.xxx.12:9997 | xxx.xxx.xxx.9:40025 | 6m6s | 66ms | IDLE |SINGLE | root | ci | [] |6;44a0bf;4480b9 | [] | {}
> xxx.xxx.xxx.12:9997 | xxx.xxx.xxx.7:46896 | 4m48s | 5ms | IDLE |SINGLE | root | ci | [] |6;6ea092;6e809f | [] | {}
> xxx.xxx.xxx.12:9997 | xxx.xxx.xxx.5:59397 | 2m26s | 189ms | IDLE |SINGLE | root | ci | [] |6;5a004;59e048d | [] | {}
> xxx.xxx.xxx.12:9997 | xxx.xxx.xxx.8:47096 | 4m33s | 5ms | IDLE |SINGLE | root | ci | [] |6;6ca11f;6c8117 | [] | {}
> xxx.xxx.xxx.4:9997 | xxx.xxx.xxx.6:47355 | 3m23s | 8ms | IDLE |SINGLE | root | ci | [] |6;06a063;068068 | [] | {}
> xxx.xxx.xxx.4:9997 | xxx.xxx.xxx.12:56529 | 2m11s | 4ms |RUNNING |SINGLE | root | ci | [] |6;1800a;17e0ad | [] | {}
> xxx.xxx.xxx.4:9997 | xxx.xxx.xxx.12:56401 | 2m48s | 47ms | IDLE |SINGLE | root | ci | [] |6;15a0bf;1580bd | [] | {}
> xxx.xxx.xxx.4:9997 | xxx.xxx.xxx.9:44547 | 3m21s | 6ms | IDLE |SINGLE | root | ci | [] |6;5ea049;5e8043 | [] | {}
> xxx.xxx.xxx.4:9997 | xxx.xxx.xxx.8:58391 | 5m6s | 236ms | IDLE |SINGLE | root | ci | [] |6;28a19a;288196 | [] | {}
> xxx.xxx.xxx.9:9997 | xxx.xxx.xxx.8:60543 | 2m41s | 43ms | IDLE |SINGLE | root | ci | [] |6;53a069;538068 | [] | {}
> xxx.xxx.xxx.9:9997 | xxx.xxx.xxx.4:46148 | 3m6s | 235ms | IDLE |SINGLE | root | ci | [] |6;03a05c;03805e | [] | {}
> xxx.xxx.xxx.9:9997 | xxx.xxx.xxx.12:52594 | 2m8s | 319ms | IDLE |SINGLE | root | ci | [] |6;28019;27e198 | [] | {}
> xxx.xxx.xxx.9:9997 | xxx.xxx.xxx.7:51462 | 2m59s | 115ms | IDLE |SINGLE | root | ci | [] |6;19a0eb;1980ea | [] | {}
> xxx.xxx.xxx.9:9997 | xxx.xxx.xxx.5:48253 | 2m5s | 195ms | IDLE |SINGLE | root | ci | [] |6;6f803a;6f60496 | [] | {}
> xxx.xxx.xxx.9:9997 | xxx.xxx.xxx.10:43025 | 2m24s | 3ms | IDLE |SINGLE | root | ci | [] |6;2a018;29e188 | [] | {}
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira