You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@phoenix.apache.org by "James Taylor (JIRA)" <ji...@apache.org> on 2016/01/23 22:58:39 UTC
[jira] [Updated] (PHOENIX-2083) Pig maps splits are very unevent
[ https://issues.apache.org/jira/browse/PHOENIX-2083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
James Taylor updated PHOENIX-2083:
----------------------------------
Fix Version/s: 4.8.0
> Pig maps splits are very unevent
> --------------------------------
>
> Key: PHOENIX-2083
> URL: https://issues.apache.org/jira/browse/PHOENIX-2083
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 4.1.0
> Reporter: Brian Johnson
> Fix For: 4.8.0
>
>
> When running a pig job on MR with the Phoenix loader we got about 75 maps tasks, but there was huge amount of skew in how the records were allocated and the vast majority of them went to about 20 mappers and 5 got nothing at all.
> Task
> Value
> task_1433431098673_66646_m_000042 0
> task_1433431098673_66646_m_000057 0
> task_1433431098673_66646_m_000061 0
> task_1433431098673_66646_m_000067 0
> task_1433431098673_66646_r_000000 0
> task_1433431098673_66646_m_000031 127242
> task_1433431098673_66646_m_000026 130669
> task_1433431098673_66646_m_000017 179685
> task_1433431098673_66646_m_000068 190741
> task_1433431098673_66646_m_000040 191062
> task_1433431098673_66646_m_000056 191509
> task_1433431098673_66646_m_000053 191518
> task_1433431098673_66646_m_000060 191560
> task_1433431098673_66646_m_000048 191579
> task_1433431098673_66646_m_000041 191623
> task_1433431098673_66646_m_000047 191686
> task_1433431098673_66646_m_000065 191720
> task_1433431098673_66646_m_000064 191726
> task_1433431098673_66646_m_000054 191763
> task_1433431098673_66646_m_000066 191871
> task_1433431098673_66646_m_000052 191875
> task_1433431098673_66646_m_000045 191908
> task_1433431098673_66646_m_000049 191914
> task_1433431098673_66646_m_000063 192124
> task_1433431098673_66646_m_000058 192352
> task_1433431098673_66646_m_000069 192352
> task_1433431098673_66646_m_000044 192519
> task_1433431098673_66646_m_000007 529769
> task_1433431098673_66646_m_000018 584940
> task_1433431098673_66646_m_000005 585864
> task_1433431098673_66646_m_000003 697683
> task_1433431098673_66646_m_000016 709321
> task_1433431098673_66646_m_000008 710190
> task_1433431098673_66646_m_000004 710774
> task_1433431098673_66646_m_000011 711818
> task_1433431098673_66646_m_000038 713862
> task_1433431098673_66646_m_000037 714577
> task_1433431098673_66646_m_000022 716796
> task_1433431098673_66646_m_000014 717478
> task_1433431098673_66646_m_000025 722809
> task_1433431098673_66646_m_000030 723182
> task_1433431098673_66646_m_000024 723378
> task_1433431098673_66646_m_000013 731836
> task_1433431098673_66646_m_000010 732525
> task_1433431098673_66646_m_000001 734611
> task_1433431098673_66646_m_000036 739874
> task_1433431098673_66646_m_000072 1810925
> task_1433431098673_66646_m_000039 1923212
> task_1433431098673_66646_m_000059 2014210
> task_1433431098673_66646_m_000055 2287499
> task_1433431098673_66646_m_000074 2887750
> task_1433431098673_66646_m_000073 3049942
> task_1433431098673_66646_m_000029 3156535
> task_1433431098673_66646_m_000071 3841375
> task_1433431098673_66646_m_000027 4001882
> task_1433431098673_66646_m_000051 4343619
> task_1433431098673_66646_m_000034 5363718
> task_1433431098673_66646_m_000050 7734798
> task_1433431098673_66646_m_000020 9543930
> task_1433431098673_66646_m_000070 10058382
> task_1433431098673_66646_m_000046 10143291
> task_1433431098673_66646_m_000062 10263757
> task_1433431098673_66646_m_000032 10908072
> task_1433431098673_66646_m_000015 11182800
> task_1433431098673_66646_m_000000 11300385
> task_1433431098673_66646_m_000043 11359327
> task_1433431098673_66646_m_000021 12632598
> task_1433431098673_66646_m_000009 14598258
> task_1433431098673_66646_m_000028 14698359
> task_1433431098673_66646_m_000033 16407474
> task_1433431098673_66646_m_000012 17944269
> task_1433431098673_66646_m_000023 20568188
> task_1433431098673_66646_m_000035 21656353
> task_1433431098673_66646_m_000002 27413291
> task_1433431098673_66646_m_000006 35573698
> task_1433431098673_66646_m_000019 35717128
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)