You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Mich Talebzadeh <mi...@peridale.co.uk> on 2015/03/24 21:42:44 UTC
Analyze stats on 4 million rows table fails with exception Buffer underflow.
Hi all,
Trying to analyze stats on 4 million rows table imported from Oracle via
Sqoop and Hive
0: jdbc:hive2://rhes564:10010/default> analyze table tdash compute
statistics;
INFO : Hadoop job information for Stage-0: number of mappers: 121; number
of reducers: 0
INFO : 2015-03-24 20:34:25,651 Stage-0 map = 0%, reduce = 0%
INFO : 2015-03-24 20:34:40,945 Stage-0 map = 1%, reduce = 0%, Cumulative
CPU 10.1 sec
INFO : 2015-03-24 20:42:38,973 Stage-0 map = 98%, reduce = 0%, Cumulative
CPU 324.06 sec
INFO : 2015-03-24 20:42:43,061 Stage-0 map = 99%, reduce = 0%, Cumulative
CPU 325.23 sec
No rows affected (509.571 seconds)
INFO : 2015-03-24 20:42:47,150 Stage-0 map = 100%, reduce = 0%, Cumulative
CPU 327.3 sec
INFO : MapReduce Total cumulative CPU time: 5 minutes 27 seconds 300 msec
INFO : Ended Job = job_1426887201015_0012
INFO : [Warning] could not update stats.Failed with exception Buffer
underflow.
org.apache.hive.com.esotericsoftware.kryo.KryoException: Buffer underflow.
Appreciate any feedback on this
Thanks
Mich Talebzadeh
http://talebzadehmich.wordpress.com
Publications due shortly:
Creating in-memory Data Grid for Trading Systems with Oracle TimesTen and
Coherence Cache
NOTE: The information in this email is proprietary and confidential. This
message is for the designated recipient only, if you are not the intended
recipient, you should destroy it immediately. Any information in this
message shall not be understood as given or endorsed by Peridale Ltd, its
subsidiaries or their employees, unless expressly so stated. It is the
responsibility of the recipient to ensure that this email is virus free,
therefore neither Peridale Ltd, its subsidiaries nor their employees accept
any responsibility.
RE: Analyze stats on 4 million rows table fails with exception Buffer underflow.
Posted by Mich Talebzadeh <mi...@peridale.co.uk>.
Hm I ran it analyze stats again and it worked second timeL
MapReduce Total cumulative CPU time: 5 minutes 26 seconds 500 msec
Ended Job = job_1426887201015_0016
Table oraclehadoop.tdash stats: [numFiles=4, numRows=4000000,
totalSize=32568651117, rawDataSize=32564651117]
MapReduce Jobs Launched:
Stage-Stage-0: Map: 121 Cumulative CPU: 326.5 sec HDFS Read: 32570197317
HDFS Write: 10164 SUCCESS
Total MapReduce CPU Time Spent: 5 minutes 26 seconds 500 msec
OK
I wonder if we have some reliability issue here somewhere. You don't want
this to happen in prod!
Thanks
Mich Talebzadeh
http://talebzadehmich.wordpress.com
Publications due shortly:
Creating in-memory Data Grid for Trading Systems with Oracle TimesTen and
Coherence Cache
NOTE: The information in this email is proprietary and confidential. This
message is for the designated recipient only, if you are not the intended
recipient, you should destroy it immediately. Any information in this
message shall not be understood as given or endorsed by Peridale Ltd, its
subsidiaries or their employees, unless expressly so stated. It is the
responsibility of the recipient to ensure that this email is virus free,
therefore neither Peridale Ltd, its subsidiaries nor their employees accept
any responsibility.
From: Mich Talebzadeh [mailto:mich@peridale.co.uk]
Sent: 24 March 2015 20:43
To: user@hive.apache.org
Subject: Analyze stats on 4 million rows table fails with exception Buffer
underflow.
Hi all,
Trying to analyze stats on 4 million rows table imported from Oracle via
Sqoop and Hive
0: jdbc:hive2://rhes564:10010/default> analyze table tdash compute
statistics;
INFO : Hadoop job information for Stage-0: number of mappers: 121; number
of reducers: 0
INFO : 2015-03-24 20:34:25,651 Stage-0 map = 0%, reduce = 0%
INFO : 2015-03-24 20:34:40,945 Stage-0 map = 1%, reduce = 0%, Cumulative
CPU 10.1 sec
INFO : 2015-03-24 20:42:38,973 Stage-0 map = 98%, reduce = 0%, Cumulative
CPU 324.06 sec
INFO : 2015-03-24 20:42:43,061 Stage-0 map = 99%, reduce = 0%, Cumulative
CPU 325.23 sec
No rows affected (509.571 seconds)
INFO : 2015-03-24 20:42:47,150 Stage-0 map = 100%, reduce = 0%, Cumulative
CPU 327.3 sec
INFO : MapReduce Total cumulative CPU time: 5 minutes 27 seconds 300 msec
INFO : Ended Job = job_1426887201015_0012
INFO : [Warning] could not update stats.Failed with exception Buffer
underflow.
org.apache.hive.com.esotericsoftware.kryo.KryoException: Buffer underflow.
Appreciate any feedback on this
Thanks
Mich Talebzadeh
http://talebzadehmich.wordpress.com
Publications due shortly:
Creating in-memory Data Grid for Trading Systems with Oracle TimesTen and
Coherence Cache
NOTE: The information in this email is proprietary and confidential. This
message is for the designated recipient only, if you are not the intended
recipient, you should destroy it immediately. Any information in this
message shall not be understood as given or endorsed by Peridale Ltd, its
subsidiaries or their employees, unless expressly so stated. It is the
responsibility of the recipient to ensure that this email is virus free,
therefore neither Peridale Ltd, its subsidiaries nor their employees accept
any responsibility.