You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Sambit Sahoo (JIRA)" <ji...@apache.org> on 2013/04/10 14:16:15 UTC
[jira] [Updated] (PIG-3272) While running a pig job,showing delay
after Snappy native library get loaded
[ https://issues.apache.org/jira/browse/PIG-3272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sambit Sahoo updated PIG-3272:
------------------------------
Summary: While running a pig job,showing delay after Snappy native library get loaded (was: While running a pig job,showing delay after )
> While running a pig job,showing delay after Snappy native library get loaded
> ----------------------------------------------------------------------------
>
> Key: PIG-3272
> URL: https://issues.apache.org/jira/browse/PIG-3272
> Project: Pig
> Issue Type: Bug
> Components: data
> Affects Versions: 0.10.1
> Reporter: Sambit Sahoo
>
> 2013-04-09 06:29:42,639 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=MAP, sessionId=
> 2013-04-09 06:29:43,947 INFO org.apache.hadoop.mapred.MapTask: io.sort.mb = 1024
> 2013-04-09 06:29:44,434 INFO org.apache.hadoop.mapred.MapTask: data buffer = 816043776/1020054736
> 2013-04-09 06:29:44,435 INFO org.apache.hadoop.mapred.MapTask: record buffer = 2684354/3355443
> 2013-04-09 06:29:44,448 WARN org.apache.hadoop.io.compress.snappy.LoadSnappy: Snappy native library is available
> 2013-04-09 06:29:44,448 INFO org.apache.hadoop.io.compress.snappy.LoadSnappy: Snappy native library loaded
> 2013-04-09 06:48:29,783 INFO org.apache.hadoop.mapred.MapTask: Spilling map output: record full = true
> 2013-04-09 06:48:29,783 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufend = 423372603; bufvoid = 1020054736
> 2013-04-09 06:48:29,783 INFO org.apache.hadoop.mapred.MapTask: kvstart = 0; kvend = 2684354; length = 3355443
> 2013-04-09 06:49:32,969 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor
> In the above log you will see, after Snappy native library loaded it took around 20mins to proceed.
> Can anyone tell me the issue and the resolution?
> is it due to some network issue?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira