You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by Vineet Mishra <cl...@gmail.com> on 2015/06/12 15:00:44 UTC
Jobs Running with Single Mapper/Reducer
Hi All,
I am building a cube using Kylin and I could see that the job is running
with Single Mapper and Reducer for some of the intermediate process such as
Extract Fact Table Distinct Columns
Build Dimension Dictionary
Build N-Dimension Cuboid
I am not sure what's the reason behind running the job with single M/R, is
it really necessary or is it some default config. which can be tweaked, its
70 Mins and the job status is 25% !
Urgent Call!
Thanks!
Re: Problem running kylin 0.7.1 with IBM Open Platform with Apache
Hadoop 4.0
Posted by "Shi, Shaofeng" <sh...@ebay.com>.
Hi Maria,
Firstly, just for your awareness, from 0.7, Kylin will seek the kylin.properties from $KYLIN_HOME/conf folder by default, so you don’t need set KYLIN_CONF in most of the case, unless you want to manage the config in another location then use KYLIN_CONF;
Regarding the error you got, it is a problem of kylin.sh script; please remove the last line from your kylin.properties and then restart Kylin:
#kylin.sandbox=false
Although this setting was commented, the script wasn’t able to check it, so it still got “false” on this property, then uses the LDAP profile; I will create a JIRA on this;
From: Gaspare Maria <ga...@gfmintegration.it>>
Reply-To: "dev@kylin.incubator.apache.org<ma...@kylin.incubator.apache.org>" <de...@kylin.incubator.apache.org>>
Date: Friday, June 12, 2015 at 11:50 PM
To: "dev@kylin.incubator.apache.org<ma...@kylin.incubator.apache.org>" <de...@kylin.incubator.apache.org>>
Subject: Problem running kylin 0.7.1 with IBM Open Platform with Apache Hadoop 4.0
Hi,
I am trying to test kylin by using as hadoop CLI "BigInsights IBM Open Platform with Apache Hadoop 4.0".
Environment:
* kylin-0.7.1-incubating-SNAPSHOT
* BigInsights IBM Open Platform with Apache Hadoop 4.0
* KYLIN_HOME=/opt/apache/kylin/kylin-0.7.1-incubating-SNAPSHOT ==> Directory where I put kylin
* $ java -version
java version "1.7.0_75"
OpenJDK Runtime Environment (rhel-2.5.4.0.el6_6-x86_64 u75-b13)
OpenJDK 64-Bit Server VM (build 24.75-b04, mixed mode)
NOTE: IBM Open Platform with Apache Hadoop 4.0 works with the openJDK. Anyway, I tried also with SUN JDK but result is the same
* KYLIN_CONF=/etc/kylin ==> I set also KYLIN_EXTRA_START_OPTS="-DKYLIN_CONF=/etc/kylin" end copied the cfg files also in /etc/kylin
* Both hadoop CLI and kylin run on the same server
Kylin dosen's start properly and I see the following log related to problems with ldap.
[localhost-startStop-1]:[2015-06-12 17:25:25,827][INFO][org.apache.kylin.common.KylinConfig.getKylinProperties(KylinConfig.java:544)] - Use KYLIN_CONF=/etc/kylin
[localhost-startStop-1]:[2015-06-12 17:25:25,863][INFO][org.springframework.core.io.support.PropertiesLoaderSupport.loadProperties(PropertiesLoaderSupport.java:177)] - Loading properties file from resource loaded through InputStream
[localhost-startStop-1]:[2015-06-12 17:25:25,905][ERROR][org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:307)] - Context initialization failed
org.springframework.beans.factory.BeanDefinitionStoreException: Invalid bean definition with name 'ldapSource' defined in class path resource [kylinSecurity.xml]: Input length must be multiple of 16 when decrypting with padded cipher
at org.springframework.beans.factory.config.PlaceholderConfigurerSupport.doProcessProperties(PlaceholderConfigurerSupport.java:209)
at org.springframework.beans.factory.config.PropertyPlaceholderConfigurer.processProperties(PropertyPlaceholderConfigurer.java:220)
at org.springframework.beans.factory.config.PropertyResourceConfigurer.postProcessBeanFactory(PropertyResourceConfigurer.java:84)
at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:686)
at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:661)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:451)
at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:383)
at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:283)
at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:111)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:5016)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5524)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:901)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:877)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:649)
at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:1081)
at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1877)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Jun 12, 2015 5:25:25 PM org.apache.catalina.core.StandardContext startInternal
SEVERE: Error listenerStart
Jun 12, 2015 5:25:25 PM org.apache.catalina.core.StandardContext startInternal
SEVERE: Context [/kylin] startup failed due to previous errors
Jun 12, 2015 5:25:25 PM org.apache.catalina.startup.HostConfig deployWAR
INFO: Deployment of web application archive /opt/apache/kylin/kylin-0.7.1-incubating-SNAPSHOT/tomcat/webapps/kylin.war has finished in 11,533 ms
Jun 12, 2015 5:25:25 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["http-bio-7070"]
Jun 12, 2015 5:25:25 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["ajp-bio-9009"]
Jun 12, 2015 5:25:25 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in 11574 ms
rceConfigurer.java:84)
19,1 48%
Is it possible to disable LDAP usage ?
Could you please provide some feedbacks ?
In attach the log file and the cfg files.
Thanks in advance.
BR,
-- gas
Problem running kylin 0.7.1 with IBM Open Platform with Apache Hadoop
4.0
Posted by Gaspare Maria <ga...@gfmintegration.it>.
Hi,
I am trying to test kylin by using as hadoop CLI "BigInsights IBM Open
Platform with Apache Hadoop 4.0".
Environment:
* kylin-0.7.1-incubating-SNAPSHOT
* BigInsights IBM Open Platform with Apache Hadoop 4.0
* KYLIN_HOME=/opt/apache/kylin/kylin-0.7.1-incubating-SNAPSHOT ==>
Directory where I put kylin
* $ java -version
java version "1.7.0_75"
OpenJDK Runtime Environment (rhel-2.5.4.0.el6_6-x86_64 u75-b13)
OpenJDK 64-Bit Server VM (build 24.75-b04, mixed mode)
NOTE: IBM Open Platform with Apache Hadoop 4.0 works with the
openJDK. Anyway, I tried also with SUN JDK but result is the same
* KYLIN_CONF=/etc/kylin ==> I set also
KYLIN_EXTRA_START_OPTS="-DKYLIN_CONF=/etc/kylin" end copied the cfg
files also in /etc/kylin
* Both hadoop CLI and kylin run on the same server
Kylin dosen's start properly and I see the following log related to
problems with ldap.
[localhost-startStop-1]:[2015-06-12
17:25:25,827][INFO][org.apache.kylin.common.KylinConfig.getKylinProperties(KylinConfig.java:544)]
- Use KYLIN_CONF=/etc/kylin
[localhost-startStop-1]:[2015-06-12
17:25:25,863][INFO][org.springframework.core.io.support.PropertiesLoaderSupport.loadProperties(PropertiesLoaderSupport.java:177)]
- Loading properties file from resource loaded through InputStream
[localhost-startStop-1]:[2015-06-12
17:25:25,905][ERROR][org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:307)]
- Context initialization failed
org.springframework.beans.factory.BeanDefinitionStoreException: Invalid
bean definition with name 'ldapSource' defined in class path resource
[kylinSecurity.xml]: Input length must be multiple of 16 when decrypting
with padded cipher
at
org.springframework.beans.factory.config.PlaceholderConfigurerSupport.doProcessProperties(PlaceholderConfigurerSupport.java:209)
at
org.springframework.beans.factory.config.PropertyPlaceholderConfigurer.processProperties(PropertyPlaceholderConfigurer.java:220)
at
org.springframework.beans.factory.config.PropertyResourceConfigurer.postProcessBeanFactory(PropertyResourceConfigurer.java:84)
at
org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:686)
at
org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:661)
at
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:451)
at
org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:383)
at
org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:283)
at
org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:111)
at
org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:5016)
at
org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5524)
at
org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:901)
at
org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:877)
at
org.apache.catalina.core.StandardHost.addChild(StandardHost.java:649)
at
org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:1081)
at
org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1877)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Jun 12, 2015 5:25:25 PM org.apache.catalina.core.StandardContext
startInternal
SEVERE: Error listenerStart
Jun 12, 2015 5:25:25 PM org.apache.catalina.core.StandardContext
startInternal
SEVERE: Context [/kylin] startup failed due to previous errors
Jun 12, 2015 5:25:25 PM org.apache.catalina.startup.HostConfig deployWAR
INFO: Deployment of web application archive
/opt/apache/kylin/kylin-0.7.1-incubating-SNAPSHOT/tomcat/webapps/kylin.war
has finished in 11,533 ms
Jun 12, 2015 5:25:25 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["http-bio-7070"]
Jun 12, 2015 5:25:25 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["ajp-bio-9009"]
Jun 12, 2015 5:25:25 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in 11574 ms
rceConfigurer.java:84)
19,1 48%
Is it possible to disable LDAP usage ?
Could you please provide some feedbacks ?
In attach the log file and the cfg files.
Thanks in advance.
BR,
-- gas
Re: Jobs Running with Single Mapper/Reducer
Posted by Vineet Mishra <cl...@gmail.com>.
Sure Shi! :)
On Tue, Jun 16, 2015 at 2:27 PM, ShaoFeng Shi <sh...@gmail.com> wrote:
> That's more clear, thank you; In theory the cube size is fixed on the same
> input, you must miss some change between cube1 and cube2 :-)
>
> Anyway, suggest you try more times, use the same cube, disable it, purge
> it, and then build again with the same date range; If the problem repeats,
> welcome you to open a JIRA to Kylin:
> https://issues.apache.org/jira/secure/Dashboard.jspa
>
> 2015-06-16 16:33 GMT+08:00 Vineet Mishra <cl...@gmail.com>:
>
> > Let me clarify,
> >
> > 1) So initially I had a cube say cube1 with some dimension and measures
> > which was of size 420Mb
> > 2) later I created another cube say cube2 with same dimension as cube1
> but
> > with added measures(so in cube1 it was 1 measure while in cube2 it was 8)
> > which was of size 130Mb(expected greater size due to addition of multiple
> > measure but it was lesser than the cube1)
> >
> > 3) thereafter I created another cube say cube3 with same properties as
> > cube2 and made the build and it was 120Mb(slightly lesser than the last
> > build with the same data source)
> >
> > So I was looking out as what's the reason as the cube size is varying and
> > moreover to my surprise the cube2 which is holding extra measures as in
> > comparison to cube1 is even lesser in size.
> >
> > I guess that makes things clear to you.
> >
> > Thanks,
> >
> > On Tue, Jun 16, 2015 at 1:25 PM, ShaoFeng Shi <sh...@gmail.com>
> > wrote:
> >
> > > I'm confused; You said these two cubes are exactly same in an early
> > reply,
> > > but now you mentioned the seoncd has more measures... they are NOT the
> > > same; Different measures takes different storage spaces; What kind of
> the
> > > measures defined in your former cube? Did you use DISTINCT COUNT
> measure?
> > >
> > > 2015-06-16 3:28 GMT+08:00 Vineet Mishra <cl...@gmail.com>:
> > >
> > > > Hi Shi,
> > > >
> > > > Well yes these cubes are build for the same date range without having
> > the
> > > > data changed on the table and moreover hbase compression was not even
> > > > touched so far.
> > > >
> > > > One interesting thing which I got was the later cube which was small
> in
> > > > size and completing in lesser time was having multiple
> measures(around
> > 7
> > > or
> > > > 8) in comparison to the former cube having just 1 or perhaps 2
> measure
> > > with
> > > > more time to complete.
> > > >
> > > > Running the same job again, it came up with some different size for
> the
> > > > same data, so last time it was reduced 130Mb now its even less,
> around
> > > > 123Mb. This change is observed within 4 hour of the running the last
> > job
> > > > with the same data set.
> > > >
> > > > I am not sure but how the kylin is internally building these cube
> > > dimension
> > > > so as to inflate the data size by 3 and more over when the measures
> are
> > > > very few as in comparison.
> > > >
> > > > Let me know if you could get anything out of it.
> > > >
> > > > Thanks!
> > > >
> > > > On Mon, Jun 15, 2015 at 8:51 PM, Shi, Shaofeng <sh...@ebay.com>
> > wrote:
> > > >
> > > > > Just guess: Are these two cube builds cross the same date range?
> Was
> > > the
> > > > > data on fact table changed between these two builds? Did HBase
> enable
> > > > > compression recently?
> > > > >
> > > > > You can build once more with the same cube and date range to see
> > > whether
> > > > > it can be re-produced; The cube size should be consistent in my
> > > > knowledge;
> > > > >
> > > > > On 6/15/15, 10:52 PM, "Vineet Mishra" <cl...@gmail.com>
> > wrote:
> > > > >
> > > > > >Thanks Shi,
> > > > > >
> > > > > >By the way I was curious as for the same table, dimension and
> > metric,
> > > I
> > > > > >could see the final cube size varying from 130 Mb to 420Mb.
> > > > > >
> > > > > >The cube is exactly the same as another one but when the cube
> build
> > > job
> > > > is
> > > > > >run I could see the cube size difference as mentioned below,
> > > > > >
> > > > > >*Earlier Cube Build*
> > > > > >Size - 420 Mb
> > > > > >Time Taken for building cube - 140 mins
> > > > > >Source Records - ~11Million
> > > > > >
> > > > > >*Today's Cube Build*
> > > > > >Size - 130 Mb
> > > > > >Time Taken for building cube - 22 mins
> > > > > >Source Records - ~11Million
> > > > > >
> > > > > >I don't find any reason to have three time cube size difference.
> Any
> > > > ideas
> > > > > >about this?
> > > > > >
> > > > > >Thanks!
> > > > > >
> > > > > >On Mon, Jun 15, 2015 at 8:08 PM, Shi, Shaofeng <sh...@ebay.com>
> > > > wrote:
> > > > > >
> > > > > >> Good catch, and thanks for the sharing;
> > > > > >>
> > > > > >> On 6/15/15, 10:20 PM, "Vineet Mishra" <cl...@gmail.com>
> > > wrote:
> > > > > >>
> > > > > >> >Hi All,
> > > > > >> >
> > > > > >> >Well I got it through.
> > > > > >> >
> > > > > >> >Actually it was basically the pre-process which was converting
> my
> > > 3Gb
> > > > > >>hive
> > > > > >> >table to 60Mb sequence file which due to block size of 128Mb
> was
> > > > > >>running
> > > > > >> >in
> > > > > >> >a single Mapper job.
> > > > > >> >
> > > > > >> >I changed the split size to 20Mb and it was able to spawn 3
> > Mapper
> > > > > >>after
> > > > > >> >that.
> > > > > >> >
> > > > > >> >Anyways, thanks all for your quick response.
> > > > > >> >
> > > > > >> >Thanks!
> > > > > >> >
> > > > > >> >On Mon, Jun 15, 2015 at 3:27 PM, Vineet Mishra <
> > > > clearmidoubt@gmail.com
> > > > > >
> > > > > >> >wrote:
> > > > > >> >
> > > > > >> >> Hi,
> > > > > >> >>
> > > > > >> >> Sorry I misunderstood this value, actually 30Mb was the
> output
> > > for
> > > > > >>that
> > > > > >> >> job. But I guess I have the got reason, (not exactly
> confirmed)
> > > as
> > > > > >>why
> > > > > >> >>its
> > > > > >> >> behaving like this, the input to this job is sequence file
> > which
> > > is
> > > > > >> >>60Mb in
> > > > > >> >> size and compressed.
> > > > > >> >>
> > > > > >> >> No doubt when it will be passed to the mapper, it is very
> much
> > > > > >>certain
> > > > > >> >>to
> > > > > >> >> have the single mapper invoked due to the pre-configured
> block
> > > > size.
> > > > > >> >>Now I
> > > > > >> >> need to check If I can override the splitSize property for
> the
> > > job.
> > > > > >> >>
> > > > > >> >> Moreover I can see that the job is running seamlessly slow,
> > > kindly
> > > > > >>find
> > > > > >> >> the stack trace for the job mentioned below.
> > > > > >> >>
> > > > > >> >> 2015-06-13 10:08:00,940 INFO [main]
> > > > org.apache.hadoop.mapred.MapTask:
> > > > > >> >> Processing split: hdfs://
> > > > > >> >>
> > > > > >> >>
> > > > > >>
> > > > > >>
> > > > >
> > > >
> > >
> >
> dev-hadoop-namenode.com:8020/tmp/kylin-9b675c66-4ce9-4a33-a356-2ccf9dbaca
> > > > > >>
> > > > >
> > > >
> > >
> >
> >>>>6a/kylin_intermediate_xyz_20150401000000_20150611000000_9b675c66_4ce9_4
> > > > > >>>>a3
> > > > > >> >>3_a356_2ccf9dbaca6a/000000_0:0+60440271
> > > > > >> >> 2015-06-13 10:08:01,014 INFO [main]
> > > > org.apache.hadoop.mapred.MapTask:
> > > > > >> >> (EQUATOR) 0 kvi 67108860(268435440)
> > > > > >> >> 2015-06-13 10:08:01,015 INFO [main]
> > > > org.apache.hadoop.mapred.MapTask:
> > > > > >> >> mapreduce.task.io.sort.mb: 256
> > > > > >> >> 2015-06-13 10:08:01,015 INFO [main]
> > > > org.apache.hadoop.mapred.MapTask:
> > > > > >> >>soft
> > > > > >> >> limit at 214748368
> > > > > >> >> 2015-06-13 10:08:01,015 INFO [main]
> > > > org.apache.hadoop.mapred.MapTask:
> > > > > >> >> bufstart = 0; bufvoid = 268435456
> > > > > >> >> 2015-06-13 10:08:01,015 INFO [main]
> > > > org.apache.hadoop.mapred.MapTask:
> > > > > >> >> kvstart = 67108860; length = 16777216
> > > > > >> >> 2015-06-13 10:08:01,023 INFO [main]
> > > > org.apache.hadoop.mapred.MapTask:
> > > > > >> >>Map
> > > > > >> >> output collector class =
> > > > > >> >>org.apache.hadoop.mapred.MapTask$MapOutputBuffer
> > > > > >> >> 2015-06-13 10:08:01,048 INFO [main]
> > > > > >> >> org.apache.hadoop.io.compress.zlib.ZlibFactory: Successfully
> > > > loaded &
> > > > > >> >> initialized native-zlib library
> > > > > >> >> 2015-06-13 10:08:01,048 INFO [main]
> > > > > >> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new
> > > decompressor
> > > > > >> >> [.deflate]
> > > > > >> >> 2015-06-13 10:08:01,052 INFO [main]
> > > > > >> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new
> > > decompressor
> > > > > >> >> [.deflate]
> > > > > >> >> 2015-06-13 10:08:01,052 INFO [main]
> > > > > >> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new
> > > decompressor
> > > > > >> >> [.deflate]
> > > > > >> >> 2015-06-13 10:08:01,053 INFO [main]
> > > > > >> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new
> > > decompressor
> > > > > >> >> [.deflate]
> > > > > >> >> 2015-06-13 10:08:01,058 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.AbstractHadoopJob: The absolute
> > path
> > > > for
> > > > > >> >>meta
> > > > > >> >> dir is
> > > > > >> >>
> > > > > >>
> > > > >
> > > >
> > >
> >
> >>>>/yarn/nm/usercache/biops/appcache/application_1433833901375_0139/contai
> > > > > >>>>ne
> > > > > >> >>r_1433833901375_0139_01_000002/meta
> > > > > >> >> 2015-06-13 10:08:01,059 INFO [main]
> > > > > >>org.apache.kylin.common.KylinConfig:
> > > > > >> >> Use
> > > > > >> >>
> > > > > >>
> > > > >
> > > >
> > >
> >
> >>>>KYLIN_CONF=/yarn/nm/usercache/biops/appcache/application_1433833901375_
> > > > > >>>>01
> > > > > >> >>39/container_1433833901375_0139_01_000002/meta
> > > > > >> >> 2015-06-13 10:08:01,084 INFO [main]
> > > > > >>org.apache.kylin.cube.CubeManager:
> > > > > >> >> Initializing CubeManager with config
> > > > > >> >> /yarn/nm/usercache/biops/filecache/19452/meta
> > > > > >> >> 2015-06-13 10:08:01,086 INFO [main]
> > > > > >> >> org.apache.kylin.common.persistence.ResourceStore: Using
> > metadata
> > > > url
> > > > > >> >> /yarn/nm/usercache/biops/filecache/19452/meta for resource
> > store
> > > > > >> >> 2015-06-13 10:08:01,327 INFO [main]
> > > > > >> >>org.apache.kylin.cube.CubeDescManager:
> > > > > >> >> Initializing CubeDescManager with config
> > > > > >> >> /yarn/nm/usercache/biops/filecache/19452/meta
> > > > > >> >> 2015-06-13 10:08:01,327 INFO [main]
> > > > > >> >>org.apache.kylin.cube.CubeDescManager:
> > > > > >> >> Reloading Cube Metadata from folder
> > > > > >> >> /yarn/nm/usercache/biops/filecache/19452/meta/cube_desc
> > > > > >> >> 2015-06-13 10:08:22,834 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 100000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:08:43,225 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 200000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:09:03,622 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 300000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:09:23,999 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 400000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:09:44,372 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 500000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:10:04,780 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 600000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:10:25,143 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 700000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:10:45,512 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 800000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:11:05,895 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 900000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:11:26,284 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 1000000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:11:46,716 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 1100000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:12:07,174 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 1200000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:12:27,646 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 1300000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:12:48,127 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 1400000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:13:08,614 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 1500000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:13:29,148 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 1600000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:13:49,686 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 1700000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:14:10,167 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 1800000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:14:30,652 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 1900000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:14:51,199 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 2000000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:15:11,719 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 2100000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:15:32,221 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 2200000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:15:52,717 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 2300000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:16:13,252 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 2400000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:16:33,769 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 2500000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:16:54,263 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 2600000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:17:14,741 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 2700000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:17:35,213 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 2800000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:17:55,699 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 2900000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:18:16,204 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 3000000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:18:36,721 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 3100000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:18:57,249 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 3200000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:19:17,743 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 3300000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:19:38,275 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 3400000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:19:58,812 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 3500000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:20:19,312 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 3600000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:20:39,920 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 3700000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:21:00,518 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 3800000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:21:21,008 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 3900000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:21:41,525 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 4000000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:22:02,023 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 4100000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:22:22,534 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 4200000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:22:43,063 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 4300000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:23:03,558 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 4400000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:23:24,043 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 4500000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:23:44,537 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 4600000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:24:05,015 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 4700000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:24:25,490 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 4800000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:24:45,995 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 4900000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:25:06,487 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 5000000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:25:26,964 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 5100000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:25:47,524 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 5200000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:26:08,032 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 5300000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:26:28,514 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 5400000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:26:49,053 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 5500000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:26:50,360 INFO [main]
> > > > org.apache.hadoop.mapred.MapTask:
> > > > > >> >> Spilling map output
> > > > > >> >> 2015-06-13 10:26:50,360 INFO [main]
> > > > org.apache.hadoop.mapred.MapTask:
> > > > > >> >> bufstart = 0; bufend = 126646464; bufvoid = 268435456
> > > > > >> >> 2015-06-13 10:26:50,360 INFO [main]
> > > > org.apache.hadoop.mapred.MapTask:
> > > > > >> >> kvstart = 67108860(268435440); kvend = 45083392(180333568);
> > > length
> > > > =
> > > > > >> >> 22025469/16777216
> > > > > >> >> 2015-06-13 10:26:50,360 INFO [main]
> > > > org.apache.hadoop.mapred.MapTask:
> > > > > >> >> (EQUATOR) 148671936 kvi 37167980(148671920)
> > > > > >> >> 2015-06-13 10:26:54,802 INFO [SpillThread]
> > > > > >> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new
> > compressor
> > > > > >> >>[.snappy]
> > > > > >> >> 2015-06-13 10:26:54,883 INFO [SpillThread]
> > > > > >> >> org.apache.kylin.job.hadoop.AbstractHadoopJob: The absolute
> > path
> > > > for
> > > > > >> >>meta
> > > > > >> >> dir is
> > > > > >> >>
> > > > > >>
> > > > >
> > > >
> > >
> >
> >>>>/yarn/nm/usercache/biops/appcache/application_1433833901375_0139/contai
> > > > > >>>>ne
> > > > > >> >>r_1433833901375_0139_01_000002/meta
> > > > > >> >> 2015-06-13 10:27:09,614 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 5600000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:27:30,135 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 5700000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:27:50,657 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 5800000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:27:59,691 INFO [SpillThread]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.CuboidReducer: Handled
> 100000
> > > > > >>records!
> > > > > >> >> 2015-06-13 10:28:11,185 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 5900000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:28:31,751 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 6000000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:28:52,303 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 6100000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:29:03,235 INFO [SpillThread]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.CuboidReducer: Handled
> 200000
> > > > > >>records!
> > > > > >> >> 2015-06-13 10:29:12,873 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 6200000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:29:33,424 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 6300000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:29:54,168 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 6400000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:30:14,771 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 6500000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:30:18,479 INFO [SpillThread]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.CuboidReducer: Handled
> 300000
> > > > > >>records!
> > > > > >> >> 2015-06-13 10:30:35,325 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 6600000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:30:39,831 INFO [SpillThread]
> > > > > >> >> org.apache.hadoop.mapred.MapTask: Finished spill 0
> > > > > >> >> 2015-06-13 10:30:39,831 INFO [main]
> > > > org.apache.hadoop.mapred.MapTask:
> > > > > >> >> (RESET) equator 148671936 kv 37167980(148671920) kvi
> > > > > >>32705668(130822672)
> > > > > >> >> 2015-06-13 10:30:55,807 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 6700000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:31:16,301 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 6800000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:31:36,804 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 6900000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:31:57,302 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 7000000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:32:17,790 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 7100000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:32:38,294 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 7200000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:32:58,830 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 7300000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:33:19,354 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 7400000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:33:39,866 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 7500000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:34:00,371 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 7600000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:34:20,872 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 7700000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:34:41,387 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 7800000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:35:01,884 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 7900000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:35:22,368 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 8000000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:35:42,868 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 8100000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:36:03,361 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 8200000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:36:23,981 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 8300000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:36:44,529 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 8400000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:37:05,034 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 8500000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:37:25,609 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 8600000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:37:46,152 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 8700000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:38:06,725 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 8800000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:38:27,296 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 8900000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:38:47,905 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 9000000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:39:08,430 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 9100000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:39:28,952 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 9200000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:39:49,460 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 9300000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:40:09,987 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 9400000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:40:30,551 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 9500000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:40:51,080 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 9600000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:41:11,579 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 9700000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:41:32,122 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 9800000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:41:52,623 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 9900000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:42:13,128 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 10000000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:42:33,628 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 10100000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:42:54,128 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 10200000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:43:14,629 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 10300000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:43:35,149 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 10400000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:43:55,764 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 10500000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:44:16,303 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 10600000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:44:36,857 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 10700000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:44:57,349 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 10800000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:45:17,840 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 10900000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:45:38,441 INFO [main]
> > > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > > 11000000
> > > > > >> >>records!
> > > > > >> >> 2015-06-13 10:45:41,057 INFO [main]
> > > > org.apache.hadoop.mapred.MapTask:
> > > > > >> >> Spilling map output
> > > > > >> >> 2015-06-13 10:45:41,057 INFO [main]
> > > > org.apache.hadoop.mapred.MapTask:
> > > > > >> >> bufstart = 148671936; bufend = 6882957; bufvoid = 268435443
> > > > > >> >> 2015-06-13 10:45:41,057 INFO [main]
> > > > org.apache.hadoop.mapred.MapTask:
> > > > > >> >> kvstart = 37167980(148671920); kvend = 15142512(60570048);
> > > length =
> > > > > >> >> 22025469/16777216
> > > > > >> >> 2015-06-13 10:45:41,057 INFO [main]
> > > > org.apache.hadoop.mapred.MapTask:
> > > > > >> >> (EQUATOR) 28908413 kvi 7227096(28908384)
> > > > > >> >>
> > > > > >> >> Thanks,
> > > > > >> >>
> > > > > >> >> On Mon, Jun 15, 2015 at 3:11 PM, 周千昊 <qh...@apache.org>
> > wrote:
> > > > > >> >>
> > > > > >> >>> Hi Mishra
> > > > > >> >>> It is as what you described, since the data size is
> 30MB
> > > > only,
> > > > > >> >>>hadoop
> > > > > >> >>> will only manage to run MR in single mapper and reduce.
> > > > > >> >>> As for your question about why it takes so long to run
> > on a
> > > > > >>small
> > > > > >> >>> dataset, can you please dig into the web page of *map reduce
> > > task
> > > > > >> >>>status
> > > > > >> >>> *to
> > > > > >> >>> check how much time the MR job really take, so that we can
> > make
> > > > > >>sure if
> > > > > >> >>> the
> > > > > >> >>> time is consuming on the MR job or kylin job scheduling
> module
> > > > > >> >>>
> > > > > >> >>> Vineet Mishra <cl...@gmail.com>于2015年6月15日周一
> 下午4:51写道：
> > > > > >> >>>
> > > > > >> >>> > Shi,
> > > > > >> >>> >
> > > > > >> >>> > Hadoop is setup correctly on my cluster with the default
> > block
> > > > > >>size
> > > > > >> >>>of
> > > > > >> >>> > 128Mb and its indeed very much running multiple
> > Mapper/Reducer
> > > > > >>based
> > > > > >> >>> jobs
> > > > > >> >>> > for other cases.
> > > > > >> >>> >
> > > > > >> >>> > Its the only Kylin Cube building which is running through
> > > single
> > > > > >>M/R
> > > > > >> >>> job.
> > > > > >> >>> >
> > > > > >> >>> > Moreover to my surprise, the 4th running job which is
> Build
> > > Base
> > > > > >> >>>Cuboid
> > > > > >> >>> > Data shows Data size as 30Mb, is it the reason due to
> which
> > > > single
> > > > > >> >>> mapper
> > > > > >> >>> > is getting invoked, if that being the case then even to
> > > process
> > > > > >>such
> > > > > >> >>>a
> > > > > >> >>> > small data set why is it taking around 50min.
> > > > > >> >>> >
> > > > > >> >>> > Thanks,
> > > > > >> >>> >
> > > > > >> >>> > On Mon, Jun 15, 2015 at 11:17 AM, Shi, Shaofeng <
> > > > shaoshi@ebay.com
> > > > > >
> > > > > >> >>> wrote:
> > > > > >> >>> >
> > > > > >> >>> > > Yes you can adjust these parameters, for example give a
> > > > smaller
> > > > > >> >>>value
> > > > > >> >>> for
> > > > > >> >>> > > kylin.job.mapreduce.default.reduce.input.mb; but it only
> > > > affects
> > > > > >> >>>the
> > > > > >> >>> > > reducer number;
> > > > > >> >>> > >
> > > > > >> >>> > > I suggest you investigate why there is only 1 mapper be
> > > > started;
> > > > > >> >>>Some
> > > > > >> >>> > > factors like hadoop cluster size, HDFS file block size
> > will
> > > > > >>impact
> > > > > >> >>> this;
> > > > > >> >>> > > You can run a SQL (a query which need run MR, not simple
> > > > select
> > > > > >>*)
> > > > > >> >>> with
> > > > > >> >>> > > hive -e, and then use the MR job track URL to see how
> many
> > > > > >>mappers
> > > > > >> >>>be
> > > > > >> >>> > > triggered; If it is still single, then the problem is in
> > > your
> > > > > >> >>>hadoop
> > > > > >> >>> > > configuration; Otherwise it may exists in Kylin, check
> if
> > > you
> > > > > >>put
> > > > > >> >>>some
> > > > > >> >>> > > additional parameter in conf/kylin_job_conf.xml.
> > > > > >> >>> > >
> > > > > >> >>> > >
> > > > > >> >>> > > On 6/15/15, 2:52 AM, "Vineet Mishra" <
> > > clearmidoubt@gmail.com>
> > > > > >> >>>wrote:
> > > > > >> >>> > >
> > > > > >> >>> > > >Can I have specification for these properties?
> > > > > >> >>> > > >
> > > > > >> >>> > > >KYLIN_JOB_MAPREDUCE_DEFAULT_REDUCE_COUNT_RATIO =
> > > > > >> >>> > > >"kylin.job.mapreduce.default.reduce.count.ratio";
> > > > > >> >>> > > >KYLIN_JOB_MAPREDUCE_DEFAULT_REDUCE_INPUT_MB =
> > > > > >> >>> > > >"kylin.job.mapreduce.default.reduce.input.mb";
> > > > > >> >>> > > >KYLIN_JOB_MAPREDUCE_MAX_REDUCER_NUMBER =
> > > > > >> >>> > > >"kylin.job.mapreduce.max.reducer.number";
> > > > > >> >>> > > >
> > > > > >> >>> > > >Thanks!
> > > > > >> >>> > > >
> > > > > >> >>> > > >On Sun, Jun 14, 2015 at 11:59 PM, Vineet Mishra <
> > > > > >> >>> clearmidoubt@gmail.com
> > > > > >> >>> > >
> > > > > >> >>> > > >wrote:
> > > > > >> >>> > > >
> > > > > >> >>> > > >> Hi Shi,
> > > > > >> >>> > > >>
> > > > > >> >>> > > >> Its alright!
> > > > > >> >>> > > >> So I was wondering my source hive Table is around 3
> Gb,
> > > > > >>despite
> > > > > >> >>>of
> > > > > >> >>> my
> > > > > >> >>> > > >>hive
> > > > > >> >>> > > >> table being partitioned and holding the data around
> > 50-70
> > > > Mb
> > > > > >>per
> > > > > >> >>> > > >>partition
> > > > > >> >>> > > >> the Mapper and Reducer getting spawned are single.
> The
> > > > > >>amount of
> > > > > >> >>> data
> > > > > >> >>> > > >>that
> > > > > >> >>> > > >> is being processed in the M/R is nothing as expected
> > but
> > > it
> > > > > >> >>>takes
> > > > > >> >>> hell
> > > > > >> >>> > > >>lot
> > > > > >> >>> > > >> of time.
> > > > > >> >>> > > >>
> > > > > >> >>> > > >> As mentioned in the trailing mail that the job is
> > getting
> > > > > >>very
> > > > > >> >>> slow,
> > > > > >> >>> > the
> > > > > >> >>> > > >> process Build Base Cuboid Data itself takes around
> > 50mins
> > > > to
> > > > > >>get
> > > > > >> >>> > > >> completed.
> > > > > >> >>> > > >>
> > > > > >> >>> > > >> I can tweak the reducer parameter mentioned by you
> but
> > > do u
> > > > > >> >>>think
> > > > > >> >>> that
> > > > > >> >>> > > >> will make a difference since the mapper is where the
> > most
> > > > of
> > > > > >>the
> > > > > >> >>> time
> > > > > >> >>> > is
> > > > > >> >>> > > >> spent.
> > > > > >> >>> > > >>
> > > > > >> >>> > > >> Can you share your thoughts for performance tuning
> for
> > > the
> > > > > >>cube
> > > > > >> >>> build!
> > > > > >> >>> > > >>
> > > > > >> >>> > > >> Thanks!
> > > > > >> >>> > > >>
> > > > > >> >>> > > >> On Sun, Jun 14, 2015 at 7:26 PM, Shi, Shaofeng
> > > > > >> >>><sh...@ebay.com>
> > > > > >> >>> > > wrote:
> > > > > >> >>> > > >>
> > > > > >> >>> > > >>> Hi, sorry, a busy weekend;
> > > > > >> >>> > > >>>
> > > > > >> >>> > > >>> Usually Kylin will request proper number of mapper
> and
> > > > > >> >>>reducers;
> > > > > >> >>> If
> > > > > >> >>> > you
> > > > > >> >>> > > >>> see single mapper/recudder, how much of your input
> and
> > > > > >>output?
> > > > > >> >>>If
> > > > > >> >>> > your
> > > > > >> >>> > > >>> cube is quite small, single mapper/reducer is
> > possible;
> > > > > >> >>> > > >>>
> > > > > >> >>> > > >>> Number of mappers is decided by the FileInputFormat;
> > But
> > > > > >> >>>number of
> > > > > >> >>> > > >>>reducer
> > > > > >> >>> > > >>> was set by Kylin, see:
> > > > > >> >>> > > >>>
> > > > > >> >>> > > >>>
> > > > > >> >>> > > >>>
> > > > > >> >>> > >
> > > > > >> >>>
> > > > > >> >>>
> > > > > >>
> > > > >
> > >
> https://github.com/apache/incubator-kylin/blob/master/job/src/main/java/
> > > > > >> >>> > > >>>org
> > > > > >> >>> > > >>> /apache/kylin/job/hadoop/cube/CuboidJob.java#L141
> > > > > >> >>> > > >>>
> > > > > >> >>> > > >>><
> > > > > >> >>> > >
> > > > > >> >>>
> > > > > >>
> > > >
> > https://github.com/apache/incubator-kylin/blob/master/job/src/main/java
> > > > > >> >>> > >
> >>>/org/apache/kylin/job/hadoop/cube/CuboidJob.java#L141>
> > > > > >> >>> > > >>>
> > > > > >> >>> > > >>>
> > > > > >> >>> > > >>>
> > > > > >> >>> > > >>>
> > > > > >> >>> > > >>> On 6/14/15, 5:25 PM, "Vineet Mishra"
> > > > > >><cl...@gmail.com>
> > > > > >> >>> wrote:
> > > > > >> >>> > > >>>
> > > > > >> >>> > > >>> >Urgent call, any follow up on this?
> > > > > >> >>> > > >>> >
> > > > > >> >>> > > >>> >On Fri, Jun 12, 2015 at 6:46 PM, Vineet Mishra
> > > > > >> >>> > > >>><cl...@gmail.com>
> > > > > >> >>> > > >>> >wrote:
> > > > > >> >>> > > >>> >
> > > > > >> >>> > > >>> >>
> > > > > >> >>> > > >>> >> Why
> org.apache.kylin.job.hadoop.cube.CuboidReducer
> > is
> > > > > >> >>>running
> > > > > >> >>> > Single
> > > > > >> >>> > > >>> >> Mapper/Reducer for the job. Can I have the
> > > > understanding
> > > > > >> >>>behind
> > > > > >> >>> > the
> > > > > >> >>> > > >>> >>reason
> > > > > >> >>> > > >>> >> of running it as single mapper/reducer.
> > > > > >> >>> > > >>> >>
> > > > > >> >>> > > >>> >> Thanks!
> > > > > >> >>> > > >>> >>
> > > > > >> >>> > > >>> >> On Fri, Jun 12, 2015 at 6:30 PM, Vineet Mishra
> > > > > >> >>> > > >>><clearmidoubt@gmail.com
> > > > > >> >>> > > >>> >
> > > > > >> >>> > > >>> >> wrote:
> > > > > >> >>> > > >>> >>
> > > > > >> >>> > > >>> >>> Hi All,
> > > > > >> >>> > > >>> >>>
> > > > > >> >>> > > >>> >>> I am building a cube using Kylin and I could see
> > > that
> > > > > >>the
> > > > > >> >>>job
> > > > > >> >>> is
> > > > > >> >>> > > >>> >>>running
> > > > > >> >>> > > >>> >>> with Single Mapper and Reducer for some of the
> > > > > >>intermediate
> > > > > >> >>> > process
> > > > > >> >>> > > >>> >>>such as
> > > > > >> >>> > > >>> >>>
> > > > > >> >>> > > >>> >>> Extract Fact Table Distinct Columns
> > > > > >> >>> > > >>> >>> Build Dimension Dictionary
> > > > > >> >>> > > >>> >>> Build N-Dimension Cuboid
> > > > > >> >>> > > >>> >>>
> > > > > >> >>> > > >>> >>> I am not sure what's the reason behind running
> the
> > > job
> > > > > >>with
> > > > > >> >>> > single
> > > > > >> >>> > > >>> M/R,
> > > > > >> >>> > > >>> >>> is it really necessary or is it some default
> > config.
> > > > > >>which
> > > > > >> >>> can be
> > > > > >> >>> > > >>> >>>tweaked,
> > > > > >> >>> > > >>> >>> its 70 Mins and the job status is 25% !
> > > > > >> >>> > > >>> >>>
> > > > > >> >>> > > >>> >>> Urgent Call!
> > > > > >> >>> > > >>> >>>
> > > > > >> >>> > > >>> >>> Thanks!
> > > > > >> >>> > > >>> >>>
> > > > > >> >>> > > >>> >>
> > > > > >> >>> > > >>> >>
> > > > > >> >>> > > >>>
> > > > > >> >>> > > >>>
> > > > > >> >>> > > >>
> > > > > >> >>> > >
> > > > > >> >>> > >
> > > > > >> >>> >
> > > > > >> >>>
> > > > > >> >>
> > > > > >> >>
> > > > > >>
> > > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>
Re: Jobs Running with Single Mapper/Reducer
Posted by ShaoFeng Shi <sh...@gmail.com>.
That's more clear, thank you; In theory the cube size is fixed on the same
input, you must miss some change between cube1 and cube2 :-)
Anyway, suggest you try more times, use the same cube, disable it, purge
it, and then build again with the same date range; If the problem repeats,
welcome you to open a JIRA to Kylin:
https://issues.apache.org/jira/secure/Dashboard.jspa
2015-06-16 16:33 GMT+08:00 Vineet Mishra <cl...@gmail.com>:
> Let me clarify,
>
> 1) So initially I had a cube say cube1 with some dimension and measures
> which was of size 420Mb
> 2) later I created another cube say cube2 with same dimension as cube1 but
> with added measures(so in cube1 it was 1 measure while in cube2 it was 8)
> which was of size 130Mb(expected greater size due to addition of multiple
> measure but it was lesser than the cube1)
>
> 3) thereafter I created another cube say cube3 with same properties as
> cube2 and made the build and it was 120Mb(slightly lesser than the last
> build with the same data source)
>
> So I was looking out as what's the reason as the cube size is varying and
> moreover to my surprise the cube2 which is holding extra measures as in
> comparison to cube1 is even lesser in size.
>
> I guess that makes things clear to you.
>
> Thanks,
>
> On Tue, Jun 16, 2015 at 1:25 PM, ShaoFeng Shi <sh...@gmail.com>
> wrote:
>
> > I'm confused; You said these two cubes are exactly same in an early
> reply,
> > but now you mentioned the seoncd has more measures... they are NOT the
> > same; Different measures takes different storage spaces; What kind of the
> > measures defined in your former cube? Did you use DISTINCT COUNT measure?
> >
> > 2015-06-16 3:28 GMT+08:00 Vineet Mishra <cl...@gmail.com>:
> >
> > > Hi Shi,
> > >
> > > Well yes these cubes are build for the same date range without having
> the
> > > data changed on the table and moreover hbase compression was not even
> > > touched so far.
> > >
> > > One interesting thing which I got was the later cube which was small in
> > > size and completing in lesser time was having multiple measures(around
> 7
> > or
> > > 8) in comparison to the former cube having just 1 or perhaps 2 measure
> > with
> > > more time to complete.
> > >
> > > Running the same job again, it came up with some different size for the
> > > same data, so last time it was reduced 130Mb now its even less, around
> > > 123Mb. This change is observed within 4 hour of the running the last
> job
> > > with the same data set.
> > >
> > > I am not sure but how the kylin is internally building these cube
> > dimension
> > > so as to inflate the data size by 3 and more over when the measures are
> > > very few as in comparison.
> > >
> > > Let me know if you could get anything out of it.
> > >
> > > Thanks!
> > >
> > > On Mon, Jun 15, 2015 at 8:51 PM, Shi, Shaofeng <sh...@ebay.com>
> wrote:
> > >
> > > > Just guess: Are these two cube builds cross the same date range? Was
> > the
> > > > data on fact table changed between these two builds? Did HBase enable
> > > > compression recently?
> > > >
> > > > You can build once more with the same cube and date range to see
> > whether
> > > > it can be re-produced; The cube size should be consistent in my
> > > knowledge;
> > > >
> > > > On 6/15/15, 10:52 PM, "Vineet Mishra" <cl...@gmail.com>
> wrote:
> > > >
> > > > >Thanks Shi,
> > > > >
> > > > >By the way I was curious as for the same table, dimension and
> metric,
> > I
> > > > >could see the final cube size varying from 130 Mb to 420Mb.
> > > > >
> > > > >The cube is exactly the same as another one but when the cube build
> > job
> > > is
> > > > >run I could see the cube size difference as mentioned below,
> > > > >
> > > > >*Earlier Cube Build*
> > > > >Size - 420 Mb
> > > > >Time Taken for building cube - 140 mins
> > > > >Source Records - ~11Million
> > > > >
> > > > >*Today's Cube Build*
> > > > >Size - 130 Mb
> > > > >Time Taken for building cube - 22 mins
> > > > >Source Records - ~11Million
> > > > >
> > > > >I don't find any reason to have three time cube size difference. Any
> > > ideas
> > > > >about this?
> > > > >
> > > > >Thanks!
> > > > >
> > > > >On Mon, Jun 15, 2015 at 8:08 PM, Shi, Shaofeng <sh...@ebay.com>
> > > wrote:
> > > > >
> > > > >> Good catch, and thanks for the sharing;
> > > > >>
> > > > >> On 6/15/15, 10:20 PM, "Vineet Mishra" <cl...@gmail.com>
> > wrote:
> > > > >>
> > > > >> >Hi All,
> > > > >> >
> > > > >> >Well I got it through.
> > > > >> >
> > > > >> >Actually it was basically the pre-process which was converting my
> > 3Gb
> > > > >>hive
> > > > >> >table to 60Mb sequence file which due to block size of 128Mb was
> > > > >>running
> > > > >> >in
> > > > >> >a single Mapper job.
> > > > >> >
> > > > >> >I changed the split size to 20Mb and it was able to spawn 3
> Mapper
> > > > >>after
> > > > >> >that.
> > > > >> >
> > > > >> >Anyways, thanks all for your quick response.
> > > > >> >
> > > > >> >Thanks!
> > > > >> >
> > > > >> >On Mon, Jun 15, 2015 at 3:27 PM, Vineet Mishra <
> > > clearmidoubt@gmail.com
> > > > >
> > > > >> >wrote:
> > > > >> >
> > > > >> >> Hi,
> > > > >> >>
> > > > >> >> Sorry I misunderstood this value, actually 30Mb was the output
> > for
> > > > >>that
> > > > >> >> job. But I guess I have the got reason, (not exactly confirmed)
> > as
> > > > >>why
> > > > >> >>its
> > > > >> >> behaving like this, the input to this job is sequence file
> which
> > is
> > > > >> >>60Mb in
> > > > >> >> size and compressed.
> > > > >> >>
> > > > >> >> No doubt when it will be passed to the mapper, it is very much
> > > > >>certain
> > > > >> >>to
> > > > >> >> have the single mapper invoked due to the pre-configured block
> > > size.
> > > > >> >>Now I
> > > > >> >> need to check If I can override the splitSize property for the
> > job.
> > > > >> >>
> > > > >> >> Moreover I can see that the job is running seamlessly slow,
> > kindly
> > > > >>find
> > > > >> >> the stack trace for the job mentioned below.
> > > > >> >>
> > > > >> >> 2015-06-13 10:08:00,940 INFO [main]
> > > org.apache.hadoop.mapred.MapTask:
> > > > >> >> Processing split: hdfs://
> > > > >> >>
> > > > >> >>
> > > > >>
> > > > >>
> > > >
> > >
> >
> dev-hadoop-namenode.com:8020/tmp/kylin-9b675c66-4ce9-4a33-a356-2ccf9dbaca
> > > > >>
> > > >
> > >
> >
> >>>>6a/kylin_intermediate_xyz_20150401000000_20150611000000_9b675c66_4ce9_4
> > > > >>>>a3
> > > > >> >>3_a356_2ccf9dbaca6a/000000_0:0+60440271
> > > > >> >> 2015-06-13 10:08:01,014 INFO [main]
> > > org.apache.hadoop.mapred.MapTask:
> > > > >> >> (EQUATOR) 0 kvi 67108860(268435440)
> > > > >> >> 2015-06-13 10:08:01,015 INFO [main]
> > > org.apache.hadoop.mapred.MapTask:
> > > > >> >> mapreduce.task.io.sort.mb: 256
> > > > >> >> 2015-06-13 10:08:01,015 INFO [main]
> > > org.apache.hadoop.mapred.MapTask:
> > > > >> >>soft
> > > > >> >> limit at 214748368
> > > > >> >> 2015-06-13 10:08:01,015 INFO [main]
> > > org.apache.hadoop.mapred.MapTask:
> > > > >> >> bufstart = 0; bufvoid = 268435456
> > > > >> >> 2015-06-13 10:08:01,015 INFO [main]
> > > org.apache.hadoop.mapred.MapTask:
> > > > >> >> kvstart = 67108860; length = 16777216
> > > > >> >> 2015-06-13 10:08:01,023 INFO [main]
> > > org.apache.hadoop.mapred.MapTask:
> > > > >> >>Map
> > > > >> >> output collector class =
> > > > >> >>org.apache.hadoop.mapred.MapTask$MapOutputBuffer
> > > > >> >> 2015-06-13 10:08:01,048 INFO [main]
> > > > >> >> org.apache.hadoop.io.compress.zlib.ZlibFactory: Successfully
> > > loaded &
> > > > >> >> initialized native-zlib library
> > > > >> >> 2015-06-13 10:08:01,048 INFO [main]
> > > > >> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new
> > decompressor
> > > > >> >> [.deflate]
> > > > >> >> 2015-06-13 10:08:01,052 INFO [main]
> > > > >> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new
> > decompressor
> > > > >> >> [.deflate]
> > > > >> >> 2015-06-13 10:08:01,052 INFO [main]
> > > > >> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new
> > decompressor
> > > > >> >> [.deflate]
> > > > >> >> 2015-06-13 10:08:01,053 INFO [main]
> > > > >> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new
> > decompressor
> > > > >> >> [.deflate]
> > > > >> >> 2015-06-13 10:08:01,058 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.AbstractHadoopJob: The absolute
> path
> > > for
> > > > >> >>meta
> > > > >> >> dir is
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> >>>>/yarn/nm/usercache/biops/appcache/application_1433833901375_0139/contai
> > > > >>>>ne
> > > > >> >>r_1433833901375_0139_01_000002/meta
> > > > >> >> 2015-06-13 10:08:01,059 INFO [main]
> > > > >>org.apache.kylin.common.KylinConfig:
> > > > >> >> Use
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> >>>>KYLIN_CONF=/yarn/nm/usercache/biops/appcache/application_1433833901375_
> > > > >>>>01
> > > > >> >>39/container_1433833901375_0139_01_000002/meta
> > > > >> >> 2015-06-13 10:08:01,084 INFO [main]
> > > > >>org.apache.kylin.cube.CubeManager:
> > > > >> >> Initializing CubeManager with config
> > > > >> >> /yarn/nm/usercache/biops/filecache/19452/meta
> > > > >> >> 2015-06-13 10:08:01,086 INFO [main]
> > > > >> >> org.apache.kylin.common.persistence.ResourceStore: Using
> metadata
> > > url
> > > > >> >> /yarn/nm/usercache/biops/filecache/19452/meta for resource
> store
> > > > >> >> 2015-06-13 10:08:01,327 INFO [main]
> > > > >> >>org.apache.kylin.cube.CubeDescManager:
> > > > >> >> Initializing CubeDescManager with config
> > > > >> >> /yarn/nm/usercache/biops/filecache/19452/meta
> > > > >> >> 2015-06-13 10:08:01,327 INFO [main]
> > > > >> >>org.apache.kylin.cube.CubeDescManager:
> > > > >> >> Reloading Cube Metadata from folder
> > > > >> >> /yarn/nm/usercache/biops/filecache/19452/meta/cube_desc
> > > > >> >> 2015-06-13 10:08:22,834 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 100000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:08:43,225 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 200000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:09:03,622 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 300000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:09:23,999 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 400000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:09:44,372 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 500000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:10:04,780 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 600000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:10:25,143 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 700000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:10:45,512 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 800000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:11:05,895 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 900000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:11:26,284 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 1000000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:11:46,716 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 1100000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:12:07,174 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 1200000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:12:27,646 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 1300000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:12:48,127 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 1400000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:13:08,614 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 1500000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:13:29,148 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 1600000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:13:49,686 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 1700000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:14:10,167 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 1800000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:14:30,652 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 1900000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:14:51,199 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 2000000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:15:11,719 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 2100000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:15:32,221 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 2200000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:15:52,717 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 2300000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:16:13,252 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 2400000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:16:33,769 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 2500000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:16:54,263 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 2600000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:17:14,741 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 2700000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:17:35,213 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 2800000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:17:55,699 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 2900000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:18:16,204 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 3000000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:18:36,721 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 3100000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:18:57,249 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 3200000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:19:17,743 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 3300000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:19:38,275 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 3400000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:19:58,812 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 3500000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:20:19,312 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 3600000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:20:39,920 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 3700000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:21:00,518 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 3800000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:21:21,008 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 3900000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:21:41,525 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 4000000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:22:02,023 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 4100000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:22:22,534 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 4200000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:22:43,063 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 4300000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:23:03,558 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 4400000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:23:24,043 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 4500000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:23:44,537 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 4600000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:24:05,015 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 4700000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:24:25,490 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 4800000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:24:45,995 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 4900000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:25:06,487 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 5000000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:25:26,964 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 5100000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:25:47,524 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 5200000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:26:08,032 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 5300000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:26:28,514 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 5400000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:26:49,053 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 5500000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:26:50,360 INFO [main]
> > > org.apache.hadoop.mapred.MapTask:
> > > > >> >> Spilling map output
> > > > >> >> 2015-06-13 10:26:50,360 INFO [main]
> > > org.apache.hadoop.mapred.MapTask:
> > > > >> >> bufstart = 0; bufend = 126646464; bufvoid = 268435456
> > > > >> >> 2015-06-13 10:26:50,360 INFO [main]
> > > org.apache.hadoop.mapred.MapTask:
> > > > >> >> kvstart = 67108860(268435440); kvend = 45083392(180333568);
> > length
> > > =
> > > > >> >> 22025469/16777216
> > > > >> >> 2015-06-13 10:26:50,360 INFO [main]
> > > org.apache.hadoop.mapred.MapTask:
> > > > >> >> (EQUATOR) 148671936 kvi 37167980(148671920)
> > > > >> >> 2015-06-13 10:26:54,802 INFO [SpillThread]
> > > > >> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new
> compressor
> > > > >> >>[.snappy]
> > > > >> >> 2015-06-13 10:26:54,883 INFO [SpillThread]
> > > > >> >> org.apache.kylin.job.hadoop.AbstractHadoopJob: The absolute
> path
> > > for
> > > > >> >>meta
> > > > >> >> dir is
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> >>>>/yarn/nm/usercache/biops/appcache/application_1433833901375_0139/contai
> > > > >>>>ne
> > > > >> >>r_1433833901375_0139_01_000002/meta
> > > > >> >> 2015-06-13 10:27:09,614 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 5600000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:27:30,135 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 5700000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:27:50,657 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 5800000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:27:59,691 INFO [SpillThread]
> > > > >> >> org.apache.kylin.job.hadoop.cube.CuboidReducer: Handled 100000
> > > > >>records!
> > > > >> >> 2015-06-13 10:28:11,185 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 5900000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:28:31,751 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 6000000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:28:52,303 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 6100000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:29:03,235 INFO [SpillThread]
> > > > >> >> org.apache.kylin.job.hadoop.cube.CuboidReducer: Handled 200000
> > > > >>records!
> > > > >> >> 2015-06-13 10:29:12,873 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 6200000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:29:33,424 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 6300000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:29:54,168 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 6400000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:30:14,771 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 6500000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:30:18,479 INFO [SpillThread]
> > > > >> >> org.apache.kylin.job.hadoop.cube.CuboidReducer: Handled 300000
> > > > >>records!
> > > > >> >> 2015-06-13 10:30:35,325 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 6600000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:30:39,831 INFO [SpillThread]
> > > > >> >> org.apache.hadoop.mapred.MapTask: Finished spill 0
> > > > >> >> 2015-06-13 10:30:39,831 INFO [main]
> > > org.apache.hadoop.mapred.MapTask:
> > > > >> >> (RESET) equator 148671936 kv 37167980(148671920) kvi
> > > > >>32705668(130822672)
> > > > >> >> 2015-06-13 10:30:55,807 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 6700000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:31:16,301 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 6800000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:31:36,804 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 6900000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:31:57,302 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 7000000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:32:17,790 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 7100000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:32:38,294 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 7200000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:32:58,830 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 7300000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:33:19,354 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 7400000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:33:39,866 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 7500000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:34:00,371 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 7600000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:34:20,872 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 7700000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:34:41,387 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 7800000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:35:01,884 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 7900000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:35:22,368 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 8000000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:35:42,868 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 8100000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:36:03,361 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 8200000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:36:23,981 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 8300000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:36:44,529 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 8400000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:37:05,034 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 8500000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:37:25,609 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 8600000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:37:46,152 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 8700000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:38:06,725 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 8800000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:38:27,296 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 8900000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:38:47,905 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 9000000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:39:08,430 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 9100000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:39:28,952 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 9200000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:39:49,460 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 9300000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:40:09,987 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 9400000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:40:30,551 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 9500000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:40:51,080 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 9600000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:41:11,579 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 9700000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:41:32,122 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 9800000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:41:52,623 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 9900000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:42:13,128 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 10000000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:42:33,628 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 10100000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:42:54,128 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 10200000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:43:14,629 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 10300000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:43:35,149 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 10400000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:43:55,764 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 10500000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:44:16,303 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 10600000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:44:36,857 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 10700000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:44:57,349 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 10800000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:45:17,840 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 10900000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:45:38,441 INFO [main]
> > > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> > 11000000
> > > > >> >>records!
> > > > >> >> 2015-06-13 10:45:41,057 INFO [main]
> > > org.apache.hadoop.mapred.MapTask:
> > > > >> >> Spilling map output
> > > > >> >> 2015-06-13 10:45:41,057 INFO [main]
> > > org.apache.hadoop.mapred.MapTask:
> > > > >> >> bufstart = 148671936; bufend = 6882957; bufvoid = 268435443
> > > > >> >> 2015-06-13 10:45:41,057 INFO [main]
> > > org.apache.hadoop.mapred.MapTask:
> > > > >> >> kvstart = 37167980(148671920); kvend = 15142512(60570048);
> > length =
> > > > >> >> 22025469/16777216
> > > > >> >> 2015-06-13 10:45:41,057 INFO [main]
> > > org.apache.hadoop.mapred.MapTask:
> > > > >> >> (EQUATOR) 28908413 kvi 7227096(28908384)
> > > > >> >>
> > > > >> >> Thanks,
> > > > >> >>
> > > > >> >> On Mon, Jun 15, 2015 at 3:11 PM, 周千昊 <qh...@apache.org>
> wrote:
> > > > >> >>
> > > > >> >>> Hi Mishra
> > > > >> >>> It is as what you described, since the data size is 30MB
> > > only,
> > > > >> >>>hadoop
> > > > >> >>> will only manage to run MR in single mapper and reduce.
> > > > >> >>> As for your question about why it takes so long to run
> on a
> > > > >>small
> > > > >> >>> dataset, can you please dig into the web page of *map reduce
> > task
> > > > >> >>>status
> > > > >> >>> *to
> > > > >> >>> check how much time the MR job really take, so that we can
> make
> > > > >>sure if
> > > > >> >>> the
> > > > >> >>> time is consuming on the MR job or kylin job scheduling module
> > > > >> >>>
> > > > >> >>> Vineet Mishra <cl...@gmail.com>于2015年6月15日周一 下午4:51写道：
> > > > >> >>>
> > > > >> >>> > Shi,
> > > > >> >>> >
> > > > >> >>> > Hadoop is setup correctly on my cluster with the default
> block
> > > > >>size
> > > > >> >>>of
> > > > >> >>> > 128Mb and its indeed very much running multiple
> Mapper/Reducer
> > > > >>based
> > > > >> >>> jobs
> > > > >> >>> > for other cases.
> > > > >> >>> >
> > > > >> >>> > Its the only Kylin Cube building which is running through
> > single
> > > > >>M/R
> > > > >> >>> job.
> > > > >> >>> >
> > > > >> >>> > Moreover to my surprise, the 4th running job which is Build
> > Base
> > > > >> >>>Cuboid
> > > > >> >>> > Data shows Data size as 30Mb, is it the reason due to which
> > > single
> > > > >> >>> mapper
> > > > >> >>> > is getting invoked, if that being the case then even to
> > process
> > > > >>such
> > > > >> >>>a
> > > > >> >>> > small data set why is it taking around 50min.
> > > > >> >>> >
> > > > >> >>> > Thanks,
> > > > >> >>> >
> > > > >> >>> > On Mon, Jun 15, 2015 at 11:17 AM, Shi, Shaofeng <
> > > shaoshi@ebay.com
> > > > >
> > > > >> >>> wrote:
> > > > >> >>> >
> > > > >> >>> > > Yes you can adjust these parameters, for example give a
> > > smaller
> > > > >> >>>value
> > > > >> >>> for
> > > > >> >>> > > kylin.job.mapreduce.default.reduce.input.mb; but it only
> > > affects
> > > > >> >>>the
> > > > >> >>> > > reducer number;
> > > > >> >>> > >
> > > > >> >>> > > I suggest you investigate why there is only 1 mapper be
> > > started;
> > > > >> >>>Some
> > > > >> >>> > > factors like hadoop cluster size, HDFS file block size
> will
> > > > >>impact
> > > > >> >>> this;
> > > > >> >>> > > You can run a SQL (a query which need run MR, not simple
> > > select
> > > > >>*)
> > > > >> >>> with
> > > > >> >>> > > hive -e, and then use the MR job track URL to see how many
> > > > >>mappers
> > > > >> >>>be
> > > > >> >>> > > triggered; If it is still single, then the problem is in
> > your
> > > > >> >>>hadoop
> > > > >> >>> > > configuration; Otherwise it may exists in Kylin, check if
> > you
> > > > >>put
> > > > >> >>>some
> > > > >> >>> > > additional parameter in conf/kylin_job_conf.xml.
> > > > >> >>> > >
> > > > >> >>> > >
> > > > >> >>> > > On 6/15/15, 2:52 AM, "Vineet Mishra" <
> > clearmidoubt@gmail.com>
> > > > >> >>>wrote:
> > > > >> >>> > >
> > > > >> >>> > > >Can I have specification for these properties?
> > > > >> >>> > > >
> > > > >> >>> > > >KYLIN_JOB_MAPREDUCE_DEFAULT_REDUCE_COUNT_RATIO =
> > > > >> >>> > > >"kylin.job.mapreduce.default.reduce.count.ratio";
> > > > >> >>> > > >KYLIN_JOB_MAPREDUCE_DEFAULT_REDUCE_INPUT_MB =
> > > > >> >>> > > >"kylin.job.mapreduce.default.reduce.input.mb";
> > > > >> >>> > > >KYLIN_JOB_MAPREDUCE_MAX_REDUCER_NUMBER =
> > > > >> >>> > > >"kylin.job.mapreduce.max.reducer.number";
> > > > >> >>> > > >
> > > > >> >>> > > >Thanks!
> > > > >> >>> > > >
> > > > >> >>> > > >On Sun, Jun 14, 2015 at 11:59 PM, Vineet Mishra <
> > > > >> >>> clearmidoubt@gmail.com
> > > > >> >>> > >
> > > > >> >>> > > >wrote:
> > > > >> >>> > > >
> > > > >> >>> > > >> Hi Shi,
> > > > >> >>> > > >>
> > > > >> >>> > > >> Its alright!
> > > > >> >>> > > >> So I was wondering my source hive Table is around 3 Gb,
> > > > >>despite
> > > > >> >>>of
> > > > >> >>> my
> > > > >> >>> > > >>hive
> > > > >> >>> > > >> table being partitioned and holding the data around
> 50-70
> > > Mb
> > > > >>per
> > > > >> >>> > > >>partition
> > > > >> >>> > > >> the Mapper and Reducer getting spawned are single. The
> > > > >>amount of
> > > > >> >>> data
> > > > >> >>> > > >>that
> > > > >> >>> > > >> is being processed in the M/R is nothing as expected
> but
> > it
> > > > >> >>>takes
> > > > >> >>> hell
> > > > >> >>> > > >>lot
> > > > >> >>> > > >> of time.
> > > > >> >>> > > >>
> > > > >> >>> > > >> As mentioned in the trailing mail that the job is
> getting
> > > > >>very
> > > > >> >>> slow,
> > > > >> >>> > the
> > > > >> >>> > > >> process Build Base Cuboid Data itself takes around
> 50mins
> > > to
> > > > >>get
> > > > >> >>> > > >> completed.
> > > > >> >>> > > >>
> > > > >> >>> > > >> I can tweak the reducer parameter mentioned by you but
> > do u
> > > > >> >>>think
> > > > >> >>> that
> > > > >> >>> > > >> will make a difference since the mapper is where the
> most
> > > of
> > > > >>the
> > > > >> >>> time
> > > > >> >>> > is
> > > > >> >>> > > >> spent.
> > > > >> >>> > > >>
> > > > >> >>> > > >> Can you share your thoughts for performance tuning for
> > the
> > > > >>cube
> > > > >> >>> build!
> > > > >> >>> > > >>
> > > > >> >>> > > >> Thanks!
> > > > >> >>> > > >>
> > > > >> >>> > > >> On Sun, Jun 14, 2015 at 7:26 PM, Shi, Shaofeng
> > > > >> >>><sh...@ebay.com>
> > > > >> >>> > > wrote:
> > > > >> >>> > > >>
> > > > >> >>> > > >>> Hi, sorry, a busy weekend;
> > > > >> >>> > > >>>
> > > > >> >>> > > >>> Usually Kylin will request proper number of mapper and
> > > > >> >>>reducers;
> > > > >> >>> If
> > > > >> >>> > you
> > > > >> >>> > > >>> see single mapper/recudder, how much of your input and
> > > > >>output?
> > > > >> >>>If
> > > > >> >>> > your
> > > > >> >>> > > >>> cube is quite small, single mapper/reducer is
> possible;
> > > > >> >>> > > >>>
> > > > >> >>> > > >>> Number of mappers is decided by the FileInputFormat;
> But
> > > > >> >>>number of
> > > > >> >>> > > >>>reducer
> > > > >> >>> > > >>> was set by Kylin, see:
> > > > >> >>> > > >>>
> > > > >> >>> > > >>>
> > > > >> >>> > > >>>
> > > > >> >>> > >
> > > > >> >>>
> > > > >> >>>
> > > > >>
> > > >
> > https://github.com/apache/incubator-kylin/blob/master/job/src/main/java/
> > > > >> >>> > > >>>org
> > > > >> >>> > > >>> /apache/kylin/job/hadoop/cube/CuboidJob.java#L141
> > > > >> >>> > > >>>
> > > > >> >>> > > >>><
> > > > >> >>> > >
> > > > >> >>>
> > > > >>
> > >
> https://github.com/apache/incubator-kylin/blob/master/job/src/main/java
> > > > >> >>> > > >>>/org/apache/kylin/job/hadoop/cube/CuboidJob.java#L141>
> > > > >> >>> > > >>>
> > > > >> >>> > > >>>
> > > > >> >>> > > >>>
> > > > >> >>> > > >>>
> > > > >> >>> > > >>> On 6/14/15, 5:25 PM, "Vineet Mishra"
> > > > >><cl...@gmail.com>
> > > > >> >>> wrote:
> > > > >> >>> > > >>>
> > > > >> >>> > > >>> >Urgent call, any follow up on this?
> > > > >> >>> > > >>> >
> > > > >> >>> > > >>> >On Fri, Jun 12, 2015 at 6:46 PM, Vineet Mishra
> > > > >> >>> > > >>><cl...@gmail.com>
> > > > >> >>> > > >>> >wrote:
> > > > >> >>> > > >>> >
> > > > >> >>> > > >>> >>
> > > > >> >>> > > >>> >> Why org.apache.kylin.job.hadoop.cube.CuboidReducer
> is
> > > > >> >>>running
> > > > >> >>> > Single
> > > > >> >>> > > >>> >> Mapper/Reducer for the job. Can I have the
> > > understanding
> > > > >> >>>behind
> > > > >> >>> > the
> > > > >> >>> > > >>> >>reason
> > > > >> >>> > > >>> >> of running it as single mapper/reducer.
> > > > >> >>> > > >>> >>
> > > > >> >>> > > >>> >> Thanks!
> > > > >> >>> > > >>> >>
> > > > >> >>> > > >>> >> On Fri, Jun 12, 2015 at 6:30 PM, Vineet Mishra
> > > > >> >>> > > >>><clearmidoubt@gmail.com
> > > > >> >>> > > >>> >
> > > > >> >>> > > >>> >> wrote:
> > > > >> >>> > > >>> >>
> > > > >> >>> > > >>> >>> Hi All,
> > > > >> >>> > > >>> >>>
> > > > >> >>> > > >>> >>> I am building a cube using Kylin and I could see
> > that
> > > > >>the
> > > > >> >>>job
> > > > >> >>> is
> > > > >> >>> > > >>> >>>running
> > > > >> >>> > > >>> >>> with Single Mapper and Reducer for some of the
> > > > >>intermediate
> > > > >> >>> > process
> > > > >> >>> > > >>> >>>such as
> > > > >> >>> > > >>> >>>
> > > > >> >>> > > >>> >>> Extract Fact Table Distinct Columns
> > > > >> >>> > > >>> >>> Build Dimension Dictionary
> > > > >> >>> > > >>> >>> Build N-Dimension Cuboid
> > > > >> >>> > > >>> >>>
> > > > >> >>> > > >>> >>> I am not sure what's the reason behind running the
> > job
> > > > >>with
> > > > >> >>> > single
> > > > >> >>> > > >>> M/R,
> > > > >> >>> > > >>> >>> is it really necessary or is it some default
> config.
> > > > >>which
> > > > >> >>> can be
> > > > >> >>> > > >>> >>>tweaked,
> > > > >> >>> > > >>> >>> its 70 Mins and the job status is 25% !
> > > > >> >>> > > >>> >>>
> > > > >> >>> > > >>> >>> Urgent Call!
> > > > >> >>> > > >>> >>>
> > > > >> >>> > > >>> >>> Thanks!
> > > > >> >>> > > >>> >>>
> > > > >> >>> > > >>> >>
> > > > >> >>> > > >>> >>
> > > > >> >>> > > >>>
> > > > >> >>> > > >>>
> > > > >> >>> > > >>
> > > > >> >>> > >
> > > > >> >>> > >
> > > > >> >>> >
> > > > >> >>>
> > > > >> >>
> > > > >> >>
> > > > >>
> > > > >>
> > > >
> > > >
> > >
> >
>
Re: Jobs Running with Single Mapper/Reducer
Posted by Vineet Mishra <cl...@gmail.com>.
Let me clarify,
1) So initially I had a cube say cube1 with some dimension and measures
which was of size 420Mb
2) later I created another cube say cube2 with same dimension as cube1 but
with added measures(so in cube1 it was 1 measure while in cube2 it was 8)
which was of size 130Mb(expected greater size due to addition of multiple
measure but it was lesser than the cube1)
3) thereafter I created another cube say cube3 with same properties as
cube2 and made the build and it was 120Mb(slightly lesser than the last
build with the same data source)
So I was looking out as what's the reason as the cube size is varying and
moreover to my surprise the cube2 which is holding extra measures as in
comparison to cube1 is even lesser in size.
I guess that makes things clear to you.
Thanks,
On Tue, Jun 16, 2015 at 1:25 PM, ShaoFeng Shi <sh...@gmail.com> wrote:
> I'm confused; You said these two cubes are exactly same in an early reply,
> but now you mentioned the seoncd has more measures... they are NOT the
> same; Different measures takes different storage spaces; What kind of the
> measures defined in your former cube? Did you use DISTINCT COUNT measure?
>
> 2015-06-16 3:28 GMT+08:00 Vineet Mishra <cl...@gmail.com>:
>
> > Hi Shi,
> >
> > Well yes these cubes are build for the same date range without having the
> > data changed on the table and moreover hbase compression was not even
> > touched so far.
> >
> > One interesting thing which I got was the later cube which was small in
> > size and completing in lesser time was having multiple measures(around 7
> or
> > 8) in comparison to the former cube having just 1 or perhaps 2 measure
> with
> > more time to complete.
> >
> > Running the same job again, it came up with some different size for the
> > same data, so last time it was reduced 130Mb now its even less, around
> > 123Mb. This change is observed within 4 hour of the running the last job
> > with the same data set.
> >
> > I am not sure but how the kylin is internally building these cube
> dimension
> > so as to inflate the data size by 3 and more over when the measures are
> > very few as in comparison.
> >
> > Let me know if you could get anything out of it.
> >
> > Thanks!
> >
> > On Mon, Jun 15, 2015 at 8:51 PM, Shi, Shaofeng <sh...@ebay.com> wrote:
> >
> > > Just guess: Are these two cube builds cross the same date range? Was
> the
> > > data on fact table changed between these two builds? Did HBase enable
> > > compression recently?
> > >
> > > You can build once more with the same cube and date range to see
> whether
> > > it can be re-produced; The cube size should be consistent in my
> > knowledge;
> > >
> > > On 6/15/15, 10:52 PM, "Vineet Mishra" <cl...@gmail.com> wrote:
> > >
> > > >Thanks Shi,
> > > >
> > > >By the way I was curious as for the same table, dimension and metric,
> I
> > > >could see the final cube size varying from 130 Mb to 420Mb.
> > > >
> > > >The cube is exactly the same as another one but when the cube build
> job
> > is
> > > >run I could see the cube size difference as mentioned below,
> > > >
> > > >*Earlier Cube Build*
> > > >Size - 420 Mb
> > > >Time Taken for building cube - 140 mins
> > > >Source Records - ~11Million
> > > >
> > > >*Today's Cube Build*
> > > >Size - 130 Mb
> > > >Time Taken for building cube - 22 mins
> > > >Source Records - ~11Million
> > > >
> > > >I don't find any reason to have three time cube size difference. Any
> > ideas
> > > >about this?
> > > >
> > > >Thanks!
> > > >
> > > >On Mon, Jun 15, 2015 at 8:08 PM, Shi, Shaofeng <sh...@ebay.com>
> > wrote:
> > > >
> > > >> Good catch, and thanks for the sharing;
> > > >>
> > > >> On 6/15/15, 10:20 PM, "Vineet Mishra" <cl...@gmail.com>
> wrote:
> > > >>
> > > >> >Hi All,
> > > >> >
> > > >> >Well I got it through.
> > > >> >
> > > >> >Actually it was basically the pre-process which was converting my
> 3Gb
> > > >>hive
> > > >> >table to 60Mb sequence file which due to block size of 128Mb was
> > > >>running
> > > >> >in
> > > >> >a single Mapper job.
> > > >> >
> > > >> >I changed the split size to 20Mb and it was able to spawn 3 Mapper
> > > >>after
> > > >> >that.
> > > >> >
> > > >> >Anyways, thanks all for your quick response.
> > > >> >
> > > >> >Thanks!
> > > >> >
> > > >> >On Mon, Jun 15, 2015 at 3:27 PM, Vineet Mishra <
> > clearmidoubt@gmail.com
> > > >
> > > >> >wrote:
> > > >> >
> > > >> >> Hi,
> > > >> >>
> > > >> >> Sorry I misunderstood this value, actually 30Mb was the output
> for
> > > >>that
> > > >> >> job. But I guess I have the got reason, (not exactly confirmed)
> as
> > > >>why
> > > >> >>its
> > > >> >> behaving like this, the input to this job is sequence file which
> is
> > > >> >>60Mb in
> > > >> >> size and compressed.
> > > >> >>
> > > >> >> No doubt when it will be passed to the mapper, it is very much
> > > >>certain
> > > >> >>to
> > > >> >> have the single mapper invoked due to the pre-configured block
> > size.
> > > >> >>Now I
> > > >> >> need to check If I can override the splitSize property for the
> job.
> > > >> >>
> > > >> >> Moreover I can see that the job is running seamlessly slow,
> kindly
> > > >>find
> > > >> >> the stack trace for the job mentioned below.
> > > >> >>
> > > >> >> 2015-06-13 10:08:00,940 INFO [main]
> > org.apache.hadoop.mapred.MapTask:
> > > >> >> Processing split: hdfs://
> > > >> >>
> > > >> >>
> > > >>
> > > >>
> > >
> >
> dev-hadoop-namenode.com:8020/tmp/kylin-9b675c66-4ce9-4a33-a356-2ccf9dbaca
> > > >>
> > >
> >
> >>>>6a/kylin_intermediate_xyz_20150401000000_20150611000000_9b675c66_4ce9_4
> > > >>>>a3
> > > >> >>3_a356_2ccf9dbaca6a/000000_0:0+60440271
> > > >> >> 2015-06-13 10:08:01,014 INFO [main]
> > org.apache.hadoop.mapred.MapTask:
> > > >> >> (EQUATOR) 0 kvi 67108860(268435440)
> > > >> >> 2015-06-13 10:08:01,015 INFO [main]
> > org.apache.hadoop.mapred.MapTask:
> > > >> >> mapreduce.task.io.sort.mb: 256
> > > >> >> 2015-06-13 10:08:01,015 INFO [main]
> > org.apache.hadoop.mapred.MapTask:
> > > >> >>soft
> > > >> >> limit at 214748368
> > > >> >> 2015-06-13 10:08:01,015 INFO [main]
> > org.apache.hadoop.mapred.MapTask:
> > > >> >> bufstart = 0; bufvoid = 268435456
> > > >> >> 2015-06-13 10:08:01,015 INFO [main]
> > org.apache.hadoop.mapred.MapTask:
> > > >> >> kvstart = 67108860; length = 16777216
> > > >> >> 2015-06-13 10:08:01,023 INFO [main]
> > org.apache.hadoop.mapred.MapTask:
> > > >> >>Map
> > > >> >> output collector class =
> > > >> >>org.apache.hadoop.mapred.MapTask$MapOutputBuffer
> > > >> >> 2015-06-13 10:08:01,048 INFO [main]
> > > >> >> org.apache.hadoop.io.compress.zlib.ZlibFactory: Successfully
> > loaded &
> > > >> >> initialized native-zlib library
> > > >> >> 2015-06-13 10:08:01,048 INFO [main]
> > > >> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new
> decompressor
> > > >> >> [.deflate]
> > > >> >> 2015-06-13 10:08:01,052 INFO [main]
> > > >> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new
> decompressor
> > > >> >> [.deflate]
> > > >> >> 2015-06-13 10:08:01,052 INFO [main]
> > > >> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new
> decompressor
> > > >> >> [.deflate]
> > > >> >> 2015-06-13 10:08:01,053 INFO [main]
> > > >> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new
> decompressor
> > > >> >> [.deflate]
> > > >> >> 2015-06-13 10:08:01,058 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.AbstractHadoopJob: The absolute path
> > for
> > > >> >>meta
> > > >> >> dir is
> > > >> >>
> > > >>
> > >
> >
> >>>>/yarn/nm/usercache/biops/appcache/application_1433833901375_0139/contai
> > > >>>>ne
> > > >> >>r_1433833901375_0139_01_000002/meta
> > > >> >> 2015-06-13 10:08:01,059 INFO [main]
> > > >>org.apache.kylin.common.KylinConfig:
> > > >> >> Use
> > > >> >>
> > > >>
> > >
> >
> >>>>KYLIN_CONF=/yarn/nm/usercache/biops/appcache/application_1433833901375_
> > > >>>>01
> > > >> >>39/container_1433833901375_0139_01_000002/meta
> > > >> >> 2015-06-13 10:08:01,084 INFO [main]
> > > >>org.apache.kylin.cube.CubeManager:
> > > >> >> Initializing CubeManager with config
> > > >> >> /yarn/nm/usercache/biops/filecache/19452/meta
> > > >> >> 2015-06-13 10:08:01,086 INFO [main]
> > > >> >> org.apache.kylin.common.persistence.ResourceStore: Using metadata
> > url
> > > >> >> /yarn/nm/usercache/biops/filecache/19452/meta for resource store
> > > >> >> 2015-06-13 10:08:01,327 INFO [main]
> > > >> >>org.apache.kylin.cube.CubeDescManager:
> > > >> >> Initializing CubeDescManager with config
> > > >> >> /yarn/nm/usercache/biops/filecache/19452/meta
> > > >> >> 2015-06-13 10:08:01,327 INFO [main]
> > > >> >>org.apache.kylin.cube.CubeDescManager:
> > > >> >> Reloading Cube Metadata from folder
> > > >> >> /yarn/nm/usercache/biops/filecache/19452/meta/cube_desc
> > > >> >> 2015-06-13 10:08:22,834 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 100000
> > > >> >>records!
> > > >> >> 2015-06-13 10:08:43,225 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 200000
> > > >> >>records!
> > > >> >> 2015-06-13 10:09:03,622 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 300000
> > > >> >>records!
> > > >> >> 2015-06-13 10:09:23,999 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 400000
> > > >> >>records!
> > > >> >> 2015-06-13 10:09:44,372 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 500000
> > > >> >>records!
> > > >> >> 2015-06-13 10:10:04,780 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 600000
> > > >> >>records!
> > > >> >> 2015-06-13 10:10:25,143 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 700000
> > > >> >>records!
> > > >> >> 2015-06-13 10:10:45,512 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 800000
> > > >> >>records!
> > > >> >> 2015-06-13 10:11:05,895 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 900000
> > > >> >>records!
> > > >> >> 2015-06-13 10:11:26,284 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 1000000
> > > >> >>records!
> > > >> >> 2015-06-13 10:11:46,716 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 1100000
> > > >> >>records!
> > > >> >> 2015-06-13 10:12:07,174 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 1200000
> > > >> >>records!
> > > >> >> 2015-06-13 10:12:27,646 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 1300000
> > > >> >>records!
> > > >> >> 2015-06-13 10:12:48,127 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 1400000
> > > >> >>records!
> > > >> >> 2015-06-13 10:13:08,614 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 1500000
> > > >> >>records!
> > > >> >> 2015-06-13 10:13:29,148 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 1600000
> > > >> >>records!
> > > >> >> 2015-06-13 10:13:49,686 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 1700000
> > > >> >>records!
> > > >> >> 2015-06-13 10:14:10,167 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 1800000
> > > >> >>records!
> > > >> >> 2015-06-13 10:14:30,652 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 1900000
> > > >> >>records!
> > > >> >> 2015-06-13 10:14:51,199 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 2000000
> > > >> >>records!
> > > >> >> 2015-06-13 10:15:11,719 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 2100000
> > > >> >>records!
> > > >> >> 2015-06-13 10:15:32,221 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 2200000
> > > >> >>records!
> > > >> >> 2015-06-13 10:15:52,717 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 2300000
> > > >> >>records!
> > > >> >> 2015-06-13 10:16:13,252 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 2400000
> > > >> >>records!
> > > >> >> 2015-06-13 10:16:33,769 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 2500000
> > > >> >>records!
> > > >> >> 2015-06-13 10:16:54,263 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 2600000
> > > >> >>records!
> > > >> >> 2015-06-13 10:17:14,741 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 2700000
> > > >> >>records!
> > > >> >> 2015-06-13 10:17:35,213 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 2800000
> > > >> >>records!
> > > >> >> 2015-06-13 10:17:55,699 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 2900000
> > > >> >>records!
> > > >> >> 2015-06-13 10:18:16,204 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 3000000
> > > >> >>records!
> > > >> >> 2015-06-13 10:18:36,721 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 3100000
> > > >> >>records!
> > > >> >> 2015-06-13 10:18:57,249 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 3200000
> > > >> >>records!
> > > >> >> 2015-06-13 10:19:17,743 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 3300000
> > > >> >>records!
> > > >> >> 2015-06-13 10:19:38,275 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 3400000
> > > >> >>records!
> > > >> >> 2015-06-13 10:19:58,812 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 3500000
> > > >> >>records!
> > > >> >> 2015-06-13 10:20:19,312 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 3600000
> > > >> >>records!
> > > >> >> 2015-06-13 10:20:39,920 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 3700000
> > > >> >>records!
> > > >> >> 2015-06-13 10:21:00,518 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 3800000
> > > >> >>records!
> > > >> >> 2015-06-13 10:21:21,008 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 3900000
> > > >> >>records!
> > > >> >> 2015-06-13 10:21:41,525 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 4000000
> > > >> >>records!
> > > >> >> 2015-06-13 10:22:02,023 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 4100000
> > > >> >>records!
> > > >> >> 2015-06-13 10:22:22,534 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 4200000
> > > >> >>records!
> > > >> >> 2015-06-13 10:22:43,063 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 4300000
> > > >> >>records!
> > > >> >> 2015-06-13 10:23:03,558 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 4400000
> > > >> >>records!
> > > >> >> 2015-06-13 10:23:24,043 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 4500000
> > > >> >>records!
> > > >> >> 2015-06-13 10:23:44,537 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 4600000
> > > >> >>records!
> > > >> >> 2015-06-13 10:24:05,015 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 4700000
> > > >> >>records!
> > > >> >> 2015-06-13 10:24:25,490 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 4800000
> > > >> >>records!
> > > >> >> 2015-06-13 10:24:45,995 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 4900000
> > > >> >>records!
> > > >> >> 2015-06-13 10:25:06,487 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 5000000
> > > >> >>records!
> > > >> >> 2015-06-13 10:25:26,964 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 5100000
> > > >> >>records!
> > > >> >> 2015-06-13 10:25:47,524 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 5200000
> > > >> >>records!
> > > >> >> 2015-06-13 10:26:08,032 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 5300000
> > > >> >>records!
> > > >> >> 2015-06-13 10:26:28,514 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 5400000
> > > >> >>records!
> > > >> >> 2015-06-13 10:26:49,053 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 5500000
> > > >> >>records!
> > > >> >> 2015-06-13 10:26:50,360 INFO [main]
> > org.apache.hadoop.mapred.MapTask:
> > > >> >> Spilling map output
> > > >> >> 2015-06-13 10:26:50,360 INFO [main]
> > org.apache.hadoop.mapred.MapTask:
> > > >> >> bufstart = 0; bufend = 126646464; bufvoid = 268435456
> > > >> >> 2015-06-13 10:26:50,360 INFO [main]
> > org.apache.hadoop.mapred.MapTask:
> > > >> >> kvstart = 67108860(268435440); kvend = 45083392(180333568);
> length
> > =
> > > >> >> 22025469/16777216
> > > >> >> 2015-06-13 10:26:50,360 INFO [main]
> > org.apache.hadoop.mapred.MapTask:
> > > >> >> (EQUATOR) 148671936 kvi 37167980(148671920)
> > > >> >> 2015-06-13 10:26:54,802 INFO [SpillThread]
> > > >> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor
> > > >> >>[.snappy]
> > > >> >> 2015-06-13 10:26:54,883 INFO [SpillThread]
> > > >> >> org.apache.kylin.job.hadoop.AbstractHadoopJob: The absolute path
> > for
> > > >> >>meta
> > > >> >> dir is
> > > >> >>
> > > >>
> > >
> >
> >>>>/yarn/nm/usercache/biops/appcache/application_1433833901375_0139/contai
> > > >>>>ne
> > > >> >>r_1433833901375_0139_01_000002/meta
> > > >> >> 2015-06-13 10:27:09,614 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 5600000
> > > >> >>records!
> > > >> >> 2015-06-13 10:27:30,135 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 5700000
> > > >> >>records!
> > > >> >> 2015-06-13 10:27:50,657 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 5800000
> > > >> >>records!
> > > >> >> 2015-06-13 10:27:59,691 INFO [SpillThread]
> > > >> >> org.apache.kylin.job.hadoop.cube.CuboidReducer: Handled 100000
> > > >>records!
> > > >> >> 2015-06-13 10:28:11,185 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 5900000
> > > >> >>records!
> > > >> >> 2015-06-13 10:28:31,751 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 6000000
> > > >> >>records!
> > > >> >> 2015-06-13 10:28:52,303 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 6100000
> > > >> >>records!
> > > >> >> 2015-06-13 10:29:03,235 INFO [SpillThread]
> > > >> >> org.apache.kylin.job.hadoop.cube.CuboidReducer: Handled 200000
> > > >>records!
> > > >> >> 2015-06-13 10:29:12,873 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 6200000
> > > >> >>records!
> > > >> >> 2015-06-13 10:29:33,424 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 6300000
> > > >> >>records!
> > > >> >> 2015-06-13 10:29:54,168 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 6400000
> > > >> >>records!
> > > >> >> 2015-06-13 10:30:14,771 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 6500000
> > > >> >>records!
> > > >> >> 2015-06-13 10:30:18,479 INFO [SpillThread]
> > > >> >> org.apache.kylin.job.hadoop.cube.CuboidReducer: Handled 300000
> > > >>records!
> > > >> >> 2015-06-13 10:30:35,325 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 6600000
> > > >> >>records!
> > > >> >> 2015-06-13 10:30:39,831 INFO [SpillThread]
> > > >> >> org.apache.hadoop.mapred.MapTask: Finished spill 0
> > > >> >> 2015-06-13 10:30:39,831 INFO [main]
> > org.apache.hadoop.mapred.MapTask:
> > > >> >> (RESET) equator 148671936 kv 37167980(148671920) kvi
> > > >>32705668(130822672)
> > > >> >> 2015-06-13 10:30:55,807 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 6700000
> > > >> >>records!
> > > >> >> 2015-06-13 10:31:16,301 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 6800000
> > > >> >>records!
> > > >> >> 2015-06-13 10:31:36,804 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 6900000
> > > >> >>records!
> > > >> >> 2015-06-13 10:31:57,302 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 7000000
> > > >> >>records!
> > > >> >> 2015-06-13 10:32:17,790 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 7100000
> > > >> >>records!
> > > >> >> 2015-06-13 10:32:38,294 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 7200000
> > > >> >>records!
> > > >> >> 2015-06-13 10:32:58,830 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 7300000
> > > >> >>records!
> > > >> >> 2015-06-13 10:33:19,354 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 7400000
> > > >> >>records!
> > > >> >> 2015-06-13 10:33:39,866 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 7500000
> > > >> >>records!
> > > >> >> 2015-06-13 10:34:00,371 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 7600000
> > > >> >>records!
> > > >> >> 2015-06-13 10:34:20,872 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 7700000
> > > >> >>records!
> > > >> >> 2015-06-13 10:34:41,387 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 7800000
> > > >> >>records!
> > > >> >> 2015-06-13 10:35:01,884 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 7900000
> > > >> >>records!
> > > >> >> 2015-06-13 10:35:22,368 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 8000000
> > > >> >>records!
> > > >> >> 2015-06-13 10:35:42,868 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 8100000
> > > >> >>records!
> > > >> >> 2015-06-13 10:36:03,361 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 8200000
> > > >> >>records!
> > > >> >> 2015-06-13 10:36:23,981 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 8300000
> > > >> >>records!
> > > >> >> 2015-06-13 10:36:44,529 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 8400000
> > > >> >>records!
> > > >> >> 2015-06-13 10:37:05,034 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 8500000
> > > >> >>records!
> > > >> >> 2015-06-13 10:37:25,609 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 8600000
> > > >> >>records!
> > > >> >> 2015-06-13 10:37:46,152 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 8700000
> > > >> >>records!
> > > >> >> 2015-06-13 10:38:06,725 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 8800000
> > > >> >>records!
> > > >> >> 2015-06-13 10:38:27,296 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 8900000
> > > >> >>records!
> > > >> >> 2015-06-13 10:38:47,905 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 9000000
> > > >> >>records!
> > > >> >> 2015-06-13 10:39:08,430 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 9100000
> > > >> >>records!
> > > >> >> 2015-06-13 10:39:28,952 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 9200000
> > > >> >>records!
> > > >> >> 2015-06-13 10:39:49,460 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 9300000
> > > >> >>records!
> > > >> >> 2015-06-13 10:40:09,987 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 9400000
> > > >> >>records!
> > > >> >> 2015-06-13 10:40:30,551 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 9500000
> > > >> >>records!
> > > >> >> 2015-06-13 10:40:51,080 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 9600000
> > > >> >>records!
> > > >> >> 2015-06-13 10:41:11,579 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 9700000
> > > >> >>records!
> > > >> >> 2015-06-13 10:41:32,122 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 9800000
> > > >> >>records!
> > > >> >> 2015-06-13 10:41:52,623 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 9900000
> > > >> >>records!
> > > >> >> 2015-06-13 10:42:13,128 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 10000000
> > > >> >>records!
> > > >> >> 2015-06-13 10:42:33,628 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 10100000
> > > >> >>records!
> > > >> >> 2015-06-13 10:42:54,128 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 10200000
> > > >> >>records!
> > > >> >> 2015-06-13 10:43:14,629 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 10300000
> > > >> >>records!
> > > >> >> 2015-06-13 10:43:35,149 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 10400000
> > > >> >>records!
> > > >> >> 2015-06-13 10:43:55,764 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 10500000
> > > >> >>records!
> > > >> >> 2015-06-13 10:44:16,303 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 10600000
> > > >> >>records!
> > > >> >> 2015-06-13 10:44:36,857 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 10700000
> > > >> >>records!
> > > >> >> 2015-06-13 10:44:57,349 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 10800000
> > > >> >>records!
> > > >> >> 2015-06-13 10:45:17,840 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 10900000
> > > >> >>records!
> > > >> >> 2015-06-13 10:45:38,441 INFO [main]
> > > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled
> 11000000
> > > >> >>records!
> > > >> >> 2015-06-13 10:45:41,057 INFO [main]
> > org.apache.hadoop.mapred.MapTask:
> > > >> >> Spilling map output
> > > >> >> 2015-06-13 10:45:41,057 INFO [main]
> > org.apache.hadoop.mapred.MapTask:
> > > >> >> bufstart = 148671936; bufend = 6882957; bufvoid = 268435443
> > > >> >> 2015-06-13 10:45:41,057 INFO [main]
> > org.apache.hadoop.mapred.MapTask:
> > > >> >> kvstart = 37167980(148671920); kvend = 15142512(60570048);
> length =
> > > >> >> 22025469/16777216
> > > >> >> 2015-06-13 10:45:41,057 INFO [main]
> > org.apache.hadoop.mapred.MapTask:
> > > >> >> (EQUATOR) 28908413 kvi 7227096(28908384)
> > > >> >>
> > > >> >> Thanks,
> > > >> >>
> > > >> >> On Mon, Jun 15, 2015 at 3:11 PM, 周千昊 <qh...@apache.org> wrote:
> > > >> >>
> > > >> >>> Hi Mishra
> > > >> >>> It is as what you described, since the data size is 30MB
> > only,
> > > >> >>>hadoop
> > > >> >>> will only manage to run MR in single mapper and reduce.
> > > >> >>> As for your question about why it takes so long to run on a
> > > >>small
> > > >> >>> dataset, can you please dig into the web page of *map reduce
> task
> > > >> >>>status
> > > >> >>> *to
> > > >> >>> check how much time the MR job really take, so that we can make
> > > >>sure if
> > > >> >>> the
> > > >> >>> time is consuming on the MR job or kylin job scheduling module
> > > >> >>>
> > > >> >>> Vineet Mishra <cl...@gmail.com>于2015年6月15日周一 下午4:51写道：
> > > >> >>>
> > > >> >>> > Shi,
> > > >> >>> >
> > > >> >>> > Hadoop is setup correctly on my cluster with the default block
> > > >>size
> > > >> >>>of
> > > >> >>> > 128Mb and its indeed very much running multiple Mapper/Reducer
> > > >>based
> > > >> >>> jobs
> > > >> >>> > for other cases.
> > > >> >>> >
> > > >> >>> > Its the only Kylin Cube building which is running through
> single
> > > >>M/R
> > > >> >>> job.
> > > >> >>> >
> > > >> >>> > Moreover to my surprise, the 4th running job which is Build
> Base
> > > >> >>>Cuboid
> > > >> >>> > Data shows Data size as 30Mb, is it the reason due to which
> > single
> > > >> >>> mapper
> > > >> >>> > is getting invoked, if that being the case then even to
> process
> > > >>such
> > > >> >>>a
> > > >> >>> > small data set why is it taking around 50min.
> > > >> >>> >
> > > >> >>> > Thanks,
> > > >> >>> >
> > > >> >>> > On Mon, Jun 15, 2015 at 11:17 AM, Shi, Shaofeng <
> > shaoshi@ebay.com
> > > >
> > > >> >>> wrote:
> > > >> >>> >
> > > >> >>> > > Yes you can adjust these parameters, for example give a
> > smaller
> > > >> >>>value
> > > >> >>> for
> > > >> >>> > > kylin.job.mapreduce.default.reduce.input.mb; but it only
> > affects
> > > >> >>>the
> > > >> >>> > > reducer number;
> > > >> >>> > >
> > > >> >>> > > I suggest you investigate why there is only 1 mapper be
> > started;
> > > >> >>>Some
> > > >> >>> > > factors like hadoop cluster size, HDFS file block size will
> > > >>impact
> > > >> >>> this;
> > > >> >>> > > You can run a SQL (a query which need run MR, not simple
> > select
> > > >>*)
> > > >> >>> with
> > > >> >>> > > hive -e, and then use the MR job track URL to see how many
> > > >>mappers
> > > >> >>>be
> > > >> >>> > > triggered; If it is still single, then the problem is in
> your
> > > >> >>>hadoop
> > > >> >>> > > configuration; Otherwise it may exists in Kylin, check if
> you
> > > >>put
> > > >> >>>some
> > > >> >>> > > additional parameter in conf/kylin_job_conf.xml.
> > > >> >>> > >
> > > >> >>> > >
> > > >> >>> > > On 6/15/15, 2:52 AM, "Vineet Mishra" <
> clearmidoubt@gmail.com>
> > > >> >>>wrote:
> > > >> >>> > >
> > > >> >>> > > >Can I have specification for these properties?
> > > >> >>> > > >
> > > >> >>> > > >KYLIN_JOB_MAPREDUCE_DEFAULT_REDUCE_COUNT_RATIO =
> > > >> >>> > > >"kylin.job.mapreduce.default.reduce.count.ratio";
> > > >> >>> > > >KYLIN_JOB_MAPREDUCE_DEFAULT_REDUCE_INPUT_MB =
> > > >> >>> > > >"kylin.job.mapreduce.default.reduce.input.mb";
> > > >> >>> > > >KYLIN_JOB_MAPREDUCE_MAX_REDUCER_NUMBER =
> > > >> >>> > > >"kylin.job.mapreduce.max.reducer.number";
> > > >> >>> > > >
> > > >> >>> > > >Thanks!
> > > >> >>> > > >
> > > >> >>> > > >On Sun, Jun 14, 2015 at 11:59 PM, Vineet Mishra <
> > > >> >>> clearmidoubt@gmail.com
> > > >> >>> > >
> > > >> >>> > > >wrote:
> > > >> >>> > > >
> > > >> >>> > > >> Hi Shi,
> > > >> >>> > > >>
> > > >> >>> > > >> Its alright!
> > > >> >>> > > >> So I was wondering my source hive Table is around 3 Gb,
> > > >>despite
> > > >> >>>of
> > > >> >>> my
> > > >> >>> > > >>hive
> > > >> >>> > > >> table being partitioned and holding the data around 50-70
> > Mb
> > > >>per
> > > >> >>> > > >>partition
> > > >> >>> > > >> the Mapper and Reducer getting spawned are single. The
> > > >>amount of
> > > >> >>> data
> > > >> >>> > > >>that
> > > >> >>> > > >> is being processed in the M/R is nothing as expected but
> it
> > > >> >>>takes
> > > >> >>> hell
> > > >> >>> > > >>lot
> > > >> >>> > > >> of time.
> > > >> >>> > > >>
> > > >> >>> > > >> As mentioned in the trailing mail that the job is getting
> > > >>very
> > > >> >>> slow,
> > > >> >>> > the
> > > >> >>> > > >> process Build Base Cuboid Data itself takes around 50mins
> > to
> > > >>get
> > > >> >>> > > >> completed.
> > > >> >>> > > >>
> > > >> >>> > > >> I can tweak the reducer parameter mentioned by you but
> do u
> > > >> >>>think
> > > >> >>> that
> > > >> >>> > > >> will make a difference since the mapper is where the most
> > of
> > > >>the
> > > >> >>> time
> > > >> >>> > is
> > > >> >>> > > >> spent.
> > > >> >>> > > >>
> > > >> >>> > > >> Can you share your thoughts for performance tuning for
> the
> > > >>cube
> > > >> >>> build!
> > > >> >>> > > >>
> > > >> >>> > > >> Thanks!
> > > >> >>> > > >>
> > > >> >>> > > >> On Sun, Jun 14, 2015 at 7:26 PM, Shi, Shaofeng
> > > >> >>><sh...@ebay.com>
> > > >> >>> > > wrote:
> > > >> >>> > > >>
> > > >> >>> > > >>> Hi, sorry, a busy weekend;
> > > >> >>> > > >>>
> > > >> >>> > > >>> Usually Kylin will request proper number of mapper and
> > > >> >>>reducers;
> > > >> >>> If
> > > >> >>> > you
> > > >> >>> > > >>> see single mapper/recudder, how much of your input and
> > > >>output?
> > > >> >>>If
> > > >> >>> > your
> > > >> >>> > > >>> cube is quite small, single mapper/reducer is possible;
> > > >> >>> > > >>>
> > > >> >>> > > >>> Number of mappers is decided by the FileInputFormat; But
> > > >> >>>number of
> > > >> >>> > > >>>reducer
> > > >> >>> > > >>> was set by Kylin, see:
> > > >> >>> > > >>>
> > > >> >>> > > >>>
> > > >> >>> > > >>>
> > > >> >>> > >
> > > >> >>>
> > > >> >>>
> > > >>
> > >
> https://github.com/apache/incubator-kylin/blob/master/job/src/main/java/
> > > >> >>> > > >>>org
> > > >> >>> > > >>> /apache/kylin/job/hadoop/cube/CuboidJob.java#L141
> > > >> >>> > > >>>
> > > >> >>> > > >>><
> > > >> >>> > >
> > > >> >>>
> > > >>
> > https://github.com/apache/incubator-kylin/blob/master/job/src/main/java
> > > >> >>> > > >>>/org/apache/kylin/job/hadoop/cube/CuboidJob.java#L141>
> > > >> >>> > > >>>
> > > >> >>> > > >>>
> > > >> >>> > > >>>
> > > >> >>> > > >>>
> > > >> >>> > > >>> On 6/14/15, 5:25 PM, "Vineet Mishra"
> > > >><cl...@gmail.com>
> > > >> >>> wrote:
> > > >> >>> > > >>>
> > > >> >>> > > >>> >Urgent call, any follow up on this?
> > > >> >>> > > >>> >
> > > >> >>> > > >>> >On Fri, Jun 12, 2015 at 6:46 PM, Vineet Mishra
> > > >> >>> > > >>><cl...@gmail.com>
> > > >> >>> > > >>> >wrote:
> > > >> >>> > > >>> >
> > > >> >>> > > >>> >>
> > > >> >>> > > >>> >> Why org.apache.kylin.job.hadoop.cube.CuboidReducer is
> > > >> >>>running
> > > >> >>> > Single
> > > >> >>> > > >>> >> Mapper/Reducer for the job. Can I have the
> > understanding
> > > >> >>>behind
> > > >> >>> > the
> > > >> >>> > > >>> >>reason
> > > >> >>> > > >>> >> of running it as single mapper/reducer.
> > > >> >>> > > >>> >>
> > > >> >>> > > >>> >> Thanks!
> > > >> >>> > > >>> >>
> > > >> >>> > > >>> >> On Fri, Jun 12, 2015 at 6:30 PM, Vineet Mishra
> > > >> >>> > > >>><clearmidoubt@gmail.com
> > > >> >>> > > >>> >
> > > >> >>> > > >>> >> wrote:
> > > >> >>> > > >>> >>
> > > >> >>> > > >>> >>> Hi All,
> > > >> >>> > > >>> >>>
> > > >> >>> > > >>> >>> I am building a cube using Kylin and I could see
> that
> > > >>the
> > > >> >>>job
> > > >> >>> is
> > > >> >>> > > >>> >>>running
> > > >> >>> > > >>> >>> with Single Mapper and Reducer for some of the
> > > >>intermediate
> > > >> >>> > process
> > > >> >>> > > >>> >>>such as
> > > >> >>> > > >>> >>>
> > > >> >>> > > >>> >>> Extract Fact Table Distinct Columns
> > > >> >>> > > >>> >>> Build Dimension Dictionary
> > > >> >>> > > >>> >>> Build N-Dimension Cuboid
> > > >> >>> > > >>> >>>
> > > >> >>> > > >>> >>> I am not sure what's the reason behind running the
> job
> > > >>with
> > > >> >>> > single
> > > >> >>> > > >>> M/R,
> > > >> >>> > > >>> >>> is it really necessary or is it some default config.
> > > >>which
> > > >> >>> can be
> > > >> >>> > > >>> >>>tweaked,
> > > >> >>> > > >>> >>> its 70 Mins and the job status is 25% !
> > > >> >>> > > >>> >>>
> > > >> >>> > > >>> >>> Urgent Call!
> > > >> >>> > > >>> >>>
> > > >> >>> > > >>> >>> Thanks!
> > > >> >>> > > >>> >>>
> > > >> >>> > > >>> >>
> > > >> >>> > > >>> >>
> > > >> >>> > > >>>
> > > >> >>> > > >>>
> > > >> >>> > > >>
> > > >> >>> > >
> > > >> >>> > >
> > > >> >>> >
> > > >> >>>
> > > >> >>
> > > >> >>
> > > >>
> > > >>
> > >
> > >
> >
>
Re: Jobs Running with Single Mapper/Reducer
Posted by ShaoFeng Shi <sh...@gmail.com>.
I'm confused; You said these two cubes are exactly same in an early reply,
but now you mentioned the seoncd has more measures... they are NOT the
same; Different measures takes different storage spaces; What kind of the
measures defined in your former cube? Did you use DISTINCT COUNT measure?
2015-06-16 3:28 GMT+08:00 Vineet Mishra <cl...@gmail.com>:
> Hi Shi,
>
> Well yes these cubes are build for the same date range without having the
> data changed on the table and moreover hbase compression was not even
> touched so far.
>
> One interesting thing which I got was the later cube which was small in
> size and completing in lesser time was having multiple measures(around 7 or
> 8) in comparison to the former cube having just 1 or perhaps 2 measure with
> more time to complete.
>
> Running the same job again, it came up with some different size for the
> same data, so last time it was reduced 130Mb now its even less, around
> 123Mb. This change is observed within 4 hour of the running the last job
> with the same data set.
>
> I am not sure but how the kylin is internally building these cube dimension
> so as to inflate the data size by 3 and more over when the measures are
> very few as in comparison.
>
> Let me know if you could get anything out of it.
>
> Thanks!
>
> On Mon, Jun 15, 2015 at 8:51 PM, Shi, Shaofeng <sh...@ebay.com> wrote:
>
> > Just guess: Are these two cube builds cross the same date range? Was the
> > data on fact table changed between these two builds? Did HBase enable
> > compression recently?
> >
> > You can build once more with the same cube and date range to see whether
> > it can be re-produced; The cube size should be consistent in my
> knowledge;
> >
> > On 6/15/15, 10:52 PM, "Vineet Mishra" <cl...@gmail.com> wrote:
> >
> > >Thanks Shi,
> > >
> > >By the way I was curious as for the same table, dimension and metric, I
> > >could see the final cube size varying from 130 Mb to 420Mb.
> > >
> > >The cube is exactly the same as another one but when the cube build job
> is
> > >run I could see the cube size difference as mentioned below,
> > >
> > >*Earlier Cube Build*
> > >Size - 420 Mb
> > >Time Taken for building cube - 140 mins
> > >Source Records - ~11Million
> > >
> > >*Today's Cube Build*
> > >Size - 130 Mb
> > >Time Taken for building cube - 22 mins
> > >Source Records - ~11Million
> > >
> > >I don't find any reason to have three time cube size difference. Any
> ideas
> > >about this?
> > >
> > >Thanks!
> > >
> > >On Mon, Jun 15, 2015 at 8:08 PM, Shi, Shaofeng <sh...@ebay.com>
> wrote:
> > >
> > >> Good catch, and thanks for the sharing;
> > >>
> > >> On 6/15/15, 10:20 PM, "Vineet Mishra" <cl...@gmail.com> wrote:
> > >>
> > >> >Hi All,
> > >> >
> > >> >Well I got it through.
> > >> >
> > >> >Actually it was basically the pre-process which was converting my 3Gb
> > >>hive
> > >> >table to 60Mb sequence file which due to block size of 128Mb was
> > >>running
> > >> >in
> > >> >a single Mapper job.
> > >> >
> > >> >I changed the split size to 20Mb and it was able to spawn 3 Mapper
> > >>after
> > >> >that.
> > >> >
> > >> >Anyways, thanks all for your quick response.
> > >> >
> > >> >Thanks!
> > >> >
> > >> >On Mon, Jun 15, 2015 at 3:27 PM, Vineet Mishra <
> clearmidoubt@gmail.com
> > >
> > >> >wrote:
> > >> >
> > >> >> Hi,
> > >> >>
> > >> >> Sorry I misunderstood this value, actually 30Mb was the output for
> > >>that
> > >> >> job. But I guess I have the got reason, (not exactly confirmed) as
> > >>why
> > >> >>its
> > >> >> behaving like this, the input to this job is sequence file which is
> > >> >>60Mb in
> > >> >> size and compressed.
> > >> >>
> > >> >> No doubt when it will be passed to the mapper, it is very much
> > >>certain
> > >> >>to
> > >> >> have the single mapper invoked due to the pre-configured block
> size.
> > >> >>Now I
> > >> >> need to check If I can override the splitSize property for the job.
> > >> >>
> > >> >> Moreover I can see that the job is running seamlessly slow, kindly
> > >>find
> > >> >> the stack trace for the job mentioned below.
> > >> >>
> > >> >> 2015-06-13 10:08:00,940 INFO [main]
> org.apache.hadoop.mapred.MapTask:
> > >> >> Processing split: hdfs://
> > >> >>
> > >> >>
> > >>
> > >>
> >
> dev-hadoop-namenode.com:8020/tmp/kylin-9b675c66-4ce9-4a33-a356-2ccf9dbaca
> > >>
> >
> >>>>6a/kylin_intermediate_xyz_20150401000000_20150611000000_9b675c66_4ce9_4
> > >>>>a3
> > >> >>3_a356_2ccf9dbaca6a/000000_0:0+60440271
> > >> >> 2015-06-13 10:08:01,014 INFO [main]
> org.apache.hadoop.mapred.MapTask:
> > >> >> (EQUATOR) 0 kvi 67108860(268435440)
> > >> >> 2015-06-13 10:08:01,015 INFO [main]
> org.apache.hadoop.mapred.MapTask:
> > >> >> mapreduce.task.io.sort.mb: 256
> > >> >> 2015-06-13 10:08:01,015 INFO [main]
> org.apache.hadoop.mapred.MapTask:
> > >> >>soft
> > >> >> limit at 214748368
> > >> >> 2015-06-13 10:08:01,015 INFO [main]
> org.apache.hadoop.mapred.MapTask:
> > >> >> bufstart = 0; bufvoid = 268435456
> > >> >> 2015-06-13 10:08:01,015 INFO [main]
> org.apache.hadoop.mapred.MapTask:
> > >> >> kvstart = 67108860; length = 16777216
> > >> >> 2015-06-13 10:08:01,023 INFO [main]
> org.apache.hadoop.mapred.MapTask:
> > >> >>Map
> > >> >> output collector class =
> > >> >>org.apache.hadoop.mapred.MapTask$MapOutputBuffer
> > >> >> 2015-06-13 10:08:01,048 INFO [main]
> > >> >> org.apache.hadoop.io.compress.zlib.ZlibFactory: Successfully
> loaded &
> > >> >> initialized native-zlib library
> > >> >> 2015-06-13 10:08:01,048 INFO [main]
> > >> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
> > >> >> [.deflate]
> > >> >> 2015-06-13 10:08:01,052 INFO [main]
> > >> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
> > >> >> [.deflate]
> > >> >> 2015-06-13 10:08:01,052 INFO [main]
> > >> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
> > >> >> [.deflate]
> > >> >> 2015-06-13 10:08:01,053 INFO [main]
> > >> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
> > >> >> [.deflate]
> > >> >> 2015-06-13 10:08:01,058 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.AbstractHadoopJob: The absolute path
> for
> > >> >>meta
> > >> >> dir is
> > >> >>
> > >>
> >
> >>>>/yarn/nm/usercache/biops/appcache/application_1433833901375_0139/contai
> > >>>>ne
> > >> >>r_1433833901375_0139_01_000002/meta
> > >> >> 2015-06-13 10:08:01,059 INFO [main]
> > >>org.apache.kylin.common.KylinConfig:
> > >> >> Use
> > >> >>
> > >>
> >
> >>>>KYLIN_CONF=/yarn/nm/usercache/biops/appcache/application_1433833901375_
> > >>>>01
> > >> >>39/container_1433833901375_0139_01_000002/meta
> > >> >> 2015-06-13 10:08:01,084 INFO [main]
> > >>org.apache.kylin.cube.CubeManager:
> > >> >> Initializing CubeManager with config
> > >> >> /yarn/nm/usercache/biops/filecache/19452/meta
> > >> >> 2015-06-13 10:08:01,086 INFO [main]
> > >> >> org.apache.kylin.common.persistence.ResourceStore: Using metadata
> url
> > >> >> /yarn/nm/usercache/biops/filecache/19452/meta for resource store
> > >> >> 2015-06-13 10:08:01,327 INFO [main]
> > >> >>org.apache.kylin.cube.CubeDescManager:
> > >> >> Initializing CubeDescManager with config
> > >> >> /yarn/nm/usercache/biops/filecache/19452/meta
> > >> >> 2015-06-13 10:08:01,327 INFO [main]
> > >> >>org.apache.kylin.cube.CubeDescManager:
> > >> >> Reloading Cube Metadata from folder
> > >> >> /yarn/nm/usercache/biops/filecache/19452/meta/cube_desc
> > >> >> 2015-06-13 10:08:22,834 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 100000
> > >> >>records!
> > >> >> 2015-06-13 10:08:43,225 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 200000
> > >> >>records!
> > >> >> 2015-06-13 10:09:03,622 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 300000
> > >> >>records!
> > >> >> 2015-06-13 10:09:23,999 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 400000
> > >> >>records!
> > >> >> 2015-06-13 10:09:44,372 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 500000
> > >> >>records!
> > >> >> 2015-06-13 10:10:04,780 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 600000
> > >> >>records!
> > >> >> 2015-06-13 10:10:25,143 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 700000
> > >> >>records!
> > >> >> 2015-06-13 10:10:45,512 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 800000
> > >> >>records!
> > >> >> 2015-06-13 10:11:05,895 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 900000
> > >> >>records!
> > >> >> 2015-06-13 10:11:26,284 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1000000
> > >> >>records!
> > >> >> 2015-06-13 10:11:46,716 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1100000
> > >> >>records!
> > >> >> 2015-06-13 10:12:07,174 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1200000
> > >> >>records!
> > >> >> 2015-06-13 10:12:27,646 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1300000
> > >> >>records!
> > >> >> 2015-06-13 10:12:48,127 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1400000
> > >> >>records!
> > >> >> 2015-06-13 10:13:08,614 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1500000
> > >> >>records!
> > >> >> 2015-06-13 10:13:29,148 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1600000
> > >> >>records!
> > >> >> 2015-06-13 10:13:49,686 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1700000
> > >> >>records!
> > >> >> 2015-06-13 10:14:10,167 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1800000
> > >> >>records!
> > >> >> 2015-06-13 10:14:30,652 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1900000
> > >> >>records!
> > >> >> 2015-06-13 10:14:51,199 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2000000
> > >> >>records!
> > >> >> 2015-06-13 10:15:11,719 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2100000
> > >> >>records!
> > >> >> 2015-06-13 10:15:32,221 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2200000
> > >> >>records!
> > >> >> 2015-06-13 10:15:52,717 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2300000
> > >> >>records!
> > >> >> 2015-06-13 10:16:13,252 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2400000
> > >> >>records!
> > >> >> 2015-06-13 10:16:33,769 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2500000
> > >> >>records!
> > >> >> 2015-06-13 10:16:54,263 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2600000
> > >> >>records!
> > >> >> 2015-06-13 10:17:14,741 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2700000
> > >> >>records!
> > >> >> 2015-06-13 10:17:35,213 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2800000
> > >> >>records!
> > >> >> 2015-06-13 10:17:55,699 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2900000
> > >> >>records!
> > >> >> 2015-06-13 10:18:16,204 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3000000
> > >> >>records!
> > >> >> 2015-06-13 10:18:36,721 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3100000
> > >> >>records!
> > >> >> 2015-06-13 10:18:57,249 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3200000
> > >> >>records!
> > >> >> 2015-06-13 10:19:17,743 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3300000
> > >> >>records!
> > >> >> 2015-06-13 10:19:38,275 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3400000
> > >> >>records!
> > >> >> 2015-06-13 10:19:58,812 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3500000
> > >> >>records!
> > >> >> 2015-06-13 10:20:19,312 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3600000
> > >> >>records!
> > >> >> 2015-06-13 10:20:39,920 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3700000
> > >> >>records!
> > >> >> 2015-06-13 10:21:00,518 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3800000
> > >> >>records!
> > >> >> 2015-06-13 10:21:21,008 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3900000
> > >> >>records!
> > >> >> 2015-06-13 10:21:41,525 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4000000
> > >> >>records!
> > >> >> 2015-06-13 10:22:02,023 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4100000
> > >> >>records!
> > >> >> 2015-06-13 10:22:22,534 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4200000
> > >> >>records!
> > >> >> 2015-06-13 10:22:43,063 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4300000
> > >> >>records!
> > >> >> 2015-06-13 10:23:03,558 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4400000
> > >> >>records!
> > >> >> 2015-06-13 10:23:24,043 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4500000
> > >> >>records!
> > >> >> 2015-06-13 10:23:44,537 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4600000
> > >> >>records!
> > >> >> 2015-06-13 10:24:05,015 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4700000
> > >> >>records!
> > >> >> 2015-06-13 10:24:25,490 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4800000
> > >> >>records!
> > >> >> 2015-06-13 10:24:45,995 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4900000
> > >> >>records!
> > >> >> 2015-06-13 10:25:06,487 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5000000
> > >> >>records!
> > >> >> 2015-06-13 10:25:26,964 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5100000
> > >> >>records!
> > >> >> 2015-06-13 10:25:47,524 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5200000
> > >> >>records!
> > >> >> 2015-06-13 10:26:08,032 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5300000
> > >> >>records!
> > >> >> 2015-06-13 10:26:28,514 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5400000
> > >> >>records!
> > >> >> 2015-06-13 10:26:49,053 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5500000
> > >> >>records!
> > >> >> 2015-06-13 10:26:50,360 INFO [main]
> org.apache.hadoop.mapred.MapTask:
> > >> >> Spilling map output
> > >> >> 2015-06-13 10:26:50,360 INFO [main]
> org.apache.hadoop.mapred.MapTask:
> > >> >> bufstart = 0; bufend = 126646464; bufvoid = 268435456
> > >> >> 2015-06-13 10:26:50,360 INFO [main]
> org.apache.hadoop.mapred.MapTask:
> > >> >> kvstart = 67108860(268435440); kvend = 45083392(180333568); length
> =
> > >> >> 22025469/16777216
> > >> >> 2015-06-13 10:26:50,360 INFO [main]
> org.apache.hadoop.mapred.MapTask:
> > >> >> (EQUATOR) 148671936 kvi 37167980(148671920)
> > >> >> 2015-06-13 10:26:54,802 INFO [SpillThread]
> > >> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor
> > >> >>[.snappy]
> > >> >> 2015-06-13 10:26:54,883 INFO [SpillThread]
> > >> >> org.apache.kylin.job.hadoop.AbstractHadoopJob: The absolute path
> for
> > >> >>meta
> > >> >> dir is
> > >> >>
> > >>
> >
> >>>>/yarn/nm/usercache/biops/appcache/application_1433833901375_0139/contai
> > >>>>ne
> > >> >>r_1433833901375_0139_01_000002/meta
> > >> >> 2015-06-13 10:27:09,614 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5600000
> > >> >>records!
> > >> >> 2015-06-13 10:27:30,135 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5700000
> > >> >>records!
> > >> >> 2015-06-13 10:27:50,657 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5800000
> > >> >>records!
> > >> >> 2015-06-13 10:27:59,691 INFO [SpillThread]
> > >> >> org.apache.kylin.job.hadoop.cube.CuboidReducer: Handled 100000
> > >>records!
> > >> >> 2015-06-13 10:28:11,185 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5900000
> > >> >>records!
> > >> >> 2015-06-13 10:28:31,751 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6000000
> > >> >>records!
> > >> >> 2015-06-13 10:28:52,303 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6100000
> > >> >>records!
> > >> >> 2015-06-13 10:29:03,235 INFO [SpillThread]
> > >> >> org.apache.kylin.job.hadoop.cube.CuboidReducer: Handled 200000
> > >>records!
> > >> >> 2015-06-13 10:29:12,873 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6200000
> > >> >>records!
> > >> >> 2015-06-13 10:29:33,424 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6300000
> > >> >>records!
> > >> >> 2015-06-13 10:29:54,168 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6400000
> > >> >>records!
> > >> >> 2015-06-13 10:30:14,771 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6500000
> > >> >>records!
> > >> >> 2015-06-13 10:30:18,479 INFO [SpillThread]
> > >> >> org.apache.kylin.job.hadoop.cube.CuboidReducer: Handled 300000
> > >>records!
> > >> >> 2015-06-13 10:30:35,325 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6600000
> > >> >>records!
> > >> >> 2015-06-13 10:30:39,831 INFO [SpillThread]
> > >> >> org.apache.hadoop.mapred.MapTask: Finished spill 0
> > >> >> 2015-06-13 10:30:39,831 INFO [main]
> org.apache.hadoop.mapred.MapTask:
> > >> >> (RESET) equator 148671936 kv 37167980(148671920) kvi
> > >>32705668(130822672)
> > >> >> 2015-06-13 10:30:55,807 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6700000
> > >> >>records!
> > >> >> 2015-06-13 10:31:16,301 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6800000
> > >> >>records!
> > >> >> 2015-06-13 10:31:36,804 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6900000
> > >> >>records!
> > >> >> 2015-06-13 10:31:57,302 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7000000
> > >> >>records!
> > >> >> 2015-06-13 10:32:17,790 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7100000
> > >> >>records!
> > >> >> 2015-06-13 10:32:38,294 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7200000
> > >> >>records!
> > >> >> 2015-06-13 10:32:58,830 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7300000
> > >> >>records!
> > >> >> 2015-06-13 10:33:19,354 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7400000
> > >> >>records!
> > >> >> 2015-06-13 10:33:39,866 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7500000
> > >> >>records!
> > >> >> 2015-06-13 10:34:00,371 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7600000
> > >> >>records!
> > >> >> 2015-06-13 10:34:20,872 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7700000
> > >> >>records!
> > >> >> 2015-06-13 10:34:41,387 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7800000
> > >> >>records!
> > >> >> 2015-06-13 10:35:01,884 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7900000
> > >> >>records!
> > >> >> 2015-06-13 10:35:22,368 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8000000
> > >> >>records!
> > >> >> 2015-06-13 10:35:42,868 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8100000
> > >> >>records!
> > >> >> 2015-06-13 10:36:03,361 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8200000
> > >> >>records!
> > >> >> 2015-06-13 10:36:23,981 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8300000
> > >> >>records!
> > >> >> 2015-06-13 10:36:44,529 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8400000
> > >> >>records!
> > >> >> 2015-06-13 10:37:05,034 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8500000
> > >> >>records!
> > >> >> 2015-06-13 10:37:25,609 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8600000
> > >> >>records!
> > >> >> 2015-06-13 10:37:46,152 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8700000
> > >> >>records!
> > >> >> 2015-06-13 10:38:06,725 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8800000
> > >> >>records!
> > >> >> 2015-06-13 10:38:27,296 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8900000
> > >> >>records!
> > >> >> 2015-06-13 10:38:47,905 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9000000
> > >> >>records!
> > >> >> 2015-06-13 10:39:08,430 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9100000
> > >> >>records!
> > >> >> 2015-06-13 10:39:28,952 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9200000
> > >> >>records!
> > >> >> 2015-06-13 10:39:49,460 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9300000
> > >> >>records!
> > >> >> 2015-06-13 10:40:09,987 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9400000
> > >> >>records!
> > >> >> 2015-06-13 10:40:30,551 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9500000
> > >> >>records!
> > >> >> 2015-06-13 10:40:51,080 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9600000
> > >> >>records!
> > >> >> 2015-06-13 10:41:11,579 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9700000
> > >> >>records!
> > >> >> 2015-06-13 10:41:32,122 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9800000
> > >> >>records!
> > >> >> 2015-06-13 10:41:52,623 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9900000
> > >> >>records!
> > >> >> 2015-06-13 10:42:13,128 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10000000
> > >> >>records!
> > >> >> 2015-06-13 10:42:33,628 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10100000
> > >> >>records!
> > >> >> 2015-06-13 10:42:54,128 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10200000
> > >> >>records!
> > >> >> 2015-06-13 10:43:14,629 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10300000
> > >> >>records!
> > >> >> 2015-06-13 10:43:35,149 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10400000
> > >> >>records!
> > >> >> 2015-06-13 10:43:55,764 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10500000
> > >> >>records!
> > >> >> 2015-06-13 10:44:16,303 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10600000
> > >> >>records!
> > >> >> 2015-06-13 10:44:36,857 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10700000
> > >> >>records!
> > >> >> 2015-06-13 10:44:57,349 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10800000
> > >> >>records!
> > >> >> 2015-06-13 10:45:17,840 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10900000
> > >> >>records!
> > >> >> 2015-06-13 10:45:38,441 INFO [main]
> > >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 11000000
> > >> >>records!
> > >> >> 2015-06-13 10:45:41,057 INFO [main]
> org.apache.hadoop.mapred.MapTask:
> > >> >> Spilling map output
> > >> >> 2015-06-13 10:45:41,057 INFO [main]
> org.apache.hadoop.mapred.MapTask:
> > >> >> bufstart = 148671936; bufend = 6882957; bufvoid = 268435443
> > >> >> 2015-06-13 10:45:41,057 INFO [main]
> org.apache.hadoop.mapred.MapTask:
> > >> >> kvstart = 37167980(148671920); kvend = 15142512(60570048); length =
> > >> >> 22025469/16777216
> > >> >> 2015-06-13 10:45:41,057 INFO [main]
> org.apache.hadoop.mapred.MapTask:
> > >> >> (EQUATOR) 28908413 kvi 7227096(28908384)
> > >> >>
> > >> >> Thanks,
> > >> >>
> > >> >> On Mon, Jun 15, 2015 at 3:11 PM, 周千昊 <qh...@apache.org> wrote:
> > >> >>
> > >> >>> Hi Mishra
> > >> >>> It is as what you described, since the data size is 30MB
> only,
> > >> >>>hadoop
> > >> >>> will only manage to run MR in single mapper and reduce.
> > >> >>> As for your question about why it takes so long to run on a
> > >>small
> > >> >>> dataset, can you please dig into the web page of *map reduce task
> > >> >>>status
> > >> >>> *to
> > >> >>> check how much time the MR job really take, so that we can make
> > >>sure if
> > >> >>> the
> > >> >>> time is consuming on the MR job or kylin job scheduling module
> > >> >>>
> > >> >>> Vineet Mishra <cl...@gmail.com>于2015年6月15日周一 下午4:51写道：
> > >> >>>
> > >> >>> > Shi,
> > >> >>> >
> > >> >>> > Hadoop is setup correctly on my cluster with the default block
> > >>size
> > >> >>>of
> > >> >>> > 128Mb and its indeed very much running multiple Mapper/Reducer
> > >>based
> > >> >>> jobs
> > >> >>> > for other cases.
> > >> >>> >
> > >> >>> > Its the only Kylin Cube building which is running through single
> > >>M/R
> > >> >>> job.
> > >> >>> >
> > >> >>> > Moreover to my surprise, the 4th running job which is Build Base
> > >> >>>Cuboid
> > >> >>> > Data shows Data size as 30Mb, is it the reason due to which
> single
> > >> >>> mapper
> > >> >>> > is getting invoked, if that being the case then even to process
> > >>such
> > >> >>>a
> > >> >>> > small data set why is it taking around 50min.
> > >> >>> >
> > >> >>> > Thanks,
> > >> >>> >
> > >> >>> > On Mon, Jun 15, 2015 at 11:17 AM, Shi, Shaofeng <
> shaoshi@ebay.com
> > >
> > >> >>> wrote:
> > >> >>> >
> > >> >>> > > Yes you can adjust these parameters, for example give a
> smaller
> > >> >>>value
> > >> >>> for
> > >> >>> > > kylin.job.mapreduce.default.reduce.input.mb; but it only
> affects
> > >> >>>the
> > >> >>> > > reducer number;
> > >> >>> > >
> > >> >>> > > I suggest you investigate why there is only 1 mapper be
> started;
> > >> >>>Some
> > >> >>> > > factors like hadoop cluster size, HDFS file block size will
> > >>impact
> > >> >>> this;
> > >> >>> > > You can run a SQL (a query which need run MR, not simple
> select
> > >>*)
> > >> >>> with
> > >> >>> > > hive -e, and then use the MR job track URL to see how many
> > >>mappers
> > >> >>>be
> > >> >>> > > triggered; If it is still single, then the problem is in your
> > >> >>>hadoop
> > >> >>> > > configuration; Otherwise it may exists in Kylin, check if you
> > >>put
> > >> >>>some
> > >> >>> > > additional parameter in conf/kylin_job_conf.xml.
> > >> >>> > >
> > >> >>> > >
> > >> >>> > > On 6/15/15, 2:52 AM, "Vineet Mishra" <cl...@gmail.com>
> > >> >>>wrote:
> > >> >>> > >
> > >> >>> > > >Can I have specification for these properties?
> > >> >>> > > >
> > >> >>> > > >KYLIN_JOB_MAPREDUCE_DEFAULT_REDUCE_COUNT_RATIO =
> > >> >>> > > >"kylin.job.mapreduce.default.reduce.count.ratio";
> > >> >>> > > >KYLIN_JOB_MAPREDUCE_DEFAULT_REDUCE_INPUT_MB =
> > >> >>> > > >"kylin.job.mapreduce.default.reduce.input.mb";
> > >> >>> > > >KYLIN_JOB_MAPREDUCE_MAX_REDUCER_NUMBER =
> > >> >>> > > >"kylin.job.mapreduce.max.reducer.number";
> > >> >>> > > >
> > >> >>> > > >Thanks!
> > >> >>> > > >
> > >> >>> > > >On Sun, Jun 14, 2015 at 11:59 PM, Vineet Mishra <
> > >> >>> clearmidoubt@gmail.com
> > >> >>> > >
> > >> >>> > > >wrote:
> > >> >>> > > >
> > >> >>> > > >> Hi Shi,
> > >> >>> > > >>
> > >> >>> > > >> Its alright!
> > >> >>> > > >> So I was wondering my source hive Table is around 3 Gb,
> > >>despite
> > >> >>>of
> > >> >>> my
> > >> >>> > > >>hive
> > >> >>> > > >> table being partitioned and holding the data around 50-70
> Mb
> > >>per
> > >> >>> > > >>partition
> > >> >>> > > >> the Mapper and Reducer getting spawned are single. The
> > >>amount of
> > >> >>> data
> > >> >>> > > >>that
> > >> >>> > > >> is being processed in the M/R is nothing as expected but it
> > >> >>>takes
> > >> >>> hell
> > >> >>> > > >>lot
> > >> >>> > > >> of time.
> > >> >>> > > >>
> > >> >>> > > >> As mentioned in the trailing mail that the job is getting
> > >>very
> > >> >>> slow,
> > >> >>> > the
> > >> >>> > > >> process Build Base Cuboid Data itself takes around 50mins
> to
> > >>get
> > >> >>> > > >> completed.
> > >> >>> > > >>
> > >> >>> > > >> I can tweak the reducer parameter mentioned by you but do u
> > >> >>>think
> > >> >>> that
> > >> >>> > > >> will make a difference since the mapper is where the most
> of
> > >>the
> > >> >>> time
> > >> >>> > is
> > >> >>> > > >> spent.
> > >> >>> > > >>
> > >> >>> > > >> Can you share your thoughts for performance tuning for the
> > >>cube
> > >> >>> build!
> > >> >>> > > >>
> > >> >>> > > >> Thanks!
> > >> >>> > > >>
> > >> >>> > > >> On Sun, Jun 14, 2015 at 7:26 PM, Shi, Shaofeng
> > >> >>><sh...@ebay.com>
> > >> >>> > > wrote:
> > >> >>> > > >>
> > >> >>> > > >>> Hi, sorry, a busy weekend;
> > >> >>> > > >>>
> > >> >>> > > >>> Usually Kylin will request proper number of mapper and
> > >> >>>reducers;
> > >> >>> If
> > >> >>> > you
> > >> >>> > > >>> see single mapper/recudder, how much of your input and
> > >>output?
> > >> >>>If
> > >> >>> > your
> > >> >>> > > >>> cube is quite small, single mapper/reducer is possible;
> > >> >>> > > >>>
> > >> >>> > > >>> Number of mappers is decided by the FileInputFormat; But
> > >> >>>number of
> > >> >>> > > >>>reducer
> > >> >>> > > >>> was set by Kylin, see:
> > >> >>> > > >>>
> > >> >>> > > >>>
> > >> >>> > > >>>
> > >> >>> > >
> > >> >>>
> > >> >>>
> > >>
> > https://github.com/apache/incubator-kylin/blob/master/job/src/main/java/
> > >> >>> > > >>>org
> > >> >>> > > >>> /apache/kylin/job/hadoop/cube/CuboidJob.java#L141
> > >> >>> > > >>>
> > >> >>> > > >>><
> > >> >>> > >
> > >> >>>
> > >>
> https://github.com/apache/incubator-kylin/blob/master/job/src/main/java
> > >> >>> > > >>>/org/apache/kylin/job/hadoop/cube/CuboidJob.java#L141>
> > >> >>> > > >>>
> > >> >>> > > >>>
> > >> >>> > > >>>
> > >> >>> > > >>>
> > >> >>> > > >>> On 6/14/15, 5:25 PM, "Vineet Mishra"
> > >><cl...@gmail.com>
> > >> >>> wrote:
> > >> >>> > > >>>
> > >> >>> > > >>> >Urgent call, any follow up on this?
> > >> >>> > > >>> >
> > >> >>> > > >>> >On Fri, Jun 12, 2015 at 6:46 PM, Vineet Mishra
> > >> >>> > > >>><cl...@gmail.com>
> > >> >>> > > >>> >wrote:
> > >> >>> > > >>> >
> > >> >>> > > >>> >>
> > >> >>> > > >>> >> Why org.apache.kylin.job.hadoop.cube.CuboidReducer is
> > >> >>>running
> > >> >>> > Single
> > >> >>> > > >>> >> Mapper/Reducer for the job. Can I have the
> understanding
> > >> >>>behind
> > >> >>> > the
> > >> >>> > > >>> >>reason
> > >> >>> > > >>> >> of running it as single mapper/reducer.
> > >> >>> > > >>> >>
> > >> >>> > > >>> >> Thanks!
> > >> >>> > > >>> >>
> > >> >>> > > >>> >> On Fri, Jun 12, 2015 at 6:30 PM, Vineet Mishra
> > >> >>> > > >>><clearmidoubt@gmail.com
> > >> >>> > > >>> >
> > >> >>> > > >>> >> wrote:
> > >> >>> > > >>> >>
> > >> >>> > > >>> >>> Hi All,
> > >> >>> > > >>> >>>
> > >> >>> > > >>> >>> I am building a cube using Kylin and I could see that
> > >>the
> > >> >>>job
> > >> >>> is
> > >> >>> > > >>> >>>running
> > >> >>> > > >>> >>> with Single Mapper and Reducer for some of the
> > >>intermediate
> > >> >>> > process
> > >> >>> > > >>> >>>such as
> > >> >>> > > >>> >>>
> > >> >>> > > >>> >>> Extract Fact Table Distinct Columns
> > >> >>> > > >>> >>> Build Dimension Dictionary
> > >> >>> > > >>> >>> Build N-Dimension Cuboid
> > >> >>> > > >>> >>>
> > >> >>> > > >>> >>> I am not sure what's the reason behind running the job
> > >>with
> > >> >>> > single
> > >> >>> > > >>> M/R,
> > >> >>> > > >>> >>> is it really necessary or is it some default config.
> > >>which
> > >> >>> can be
> > >> >>> > > >>> >>>tweaked,
> > >> >>> > > >>> >>> its 70 Mins and the job status is 25% !
> > >> >>> > > >>> >>>
> > >> >>> > > >>> >>> Urgent Call!
> > >> >>> > > >>> >>>
> > >> >>> > > >>> >>> Thanks!
> > >> >>> > > >>> >>>
> > >> >>> > > >>> >>
> > >> >>> > > >>> >>
> > >> >>> > > >>>
> > >> >>> > > >>>
> > >> >>> > > >>
> > >> >>> > >
> > >> >>> > >
> > >> >>> >
> > >> >>>
> > >> >>
> > >> >>
> > >>
> > >>
> >
> >
>
Re: Jobs Running with Single Mapper/Reducer
Posted by Vineet Mishra <cl...@gmail.com>.
Hi Shi,
Well yes these cubes are build for the same date range without having the
data changed on the table and moreover hbase compression was not even
touched so far.
One interesting thing which I got was the later cube which was small in
size and completing in lesser time was having multiple measures(around 7 or
8) in comparison to the former cube having just 1 or perhaps 2 measure with
more time to complete.
Running the same job again, it came up with some different size for the
same data, so last time it was reduced 130Mb now its even less, around
123Mb. This change is observed within 4 hour of the running the last job
with the same data set.
I am not sure but how the kylin is internally building these cube dimension
so as to inflate the data size by 3 and more over when the measures are
very few as in comparison.
Let me know if you could get anything out of it.
Thanks!
On Mon, Jun 15, 2015 at 8:51 PM, Shi, Shaofeng <sh...@ebay.com> wrote:
> Just guess: Are these two cube builds cross the same date range? Was the
> data on fact table changed between these two builds? Did HBase enable
> compression recently?
>
> You can build once more with the same cube and date range to see whether
> it can be re-produced; The cube size should be consistent in my knowledge;
>
> On 6/15/15, 10:52 PM, "Vineet Mishra" <cl...@gmail.com> wrote:
>
> >Thanks Shi,
> >
> >By the way I was curious as for the same table, dimension and metric, I
> >could see the final cube size varying from 130 Mb to 420Mb.
> >
> >The cube is exactly the same as another one but when the cube build job is
> >run I could see the cube size difference as mentioned below,
> >
> >*Earlier Cube Build*
> >Size - 420 Mb
> >Time Taken for building cube - 140 mins
> >Source Records - ~11Million
> >
> >*Today's Cube Build*
> >Size - 130 Mb
> >Time Taken for building cube - 22 mins
> >Source Records - ~11Million
> >
> >I don't find any reason to have three time cube size difference. Any ideas
> >about this?
> >
> >Thanks!
> >
> >On Mon, Jun 15, 2015 at 8:08 PM, Shi, Shaofeng <sh...@ebay.com> wrote:
> >
> >> Good catch, and thanks for the sharing;
> >>
> >> On 6/15/15, 10:20 PM, "Vineet Mishra" <cl...@gmail.com> wrote:
> >>
> >> >Hi All,
> >> >
> >> >Well I got it through.
> >> >
> >> >Actually it was basically the pre-process which was converting my 3Gb
> >>hive
> >> >table to 60Mb sequence file which due to block size of 128Mb was
> >>running
> >> >in
> >> >a single Mapper job.
> >> >
> >> >I changed the split size to 20Mb and it was able to spawn 3 Mapper
> >>after
> >> >that.
> >> >
> >> >Anyways, thanks all for your quick response.
> >> >
> >> >Thanks!
> >> >
> >> >On Mon, Jun 15, 2015 at 3:27 PM, Vineet Mishra <clearmidoubt@gmail.com
> >
> >> >wrote:
> >> >
> >> >> Hi,
> >> >>
> >> >> Sorry I misunderstood this value, actually 30Mb was the output for
> >>that
> >> >> job. But I guess I have the got reason, (not exactly confirmed) as
> >>why
> >> >>its
> >> >> behaving like this, the input to this job is sequence file which is
> >> >>60Mb in
> >> >> size and compressed.
> >> >>
> >> >> No doubt when it will be passed to the mapper, it is very much
> >>certain
> >> >>to
> >> >> have the single mapper invoked due to the pre-configured block size.
> >> >>Now I
> >> >> need to check If I can override the splitSize property for the job.
> >> >>
> >> >> Moreover I can see that the job is running seamlessly slow, kindly
> >>find
> >> >> the stack trace for the job mentioned below.
> >> >>
> >> >> 2015-06-13 10:08:00,940 INFO [main] org.apache.hadoop.mapred.MapTask:
> >> >> Processing split: hdfs://
> >> >>
> >> >>
> >>
> >>
> dev-hadoop-namenode.com:8020/tmp/kylin-9b675c66-4ce9-4a33-a356-2ccf9dbaca
> >>
> >>>>6a/kylin_intermediate_xyz_20150401000000_20150611000000_9b675c66_4ce9_4
> >>>>a3
> >> >>3_a356_2ccf9dbaca6a/000000_0:0+60440271
> >> >> 2015-06-13 10:08:01,014 INFO [main] org.apache.hadoop.mapred.MapTask:
> >> >> (EQUATOR) 0 kvi 67108860(268435440)
> >> >> 2015-06-13 10:08:01,015 INFO [main] org.apache.hadoop.mapred.MapTask:
> >> >> mapreduce.task.io.sort.mb: 256
> >> >> 2015-06-13 10:08:01,015 INFO [main] org.apache.hadoop.mapred.MapTask:
> >> >>soft
> >> >> limit at 214748368
> >> >> 2015-06-13 10:08:01,015 INFO [main] org.apache.hadoop.mapred.MapTask:
> >> >> bufstart = 0; bufvoid = 268435456
> >> >> 2015-06-13 10:08:01,015 INFO [main] org.apache.hadoop.mapred.MapTask:
> >> >> kvstart = 67108860; length = 16777216
> >> >> 2015-06-13 10:08:01,023 INFO [main] org.apache.hadoop.mapred.MapTask:
> >> >>Map
> >> >> output collector class =
> >> >>org.apache.hadoop.mapred.MapTask$MapOutputBuffer
> >> >> 2015-06-13 10:08:01,048 INFO [main]
> >> >> org.apache.hadoop.io.compress.zlib.ZlibFactory: Successfully loaded &
> >> >> initialized native-zlib library
> >> >> 2015-06-13 10:08:01,048 INFO [main]
> >> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
> >> >> [.deflate]
> >> >> 2015-06-13 10:08:01,052 INFO [main]
> >> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
> >> >> [.deflate]
> >> >> 2015-06-13 10:08:01,052 INFO [main]
> >> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
> >> >> [.deflate]
> >> >> 2015-06-13 10:08:01,053 INFO [main]
> >> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
> >> >> [.deflate]
> >> >> 2015-06-13 10:08:01,058 INFO [main]
> >> >> org.apache.kylin.job.hadoop.AbstractHadoopJob: The absolute path for
> >> >>meta
> >> >> dir is
> >> >>
> >>
> >>>>/yarn/nm/usercache/biops/appcache/application_1433833901375_0139/contai
> >>>>ne
> >> >>r_1433833901375_0139_01_000002/meta
> >> >> 2015-06-13 10:08:01,059 INFO [main]
> >>org.apache.kylin.common.KylinConfig:
> >> >> Use
> >> >>
> >>
> >>>>KYLIN_CONF=/yarn/nm/usercache/biops/appcache/application_1433833901375_
> >>>>01
> >> >>39/container_1433833901375_0139_01_000002/meta
> >> >> 2015-06-13 10:08:01,084 INFO [main]
> >>org.apache.kylin.cube.CubeManager:
> >> >> Initializing CubeManager with config
> >> >> /yarn/nm/usercache/biops/filecache/19452/meta
> >> >> 2015-06-13 10:08:01,086 INFO [main]
> >> >> org.apache.kylin.common.persistence.ResourceStore: Using metadata url
> >> >> /yarn/nm/usercache/biops/filecache/19452/meta for resource store
> >> >> 2015-06-13 10:08:01,327 INFO [main]
> >> >>org.apache.kylin.cube.CubeDescManager:
> >> >> Initializing CubeDescManager with config
> >> >> /yarn/nm/usercache/biops/filecache/19452/meta
> >> >> 2015-06-13 10:08:01,327 INFO [main]
> >> >>org.apache.kylin.cube.CubeDescManager:
> >> >> Reloading Cube Metadata from folder
> >> >> /yarn/nm/usercache/biops/filecache/19452/meta/cube_desc
> >> >> 2015-06-13 10:08:22,834 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 100000
> >> >>records!
> >> >> 2015-06-13 10:08:43,225 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 200000
> >> >>records!
> >> >> 2015-06-13 10:09:03,622 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 300000
> >> >>records!
> >> >> 2015-06-13 10:09:23,999 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 400000
> >> >>records!
> >> >> 2015-06-13 10:09:44,372 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 500000
> >> >>records!
> >> >> 2015-06-13 10:10:04,780 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 600000
> >> >>records!
> >> >> 2015-06-13 10:10:25,143 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 700000
> >> >>records!
> >> >> 2015-06-13 10:10:45,512 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 800000
> >> >>records!
> >> >> 2015-06-13 10:11:05,895 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 900000
> >> >>records!
> >> >> 2015-06-13 10:11:26,284 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1000000
> >> >>records!
> >> >> 2015-06-13 10:11:46,716 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1100000
> >> >>records!
> >> >> 2015-06-13 10:12:07,174 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1200000
> >> >>records!
> >> >> 2015-06-13 10:12:27,646 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1300000
> >> >>records!
> >> >> 2015-06-13 10:12:48,127 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1400000
> >> >>records!
> >> >> 2015-06-13 10:13:08,614 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1500000
> >> >>records!
> >> >> 2015-06-13 10:13:29,148 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1600000
> >> >>records!
> >> >> 2015-06-13 10:13:49,686 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1700000
> >> >>records!
> >> >> 2015-06-13 10:14:10,167 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1800000
> >> >>records!
> >> >> 2015-06-13 10:14:30,652 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1900000
> >> >>records!
> >> >> 2015-06-13 10:14:51,199 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2000000
> >> >>records!
> >> >> 2015-06-13 10:15:11,719 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2100000
> >> >>records!
> >> >> 2015-06-13 10:15:32,221 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2200000
> >> >>records!
> >> >> 2015-06-13 10:15:52,717 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2300000
> >> >>records!
> >> >> 2015-06-13 10:16:13,252 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2400000
> >> >>records!
> >> >> 2015-06-13 10:16:33,769 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2500000
> >> >>records!
> >> >> 2015-06-13 10:16:54,263 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2600000
> >> >>records!
> >> >> 2015-06-13 10:17:14,741 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2700000
> >> >>records!
> >> >> 2015-06-13 10:17:35,213 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2800000
> >> >>records!
> >> >> 2015-06-13 10:17:55,699 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2900000
> >> >>records!
> >> >> 2015-06-13 10:18:16,204 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3000000
> >> >>records!
> >> >> 2015-06-13 10:18:36,721 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3100000
> >> >>records!
> >> >> 2015-06-13 10:18:57,249 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3200000
> >> >>records!
> >> >> 2015-06-13 10:19:17,743 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3300000
> >> >>records!
> >> >> 2015-06-13 10:19:38,275 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3400000
> >> >>records!
> >> >> 2015-06-13 10:19:58,812 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3500000
> >> >>records!
> >> >> 2015-06-13 10:20:19,312 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3600000
> >> >>records!
> >> >> 2015-06-13 10:20:39,920 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3700000
> >> >>records!
> >> >> 2015-06-13 10:21:00,518 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3800000
> >> >>records!
> >> >> 2015-06-13 10:21:21,008 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3900000
> >> >>records!
> >> >> 2015-06-13 10:21:41,525 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4000000
> >> >>records!
> >> >> 2015-06-13 10:22:02,023 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4100000
> >> >>records!
> >> >> 2015-06-13 10:22:22,534 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4200000
> >> >>records!
> >> >> 2015-06-13 10:22:43,063 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4300000
> >> >>records!
> >> >> 2015-06-13 10:23:03,558 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4400000
> >> >>records!
> >> >> 2015-06-13 10:23:24,043 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4500000
> >> >>records!
> >> >> 2015-06-13 10:23:44,537 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4600000
> >> >>records!
> >> >> 2015-06-13 10:24:05,015 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4700000
> >> >>records!
> >> >> 2015-06-13 10:24:25,490 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4800000
> >> >>records!
> >> >> 2015-06-13 10:24:45,995 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4900000
> >> >>records!
> >> >> 2015-06-13 10:25:06,487 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5000000
> >> >>records!
> >> >> 2015-06-13 10:25:26,964 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5100000
> >> >>records!
> >> >> 2015-06-13 10:25:47,524 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5200000
> >> >>records!
> >> >> 2015-06-13 10:26:08,032 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5300000
> >> >>records!
> >> >> 2015-06-13 10:26:28,514 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5400000
> >> >>records!
> >> >> 2015-06-13 10:26:49,053 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5500000
> >> >>records!
> >> >> 2015-06-13 10:26:50,360 INFO [main] org.apache.hadoop.mapred.MapTask:
> >> >> Spilling map output
> >> >> 2015-06-13 10:26:50,360 INFO [main] org.apache.hadoop.mapred.MapTask:
> >> >> bufstart = 0; bufend = 126646464; bufvoid = 268435456
> >> >> 2015-06-13 10:26:50,360 INFO [main] org.apache.hadoop.mapred.MapTask:
> >> >> kvstart = 67108860(268435440); kvend = 45083392(180333568); length =
> >> >> 22025469/16777216
> >> >> 2015-06-13 10:26:50,360 INFO [main] org.apache.hadoop.mapred.MapTask:
> >> >> (EQUATOR) 148671936 kvi 37167980(148671920)
> >> >> 2015-06-13 10:26:54,802 INFO [SpillThread]
> >> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor
> >> >>[.snappy]
> >> >> 2015-06-13 10:26:54,883 INFO [SpillThread]
> >> >> org.apache.kylin.job.hadoop.AbstractHadoopJob: The absolute path for
> >> >>meta
> >> >> dir is
> >> >>
> >>
> >>>>/yarn/nm/usercache/biops/appcache/application_1433833901375_0139/contai
> >>>>ne
> >> >>r_1433833901375_0139_01_000002/meta
> >> >> 2015-06-13 10:27:09,614 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5600000
> >> >>records!
> >> >> 2015-06-13 10:27:30,135 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5700000
> >> >>records!
> >> >> 2015-06-13 10:27:50,657 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5800000
> >> >>records!
> >> >> 2015-06-13 10:27:59,691 INFO [SpillThread]
> >> >> org.apache.kylin.job.hadoop.cube.CuboidReducer: Handled 100000
> >>records!
> >> >> 2015-06-13 10:28:11,185 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5900000
> >> >>records!
> >> >> 2015-06-13 10:28:31,751 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6000000
> >> >>records!
> >> >> 2015-06-13 10:28:52,303 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6100000
> >> >>records!
> >> >> 2015-06-13 10:29:03,235 INFO [SpillThread]
> >> >> org.apache.kylin.job.hadoop.cube.CuboidReducer: Handled 200000
> >>records!
> >> >> 2015-06-13 10:29:12,873 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6200000
> >> >>records!
> >> >> 2015-06-13 10:29:33,424 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6300000
> >> >>records!
> >> >> 2015-06-13 10:29:54,168 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6400000
> >> >>records!
> >> >> 2015-06-13 10:30:14,771 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6500000
> >> >>records!
> >> >> 2015-06-13 10:30:18,479 INFO [SpillThread]
> >> >> org.apache.kylin.job.hadoop.cube.CuboidReducer: Handled 300000
> >>records!
> >> >> 2015-06-13 10:30:35,325 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6600000
> >> >>records!
> >> >> 2015-06-13 10:30:39,831 INFO [SpillThread]
> >> >> org.apache.hadoop.mapred.MapTask: Finished spill 0
> >> >> 2015-06-13 10:30:39,831 INFO [main] org.apache.hadoop.mapred.MapTask:
> >> >> (RESET) equator 148671936 kv 37167980(148671920) kvi
> >>32705668(130822672)
> >> >> 2015-06-13 10:30:55,807 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6700000
> >> >>records!
> >> >> 2015-06-13 10:31:16,301 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6800000
> >> >>records!
> >> >> 2015-06-13 10:31:36,804 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6900000
> >> >>records!
> >> >> 2015-06-13 10:31:57,302 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7000000
> >> >>records!
> >> >> 2015-06-13 10:32:17,790 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7100000
> >> >>records!
> >> >> 2015-06-13 10:32:38,294 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7200000
> >> >>records!
> >> >> 2015-06-13 10:32:58,830 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7300000
> >> >>records!
> >> >> 2015-06-13 10:33:19,354 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7400000
> >> >>records!
> >> >> 2015-06-13 10:33:39,866 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7500000
> >> >>records!
> >> >> 2015-06-13 10:34:00,371 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7600000
> >> >>records!
> >> >> 2015-06-13 10:34:20,872 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7700000
> >> >>records!
> >> >> 2015-06-13 10:34:41,387 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7800000
> >> >>records!
> >> >> 2015-06-13 10:35:01,884 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7900000
> >> >>records!
> >> >> 2015-06-13 10:35:22,368 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8000000
> >> >>records!
> >> >> 2015-06-13 10:35:42,868 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8100000
> >> >>records!
> >> >> 2015-06-13 10:36:03,361 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8200000
> >> >>records!
> >> >> 2015-06-13 10:36:23,981 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8300000
> >> >>records!
> >> >> 2015-06-13 10:36:44,529 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8400000
> >> >>records!
> >> >> 2015-06-13 10:37:05,034 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8500000
> >> >>records!
> >> >> 2015-06-13 10:37:25,609 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8600000
> >> >>records!
> >> >> 2015-06-13 10:37:46,152 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8700000
> >> >>records!
> >> >> 2015-06-13 10:38:06,725 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8800000
> >> >>records!
> >> >> 2015-06-13 10:38:27,296 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8900000
> >> >>records!
> >> >> 2015-06-13 10:38:47,905 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9000000
> >> >>records!
> >> >> 2015-06-13 10:39:08,430 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9100000
> >> >>records!
> >> >> 2015-06-13 10:39:28,952 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9200000
> >> >>records!
> >> >> 2015-06-13 10:39:49,460 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9300000
> >> >>records!
> >> >> 2015-06-13 10:40:09,987 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9400000
> >> >>records!
> >> >> 2015-06-13 10:40:30,551 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9500000
> >> >>records!
> >> >> 2015-06-13 10:40:51,080 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9600000
> >> >>records!
> >> >> 2015-06-13 10:41:11,579 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9700000
> >> >>records!
> >> >> 2015-06-13 10:41:32,122 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9800000
> >> >>records!
> >> >> 2015-06-13 10:41:52,623 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9900000
> >> >>records!
> >> >> 2015-06-13 10:42:13,128 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10000000
> >> >>records!
> >> >> 2015-06-13 10:42:33,628 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10100000
> >> >>records!
> >> >> 2015-06-13 10:42:54,128 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10200000
> >> >>records!
> >> >> 2015-06-13 10:43:14,629 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10300000
> >> >>records!
> >> >> 2015-06-13 10:43:35,149 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10400000
> >> >>records!
> >> >> 2015-06-13 10:43:55,764 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10500000
> >> >>records!
> >> >> 2015-06-13 10:44:16,303 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10600000
> >> >>records!
> >> >> 2015-06-13 10:44:36,857 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10700000
> >> >>records!
> >> >> 2015-06-13 10:44:57,349 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10800000
> >> >>records!
> >> >> 2015-06-13 10:45:17,840 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10900000
> >> >>records!
> >> >> 2015-06-13 10:45:38,441 INFO [main]
> >> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 11000000
> >> >>records!
> >> >> 2015-06-13 10:45:41,057 INFO [main] org.apache.hadoop.mapred.MapTask:
> >> >> Spilling map output
> >> >> 2015-06-13 10:45:41,057 INFO [main] org.apache.hadoop.mapred.MapTask:
> >> >> bufstart = 148671936; bufend = 6882957; bufvoid = 268435443
> >> >> 2015-06-13 10:45:41,057 INFO [main] org.apache.hadoop.mapred.MapTask:
> >> >> kvstart = 37167980(148671920); kvend = 15142512(60570048); length =
> >> >> 22025469/16777216
> >> >> 2015-06-13 10:45:41,057 INFO [main] org.apache.hadoop.mapred.MapTask:
> >> >> (EQUATOR) 28908413 kvi 7227096(28908384)
> >> >>
> >> >> Thanks,
> >> >>
> >> >> On Mon, Jun 15, 2015 at 3:11 PM, 周千昊 <qh...@apache.org> wrote:
> >> >>
> >> >>> Hi Mishra
> >> >>> It is as what you described, since the data size is 30MB only,
> >> >>>hadoop
> >> >>> will only manage to run MR in single mapper and reduce.
> >> >>> As for your question about why it takes so long to run on a
> >>small
> >> >>> dataset, can you please dig into the web page of *map reduce task
> >> >>>status
> >> >>> *to
> >> >>> check how much time the MR job really take, so that we can make
> >>sure if
> >> >>> the
> >> >>> time is consuming on the MR job or kylin job scheduling module
> >> >>>
> >> >>> Vineet Mishra <cl...@gmail.com>于2015年6月15日周一 下午4:51写道：
> >> >>>
> >> >>> > Shi,
> >> >>> >
> >> >>> > Hadoop is setup correctly on my cluster with the default block
> >>size
> >> >>>of
> >> >>> > 128Mb and its indeed very much running multiple Mapper/Reducer
> >>based
> >> >>> jobs
> >> >>> > for other cases.
> >> >>> >
> >> >>> > Its the only Kylin Cube building which is running through single
> >>M/R
> >> >>> job.
> >> >>> >
> >> >>> > Moreover to my surprise, the 4th running job which is Build Base
> >> >>>Cuboid
> >> >>> > Data shows Data size as 30Mb, is it the reason due to which single
> >> >>> mapper
> >> >>> > is getting invoked, if that being the case then even to process
> >>such
> >> >>>a
> >> >>> > small data set why is it taking around 50min.
> >> >>> >
> >> >>> > Thanks,
> >> >>> >
> >> >>> > On Mon, Jun 15, 2015 at 11:17 AM, Shi, Shaofeng <shaoshi@ebay.com
> >
> >> >>> wrote:
> >> >>> >
> >> >>> > > Yes you can adjust these parameters, for example give a smaller
> >> >>>value
> >> >>> for
> >> >>> > > kylin.job.mapreduce.default.reduce.input.mb; but it only affects
> >> >>>the
> >> >>> > > reducer number;
> >> >>> > >
> >> >>> > > I suggest you investigate why there is only 1 mapper be started;
> >> >>>Some
> >> >>> > > factors like hadoop cluster size, HDFS file block size will
> >>impact
> >> >>> this;
> >> >>> > > You can run a SQL (a query which need run MR, not simple select
> >>*)
> >> >>> with
> >> >>> > > hive -e, and then use the MR job track URL to see how many
> >>mappers
> >> >>>be
> >> >>> > > triggered; If it is still single, then the problem is in your
> >> >>>hadoop
> >> >>> > > configuration; Otherwise it may exists in Kylin, check if you
> >>put
> >> >>>some
> >> >>> > > additional parameter in conf/kylin_job_conf.xml.
> >> >>> > >
> >> >>> > >
> >> >>> > > On 6/15/15, 2:52 AM, "Vineet Mishra" <cl...@gmail.com>
> >> >>>wrote:
> >> >>> > >
> >> >>> > > >Can I have specification for these properties?
> >> >>> > > >
> >> >>> > > >KYLIN_JOB_MAPREDUCE_DEFAULT_REDUCE_COUNT_RATIO =
> >> >>> > > >"kylin.job.mapreduce.default.reduce.count.ratio";
> >> >>> > > >KYLIN_JOB_MAPREDUCE_DEFAULT_REDUCE_INPUT_MB =
> >> >>> > > >"kylin.job.mapreduce.default.reduce.input.mb";
> >> >>> > > >KYLIN_JOB_MAPREDUCE_MAX_REDUCER_NUMBER =
> >> >>> > > >"kylin.job.mapreduce.max.reducer.number";
> >> >>> > > >
> >> >>> > > >Thanks!
> >> >>> > > >
> >> >>> > > >On Sun, Jun 14, 2015 at 11:59 PM, Vineet Mishra <
> >> >>> clearmidoubt@gmail.com
> >> >>> > >
> >> >>> > > >wrote:
> >> >>> > > >
> >> >>> > > >> Hi Shi,
> >> >>> > > >>
> >> >>> > > >> Its alright!
> >> >>> > > >> So I was wondering my source hive Table is around 3 Gb,
> >>despite
> >> >>>of
> >> >>> my
> >> >>> > > >>hive
> >> >>> > > >> table being partitioned and holding the data around 50-70 Mb
> >>per
> >> >>> > > >>partition
> >> >>> > > >> the Mapper and Reducer getting spawned are single. The
> >>amount of
> >> >>> data
> >> >>> > > >>that
> >> >>> > > >> is being processed in the M/R is nothing as expected but it
> >> >>>takes
> >> >>> hell
> >> >>> > > >>lot
> >> >>> > > >> of time.
> >> >>> > > >>
> >> >>> > > >> As mentioned in the trailing mail that the job is getting
> >>very
> >> >>> slow,
> >> >>> > the
> >> >>> > > >> process Build Base Cuboid Data itself takes around 50mins to
> >>get
> >> >>> > > >> completed.
> >> >>> > > >>
> >> >>> > > >> I can tweak the reducer parameter mentioned by you but do u
> >> >>>think
> >> >>> that
> >> >>> > > >> will make a difference since the mapper is where the most of
> >>the
> >> >>> time
> >> >>> > is
> >> >>> > > >> spent.
> >> >>> > > >>
> >> >>> > > >> Can you share your thoughts for performance tuning for the
> >>cube
> >> >>> build!
> >> >>> > > >>
> >> >>> > > >> Thanks!
> >> >>> > > >>
> >> >>> > > >> On Sun, Jun 14, 2015 at 7:26 PM, Shi, Shaofeng
> >> >>><sh...@ebay.com>
> >> >>> > > wrote:
> >> >>> > > >>
> >> >>> > > >>> Hi, sorry, a busy weekend;
> >> >>> > > >>>
> >> >>> > > >>> Usually Kylin will request proper number of mapper and
> >> >>>reducers;
> >> >>> If
> >> >>> > you
> >> >>> > > >>> see single mapper/recudder, how much of your input and
> >>output?
> >> >>>If
> >> >>> > your
> >> >>> > > >>> cube is quite small, single mapper/reducer is possible;
> >> >>> > > >>>
> >> >>> > > >>> Number of mappers is decided by the FileInputFormat; But
> >> >>>number of
> >> >>> > > >>>reducer
> >> >>> > > >>> was set by Kylin, see:
> >> >>> > > >>>
> >> >>> > > >>>
> >> >>> > > >>>
> >> >>> > >
> >> >>>
> >> >>>
> >>
> https://github.com/apache/incubator-kylin/blob/master/job/src/main/java/
> >> >>> > > >>>org
> >> >>> > > >>> /apache/kylin/job/hadoop/cube/CuboidJob.java#L141
> >> >>> > > >>>
> >> >>> > > >>><
> >> >>> > >
> >> >>>
> >> https://github.com/apache/incubator-kylin/blob/master/job/src/main/java
> >> >>> > > >>>/org/apache/kylin/job/hadoop/cube/CuboidJob.java#L141>
> >> >>> > > >>>
> >> >>> > > >>>
> >> >>> > > >>>
> >> >>> > > >>>
> >> >>> > > >>> On 6/14/15, 5:25 PM, "Vineet Mishra"
> >><cl...@gmail.com>
> >> >>> wrote:
> >> >>> > > >>>
> >> >>> > > >>> >Urgent call, any follow up on this?
> >> >>> > > >>> >
> >> >>> > > >>> >On Fri, Jun 12, 2015 at 6:46 PM, Vineet Mishra
> >> >>> > > >>><cl...@gmail.com>
> >> >>> > > >>> >wrote:
> >> >>> > > >>> >
> >> >>> > > >>> >>
> >> >>> > > >>> >> Why org.apache.kylin.job.hadoop.cube.CuboidReducer is
> >> >>>running
> >> >>> > Single
> >> >>> > > >>> >> Mapper/Reducer for the job. Can I have the understanding
> >> >>>behind
> >> >>> > the
> >> >>> > > >>> >>reason
> >> >>> > > >>> >> of running it as single mapper/reducer.
> >> >>> > > >>> >>
> >> >>> > > >>> >> Thanks!
> >> >>> > > >>> >>
> >> >>> > > >>> >> On Fri, Jun 12, 2015 at 6:30 PM, Vineet Mishra
> >> >>> > > >>><clearmidoubt@gmail.com
> >> >>> > > >>> >
> >> >>> > > >>> >> wrote:
> >> >>> > > >>> >>
> >> >>> > > >>> >>> Hi All,
> >> >>> > > >>> >>>
> >> >>> > > >>> >>> I am building a cube using Kylin and I could see that
> >>the
> >> >>>job
> >> >>> is
> >> >>> > > >>> >>>running
> >> >>> > > >>> >>> with Single Mapper and Reducer for some of the
> >>intermediate
> >> >>> > process
> >> >>> > > >>> >>>such as
> >> >>> > > >>> >>>
> >> >>> > > >>> >>> Extract Fact Table Distinct Columns
> >> >>> > > >>> >>> Build Dimension Dictionary
> >> >>> > > >>> >>> Build N-Dimension Cuboid
> >> >>> > > >>> >>>
> >> >>> > > >>> >>> I am not sure what's the reason behind running the job
> >>with
> >> >>> > single
> >> >>> > > >>> M/R,
> >> >>> > > >>> >>> is it really necessary or is it some default config.
> >>which
> >> >>> can be
> >> >>> > > >>> >>>tweaked,
> >> >>> > > >>> >>> its 70 Mins and the job status is 25% !
> >> >>> > > >>> >>>
> >> >>> > > >>> >>> Urgent Call!
> >> >>> > > >>> >>>
> >> >>> > > >>> >>> Thanks!
> >> >>> > > >>> >>>
> >> >>> > > >>> >>
> >> >>> > > >>> >>
> >> >>> > > >>>
> >> >>> > > >>>
> >> >>> > > >>
> >> >>> > >
> >> >>> > >
> >> >>> >
> >> >>>
> >> >>
> >> >>
> >>
> >>
>
>
Re: Jobs Running with Single Mapper/Reducer
Posted by "Shi, Shaofeng" <sh...@ebay.com>.
Just guess: Are these two cube builds cross the same date range? Was the
data on fact table changed between these two builds? Did HBase enable
compression recently?
You can build once more with the same cube and date range to see whether
it can be re-produced; The cube size should be consistent in my knowledge;
On 6/15/15, 10:52 PM, "Vineet Mishra" <cl...@gmail.com> wrote:
>Thanks Shi,
>
>By the way I was curious as for the same table, dimension and metric, I
>could see the final cube size varying from 130 Mb to 420Mb.
>
>The cube is exactly the same as another one but when the cube build job is
>run I could see the cube size difference as mentioned below,
>
>*Earlier Cube Build*
>Size - 420 Mb
>Time Taken for building cube - 140 mins
>Source Records - ~11Million
>
>*Today's Cube Build*
>Size - 130 Mb
>Time Taken for building cube - 22 mins
>Source Records - ~11Million
>
>I don't find any reason to have three time cube size difference. Any ideas
>about this?
>
>Thanks!
>
>On Mon, Jun 15, 2015 at 8:08 PM, Shi, Shaofeng <sh...@ebay.com> wrote:
>
>> Good catch, and thanks for the sharing;
>>
>> On 6/15/15, 10:20 PM, "Vineet Mishra" <cl...@gmail.com> wrote:
>>
>> >Hi All,
>> >
>> >Well I got it through.
>> >
>> >Actually it was basically the pre-process which was converting my 3Gb
>>hive
>> >table to 60Mb sequence file which due to block size of 128Mb was
>>running
>> >in
>> >a single Mapper job.
>> >
>> >I changed the split size to 20Mb and it was able to spawn 3 Mapper
>>after
>> >that.
>> >
>> >Anyways, thanks all for your quick response.
>> >
>> >Thanks!
>> >
>> >On Mon, Jun 15, 2015 at 3:27 PM, Vineet Mishra <cl...@gmail.com>
>> >wrote:
>> >
>> >> Hi,
>> >>
>> >> Sorry I misunderstood this value, actually 30Mb was the output for
>>that
>> >> job. But I guess I have the got reason, (not exactly confirmed) as
>>why
>> >>its
>> >> behaving like this, the input to this job is sequence file which is
>> >>60Mb in
>> >> size and compressed.
>> >>
>> >> No doubt when it will be passed to the mapper, it is very much
>>certain
>> >>to
>> >> have the single mapper invoked due to the pre-configured block size.
>> >>Now I
>> >> need to check If I can override the splitSize property for the job.
>> >>
>> >> Moreover I can see that the job is running seamlessly slow, kindly
>>find
>> >> the stack trace for the job mentioned below.
>> >>
>> >> 2015-06-13 10:08:00,940 INFO [main] org.apache.hadoop.mapred.MapTask:
>> >> Processing split: hdfs://
>> >>
>> >>
>>
>>dev-hadoop-namenode.com:8020/tmp/kylin-9b675c66-4ce9-4a33-a356-2ccf9dbaca
>>
>>>>6a/kylin_intermediate_xyz_20150401000000_20150611000000_9b675c66_4ce9_4
>>>>a3
>> >>3_a356_2ccf9dbaca6a/000000_0:0+60440271
>> >> 2015-06-13 10:08:01,014 INFO [main] org.apache.hadoop.mapred.MapTask:
>> >> (EQUATOR) 0 kvi 67108860(268435440)
>> >> 2015-06-13 10:08:01,015 INFO [main] org.apache.hadoop.mapred.MapTask:
>> >> mapreduce.task.io.sort.mb: 256
>> >> 2015-06-13 10:08:01,015 INFO [main] org.apache.hadoop.mapred.MapTask:
>> >>soft
>> >> limit at 214748368
>> >> 2015-06-13 10:08:01,015 INFO [main] org.apache.hadoop.mapred.MapTask:
>> >> bufstart = 0; bufvoid = 268435456
>> >> 2015-06-13 10:08:01,015 INFO [main] org.apache.hadoop.mapred.MapTask:
>> >> kvstart = 67108860; length = 16777216
>> >> 2015-06-13 10:08:01,023 INFO [main] org.apache.hadoop.mapred.MapTask:
>> >>Map
>> >> output collector class =
>> >>org.apache.hadoop.mapred.MapTask$MapOutputBuffer
>> >> 2015-06-13 10:08:01,048 INFO [main]
>> >> org.apache.hadoop.io.compress.zlib.ZlibFactory: Successfully loaded &
>> >> initialized native-zlib library
>> >> 2015-06-13 10:08:01,048 INFO [main]
>> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
>> >> [.deflate]
>> >> 2015-06-13 10:08:01,052 INFO [main]
>> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
>> >> [.deflate]
>> >> 2015-06-13 10:08:01,052 INFO [main]
>> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
>> >> [.deflate]
>> >> 2015-06-13 10:08:01,053 INFO [main]
>> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
>> >> [.deflate]
>> >> 2015-06-13 10:08:01,058 INFO [main]
>> >> org.apache.kylin.job.hadoop.AbstractHadoopJob: The absolute path for
>> >>meta
>> >> dir is
>> >>
>>
>>>>/yarn/nm/usercache/biops/appcache/application_1433833901375_0139/contai
>>>>ne
>> >>r_1433833901375_0139_01_000002/meta
>> >> 2015-06-13 10:08:01,059 INFO [main]
>>org.apache.kylin.common.KylinConfig:
>> >> Use
>> >>
>>
>>>>KYLIN_CONF=/yarn/nm/usercache/biops/appcache/application_1433833901375_
>>>>01
>> >>39/container_1433833901375_0139_01_000002/meta
>> >> 2015-06-13 10:08:01,084 INFO [main]
>>org.apache.kylin.cube.CubeManager:
>> >> Initializing CubeManager with config
>> >> /yarn/nm/usercache/biops/filecache/19452/meta
>> >> 2015-06-13 10:08:01,086 INFO [main]
>> >> org.apache.kylin.common.persistence.ResourceStore: Using metadata url
>> >> /yarn/nm/usercache/biops/filecache/19452/meta for resource store
>> >> 2015-06-13 10:08:01,327 INFO [main]
>> >>org.apache.kylin.cube.CubeDescManager:
>> >> Initializing CubeDescManager with config
>> >> /yarn/nm/usercache/biops/filecache/19452/meta
>> >> 2015-06-13 10:08:01,327 INFO [main]
>> >>org.apache.kylin.cube.CubeDescManager:
>> >> Reloading Cube Metadata from folder
>> >> /yarn/nm/usercache/biops/filecache/19452/meta/cube_desc
>> >> 2015-06-13 10:08:22,834 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 100000
>> >>records!
>> >> 2015-06-13 10:08:43,225 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 200000
>> >>records!
>> >> 2015-06-13 10:09:03,622 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 300000
>> >>records!
>> >> 2015-06-13 10:09:23,999 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 400000
>> >>records!
>> >> 2015-06-13 10:09:44,372 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 500000
>> >>records!
>> >> 2015-06-13 10:10:04,780 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 600000
>> >>records!
>> >> 2015-06-13 10:10:25,143 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 700000
>> >>records!
>> >> 2015-06-13 10:10:45,512 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 800000
>> >>records!
>> >> 2015-06-13 10:11:05,895 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 900000
>> >>records!
>> >> 2015-06-13 10:11:26,284 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1000000
>> >>records!
>> >> 2015-06-13 10:11:46,716 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1100000
>> >>records!
>> >> 2015-06-13 10:12:07,174 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1200000
>> >>records!
>> >> 2015-06-13 10:12:27,646 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1300000
>> >>records!
>> >> 2015-06-13 10:12:48,127 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1400000
>> >>records!
>> >> 2015-06-13 10:13:08,614 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1500000
>> >>records!
>> >> 2015-06-13 10:13:29,148 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1600000
>> >>records!
>> >> 2015-06-13 10:13:49,686 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1700000
>> >>records!
>> >> 2015-06-13 10:14:10,167 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1800000
>> >>records!
>> >> 2015-06-13 10:14:30,652 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1900000
>> >>records!
>> >> 2015-06-13 10:14:51,199 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2000000
>> >>records!
>> >> 2015-06-13 10:15:11,719 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2100000
>> >>records!
>> >> 2015-06-13 10:15:32,221 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2200000
>> >>records!
>> >> 2015-06-13 10:15:52,717 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2300000
>> >>records!
>> >> 2015-06-13 10:16:13,252 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2400000
>> >>records!
>> >> 2015-06-13 10:16:33,769 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2500000
>> >>records!
>> >> 2015-06-13 10:16:54,263 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2600000
>> >>records!
>> >> 2015-06-13 10:17:14,741 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2700000
>> >>records!
>> >> 2015-06-13 10:17:35,213 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2800000
>> >>records!
>> >> 2015-06-13 10:17:55,699 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2900000
>> >>records!
>> >> 2015-06-13 10:18:16,204 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3000000
>> >>records!
>> >> 2015-06-13 10:18:36,721 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3100000
>> >>records!
>> >> 2015-06-13 10:18:57,249 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3200000
>> >>records!
>> >> 2015-06-13 10:19:17,743 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3300000
>> >>records!
>> >> 2015-06-13 10:19:38,275 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3400000
>> >>records!
>> >> 2015-06-13 10:19:58,812 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3500000
>> >>records!
>> >> 2015-06-13 10:20:19,312 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3600000
>> >>records!
>> >> 2015-06-13 10:20:39,920 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3700000
>> >>records!
>> >> 2015-06-13 10:21:00,518 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3800000
>> >>records!
>> >> 2015-06-13 10:21:21,008 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3900000
>> >>records!
>> >> 2015-06-13 10:21:41,525 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4000000
>> >>records!
>> >> 2015-06-13 10:22:02,023 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4100000
>> >>records!
>> >> 2015-06-13 10:22:22,534 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4200000
>> >>records!
>> >> 2015-06-13 10:22:43,063 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4300000
>> >>records!
>> >> 2015-06-13 10:23:03,558 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4400000
>> >>records!
>> >> 2015-06-13 10:23:24,043 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4500000
>> >>records!
>> >> 2015-06-13 10:23:44,537 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4600000
>> >>records!
>> >> 2015-06-13 10:24:05,015 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4700000
>> >>records!
>> >> 2015-06-13 10:24:25,490 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4800000
>> >>records!
>> >> 2015-06-13 10:24:45,995 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4900000
>> >>records!
>> >> 2015-06-13 10:25:06,487 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5000000
>> >>records!
>> >> 2015-06-13 10:25:26,964 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5100000
>> >>records!
>> >> 2015-06-13 10:25:47,524 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5200000
>> >>records!
>> >> 2015-06-13 10:26:08,032 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5300000
>> >>records!
>> >> 2015-06-13 10:26:28,514 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5400000
>> >>records!
>> >> 2015-06-13 10:26:49,053 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5500000
>> >>records!
>> >> 2015-06-13 10:26:50,360 INFO [main] org.apache.hadoop.mapred.MapTask:
>> >> Spilling map output
>> >> 2015-06-13 10:26:50,360 INFO [main] org.apache.hadoop.mapred.MapTask:
>> >> bufstart = 0; bufend = 126646464; bufvoid = 268435456
>> >> 2015-06-13 10:26:50,360 INFO [main] org.apache.hadoop.mapred.MapTask:
>> >> kvstart = 67108860(268435440); kvend = 45083392(180333568); length =
>> >> 22025469/16777216
>> >> 2015-06-13 10:26:50,360 INFO [main] org.apache.hadoop.mapred.MapTask:
>> >> (EQUATOR) 148671936 kvi 37167980(148671920)
>> >> 2015-06-13 10:26:54,802 INFO [SpillThread]
>> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor
>> >>[.snappy]
>> >> 2015-06-13 10:26:54,883 INFO [SpillThread]
>> >> org.apache.kylin.job.hadoop.AbstractHadoopJob: The absolute path for
>> >>meta
>> >> dir is
>> >>
>>
>>>>/yarn/nm/usercache/biops/appcache/application_1433833901375_0139/contai
>>>>ne
>> >>r_1433833901375_0139_01_000002/meta
>> >> 2015-06-13 10:27:09,614 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5600000
>> >>records!
>> >> 2015-06-13 10:27:30,135 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5700000
>> >>records!
>> >> 2015-06-13 10:27:50,657 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5800000
>> >>records!
>> >> 2015-06-13 10:27:59,691 INFO [SpillThread]
>> >> org.apache.kylin.job.hadoop.cube.CuboidReducer: Handled 100000
>>records!
>> >> 2015-06-13 10:28:11,185 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5900000
>> >>records!
>> >> 2015-06-13 10:28:31,751 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6000000
>> >>records!
>> >> 2015-06-13 10:28:52,303 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6100000
>> >>records!
>> >> 2015-06-13 10:29:03,235 INFO [SpillThread]
>> >> org.apache.kylin.job.hadoop.cube.CuboidReducer: Handled 200000
>>records!
>> >> 2015-06-13 10:29:12,873 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6200000
>> >>records!
>> >> 2015-06-13 10:29:33,424 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6300000
>> >>records!
>> >> 2015-06-13 10:29:54,168 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6400000
>> >>records!
>> >> 2015-06-13 10:30:14,771 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6500000
>> >>records!
>> >> 2015-06-13 10:30:18,479 INFO [SpillThread]
>> >> org.apache.kylin.job.hadoop.cube.CuboidReducer: Handled 300000
>>records!
>> >> 2015-06-13 10:30:35,325 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6600000
>> >>records!
>> >> 2015-06-13 10:30:39,831 INFO [SpillThread]
>> >> org.apache.hadoop.mapred.MapTask: Finished spill 0
>> >> 2015-06-13 10:30:39,831 INFO [main] org.apache.hadoop.mapred.MapTask:
>> >> (RESET) equator 148671936 kv 37167980(148671920) kvi
>>32705668(130822672)
>> >> 2015-06-13 10:30:55,807 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6700000
>> >>records!
>> >> 2015-06-13 10:31:16,301 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6800000
>> >>records!
>> >> 2015-06-13 10:31:36,804 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6900000
>> >>records!
>> >> 2015-06-13 10:31:57,302 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7000000
>> >>records!
>> >> 2015-06-13 10:32:17,790 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7100000
>> >>records!
>> >> 2015-06-13 10:32:38,294 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7200000
>> >>records!
>> >> 2015-06-13 10:32:58,830 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7300000
>> >>records!
>> >> 2015-06-13 10:33:19,354 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7400000
>> >>records!
>> >> 2015-06-13 10:33:39,866 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7500000
>> >>records!
>> >> 2015-06-13 10:34:00,371 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7600000
>> >>records!
>> >> 2015-06-13 10:34:20,872 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7700000
>> >>records!
>> >> 2015-06-13 10:34:41,387 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7800000
>> >>records!
>> >> 2015-06-13 10:35:01,884 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7900000
>> >>records!
>> >> 2015-06-13 10:35:22,368 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8000000
>> >>records!
>> >> 2015-06-13 10:35:42,868 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8100000
>> >>records!
>> >> 2015-06-13 10:36:03,361 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8200000
>> >>records!
>> >> 2015-06-13 10:36:23,981 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8300000
>> >>records!
>> >> 2015-06-13 10:36:44,529 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8400000
>> >>records!
>> >> 2015-06-13 10:37:05,034 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8500000
>> >>records!
>> >> 2015-06-13 10:37:25,609 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8600000
>> >>records!
>> >> 2015-06-13 10:37:46,152 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8700000
>> >>records!
>> >> 2015-06-13 10:38:06,725 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8800000
>> >>records!
>> >> 2015-06-13 10:38:27,296 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8900000
>> >>records!
>> >> 2015-06-13 10:38:47,905 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9000000
>> >>records!
>> >> 2015-06-13 10:39:08,430 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9100000
>> >>records!
>> >> 2015-06-13 10:39:28,952 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9200000
>> >>records!
>> >> 2015-06-13 10:39:49,460 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9300000
>> >>records!
>> >> 2015-06-13 10:40:09,987 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9400000
>> >>records!
>> >> 2015-06-13 10:40:30,551 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9500000
>> >>records!
>> >> 2015-06-13 10:40:51,080 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9600000
>> >>records!
>> >> 2015-06-13 10:41:11,579 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9700000
>> >>records!
>> >> 2015-06-13 10:41:32,122 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9800000
>> >>records!
>> >> 2015-06-13 10:41:52,623 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9900000
>> >>records!
>> >> 2015-06-13 10:42:13,128 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10000000
>> >>records!
>> >> 2015-06-13 10:42:33,628 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10100000
>> >>records!
>> >> 2015-06-13 10:42:54,128 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10200000
>> >>records!
>> >> 2015-06-13 10:43:14,629 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10300000
>> >>records!
>> >> 2015-06-13 10:43:35,149 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10400000
>> >>records!
>> >> 2015-06-13 10:43:55,764 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10500000
>> >>records!
>> >> 2015-06-13 10:44:16,303 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10600000
>> >>records!
>> >> 2015-06-13 10:44:36,857 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10700000
>> >>records!
>> >> 2015-06-13 10:44:57,349 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10800000
>> >>records!
>> >> 2015-06-13 10:45:17,840 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10900000
>> >>records!
>> >> 2015-06-13 10:45:38,441 INFO [main]
>> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 11000000
>> >>records!
>> >> 2015-06-13 10:45:41,057 INFO [main] org.apache.hadoop.mapred.MapTask:
>> >> Spilling map output
>> >> 2015-06-13 10:45:41,057 INFO [main] org.apache.hadoop.mapred.MapTask:
>> >> bufstart = 148671936; bufend = 6882957; bufvoid = 268435443
>> >> 2015-06-13 10:45:41,057 INFO [main] org.apache.hadoop.mapred.MapTask:
>> >> kvstart = 37167980(148671920); kvend = 15142512(60570048); length =
>> >> 22025469/16777216
>> >> 2015-06-13 10:45:41,057 INFO [main] org.apache.hadoop.mapred.MapTask:
>> >> (EQUATOR) 28908413 kvi 7227096(28908384)
>> >>
>> >> Thanks,
>> >>
>> >> On Mon, Jun 15, 2015 at 3:11 PM, 周千昊 <qh...@apache.org> wrote:
>> >>
>> >>> Hi Mishra
>> >>> It is as what you described, since the data size is 30MB only,
>> >>>hadoop
>> >>> will only manage to run MR in single mapper and reduce.
>> >>> As for your question about why it takes so long to run on a
>>small
>> >>> dataset, can you please dig into the web page of *map reduce task
>> >>>status
>> >>> *to
>> >>> check how much time the MR job really take, so that we can make
>>sure if
>> >>> the
>> >>> time is consuming on the MR job or kylin job scheduling module
>> >>>
>> >>> Vineet Mishra <cl...@gmail.com>于2015年6月15日周一 下午4:51写道：
>> >>>
>> >>> > Shi,
>> >>> >
>> >>> > Hadoop is setup correctly on my cluster with the default block
>>size
>> >>>of
>> >>> > 128Mb and its indeed very much running multiple Mapper/Reducer
>>based
>> >>> jobs
>> >>> > for other cases.
>> >>> >
>> >>> > Its the only Kylin Cube building which is running through single
>>M/R
>> >>> job.
>> >>> >
>> >>> > Moreover to my surprise, the 4th running job which is Build Base
>> >>>Cuboid
>> >>> > Data shows Data size as 30Mb, is it the reason due to which single
>> >>> mapper
>> >>> > is getting invoked, if that being the case then even to process
>>such
>> >>>a
>> >>> > small data set why is it taking around 50min.
>> >>> >
>> >>> > Thanks,
>> >>> >
>> >>> > On Mon, Jun 15, 2015 at 11:17 AM, Shi, Shaofeng <sh...@ebay.com>
>> >>> wrote:
>> >>> >
>> >>> > > Yes you can adjust these parameters, for example give a smaller
>> >>>value
>> >>> for
>> >>> > > kylin.job.mapreduce.default.reduce.input.mb; but it only affects
>> >>>the
>> >>> > > reducer number;
>> >>> > >
>> >>> > > I suggest you investigate why there is only 1 mapper be started;
>> >>>Some
>> >>> > > factors like hadoop cluster size, HDFS file block size will
>>impact
>> >>> this;
>> >>> > > You can run a SQL (a query which need run MR, not simple select
>>*)
>> >>> with
>> >>> > > hive -e, and then use the MR job track URL to see how many
>>mappers
>> >>>be
>> >>> > > triggered; If it is still single, then the problem is in your
>> >>>hadoop
>> >>> > > configuration; Otherwise it may exists in Kylin, check if you
>>put
>> >>>some
>> >>> > > additional parameter in conf/kylin_job_conf.xml.
>> >>> > >
>> >>> > >
>> >>> > > On 6/15/15, 2:52 AM, "Vineet Mishra" <cl...@gmail.com>
>> >>>wrote:
>> >>> > >
>> >>> > > >Can I have specification for these properties?
>> >>> > > >
>> >>> > > >KYLIN_JOB_MAPREDUCE_DEFAULT_REDUCE_COUNT_RATIO =
>> >>> > > >"kylin.job.mapreduce.default.reduce.count.ratio";
>> >>> > > >KYLIN_JOB_MAPREDUCE_DEFAULT_REDUCE_INPUT_MB =
>> >>> > > >"kylin.job.mapreduce.default.reduce.input.mb";
>> >>> > > >KYLIN_JOB_MAPREDUCE_MAX_REDUCER_NUMBER =
>> >>> > > >"kylin.job.mapreduce.max.reducer.number";
>> >>> > > >
>> >>> > > >Thanks!
>> >>> > > >
>> >>> > > >On Sun, Jun 14, 2015 at 11:59 PM, Vineet Mishra <
>> >>> clearmidoubt@gmail.com
>> >>> > >
>> >>> > > >wrote:
>> >>> > > >
>> >>> > > >> Hi Shi,
>> >>> > > >>
>> >>> > > >> Its alright!
>> >>> > > >> So I was wondering my source hive Table is around 3 Gb,
>>despite
>> >>>of
>> >>> my
>> >>> > > >>hive
>> >>> > > >> table being partitioned and holding the data around 50-70 Mb
>>per
>> >>> > > >>partition
>> >>> > > >> the Mapper and Reducer getting spawned are single. The
>>amount of
>> >>> data
>> >>> > > >>that
>> >>> > > >> is being processed in the M/R is nothing as expected but it
>> >>>takes
>> >>> hell
>> >>> > > >>lot
>> >>> > > >> of time.
>> >>> > > >>
>> >>> > > >> As mentioned in the trailing mail that the job is getting
>>very
>> >>> slow,
>> >>> > the
>> >>> > > >> process Build Base Cuboid Data itself takes around 50mins to
>>get
>> >>> > > >> completed.
>> >>> > > >>
>> >>> > > >> I can tweak the reducer parameter mentioned by you but do u
>> >>>think
>> >>> that
>> >>> > > >> will make a difference since the mapper is where the most of
>>the
>> >>> time
>> >>> > is
>> >>> > > >> spent.
>> >>> > > >>
>> >>> > > >> Can you share your thoughts for performance tuning for the
>>cube
>> >>> build!
>> >>> > > >>
>> >>> > > >> Thanks!
>> >>> > > >>
>> >>> > > >> On Sun, Jun 14, 2015 at 7:26 PM, Shi, Shaofeng
>> >>><sh...@ebay.com>
>> >>> > > wrote:
>> >>> > > >>
>> >>> > > >>> Hi, sorry, a busy weekend;
>> >>> > > >>>
>> >>> > > >>> Usually Kylin will request proper number of mapper and
>> >>>reducers;
>> >>> If
>> >>> > you
>> >>> > > >>> see single mapper/recudder, how much of your input and
>>output?
>> >>>If
>> >>> > your
>> >>> > > >>> cube is quite small, single mapper/reducer is possible;
>> >>> > > >>>
>> >>> > > >>> Number of mappers is decided by the FileInputFormat; But
>> >>>number of
>> >>> > > >>>reducer
>> >>> > > >>> was set by Kylin, see:
>> >>> > > >>>
>> >>> > > >>>
>> >>> > > >>>
>> >>> > >
>> >>>
>> >>>
>> https://github.com/apache/incubator-kylin/blob/master/job/src/main/java/
>> >>> > > >>>org
>> >>> > > >>> /apache/kylin/job/hadoop/cube/CuboidJob.java#L141
>> >>> > > >>>
>> >>> > > >>><
>> >>> > >
>> >>>
>> https://github.com/apache/incubator-kylin/blob/master/job/src/main/java
>> >>> > > >>>/org/apache/kylin/job/hadoop/cube/CuboidJob.java#L141>
>> >>> > > >>>
>> >>> > > >>>
>> >>> > > >>>
>> >>> > > >>>
>> >>> > > >>> On 6/14/15, 5:25 PM, "Vineet Mishra"
>><cl...@gmail.com>
>> >>> wrote:
>> >>> > > >>>
>> >>> > > >>> >Urgent call, any follow up on this?
>> >>> > > >>> >
>> >>> > > >>> >On Fri, Jun 12, 2015 at 6:46 PM, Vineet Mishra
>> >>> > > >>><cl...@gmail.com>
>> >>> > > >>> >wrote:
>> >>> > > >>> >
>> >>> > > >>> >>
>> >>> > > >>> >> Why org.apache.kylin.job.hadoop.cube.CuboidReducer is
>> >>>running
>> >>> > Single
>> >>> > > >>> >> Mapper/Reducer for the job. Can I have the understanding
>> >>>behind
>> >>> > the
>> >>> > > >>> >>reason
>> >>> > > >>> >> of running it as single mapper/reducer.
>> >>> > > >>> >>
>> >>> > > >>> >> Thanks!
>> >>> > > >>> >>
>> >>> > > >>> >> On Fri, Jun 12, 2015 at 6:30 PM, Vineet Mishra
>> >>> > > >>><clearmidoubt@gmail.com
>> >>> > > >>> >
>> >>> > > >>> >> wrote:
>> >>> > > >>> >>
>> >>> > > >>> >>> Hi All,
>> >>> > > >>> >>>
>> >>> > > >>> >>> I am building a cube using Kylin and I could see that
>>the
>> >>>job
>> >>> is
>> >>> > > >>> >>>running
>> >>> > > >>> >>> with Single Mapper and Reducer for some of the
>>intermediate
>> >>> > process
>> >>> > > >>> >>>such as
>> >>> > > >>> >>>
>> >>> > > >>> >>> Extract Fact Table Distinct Columns
>> >>> > > >>> >>> Build Dimension Dictionary
>> >>> > > >>> >>> Build N-Dimension Cuboid
>> >>> > > >>> >>>
>> >>> > > >>> >>> I am not sure what's the reason behind running the job
>>with
>> >>> > single
>> >>> > > >>> M/R,
>> >>> > > >>> >>> is it really necessary or is it some default config.
>>which
>> >>> can be
>> >>> > > >>> >>>tweaked,
>> >>> > > >>> >>> its 70 Mins and the job status is 25% !
>> >>> > > >>> >>>
>> >>> > > >>> >>> Urgent Call!
>> >>> > > >>> >>>
>> >>> > > >>> >>> Thanks!
>> >>> > > >>> >>>
>> >>> > > >>> >>
>> >>> > > >>> >>
>> >>> > > >>>
>> >>> > > >>>
>> >>> > > >>
>> >>> > >
>> >>> > >
>> >>> >
>> >>>
>> >>
>> >>
>>
>>
Re: Jobs Running with Single Mapper/Reducer
Posted by Vineet Mishra <cl...@gmail.com>.
Thanks Shi,
By the way I was curious as for the same table, dimension and metric, I
could see the final cube size varying from 130 Mb to 420Mb.
The cube is exactly the same as another one but when the cube build job is
run I could see the cube size difference as mentioned below,
*Earlier Cube Build*
Size - 420 Mb
Time Taken for building cube - 140 mins
Source Records - ~11Million
*Today's Cube Build*
Size - 130 Mb
Time Taken for building cube - 22 mins
Source Records - ~11Million
I don't find any reason to have three time cube size difference. Any ideas
about this?
Thanks!
On Mon, Jun 15, 2015 at 8:08 PM, Shi, Shaofeng <sh...@ebay.com> wrote:
> Good catch, and thanks for the sharing;
>
> On 6/15/15, 10:20 PM, "Vineet Mishra" <cl...@gmail.com> wrote:
>
> >Hi All,
> >
> >Well I got it through.
> >
> >Actually it was basically the pre-process which was converting my 3Gb hive
> >table to 60Mb sequence file which due to block size of 128Mb was running
> >in
> >a single Mapper job.
> >
> >I changed the split size to 20Mb and it was able to spawn 3 Mapper after
> >that.
> >
> >Anyways, thanks all for your quick response.
> >
> >Thanks!
> >
> >On Mon, Jun 15, 2015 at 3:27 PM, Vineet Mishra <cl...@gmail.com>
> >wrote:
> >
> >> Hi,
> >>
> >> Sorry I misunderstood this value, actually 30Mb was the output for that
> >> job. But I guess I have the got reason, (not exactly confirmed) as why
> >>its
> >> behaving like this, the input to this job is sequence file which is
> >>60Mb in
> >> size and compressed.
> >>
> >> No doubt when it will be passed to the mapper, it is very much certain
> >>to
> >> have the single mapper invoked due to the pre-configured block size.
> >>Now I
> >> need to check If I can override the splitSize property for the job.
> >>
> >> Moreover I can see that the job is running seamlessly slow, kindly find
> >> the stack trace for the job mentioned below.
> >>
> >> 2015-06-13 10:08:00,940 INFO [main] org.apache.hadoop.mapred.MapTask:
> >> Processing split: hdfs://
> >>
> >>
> dev-hadoop-namenode.com:8020/tmp/kylin-9b675c66-4ce9-4a33-a356-2ccf9dbaca
> >>6a/kylin_intermediate_xyz_20150401000000_20150611000000_9b675c66_4ce9_4a3
> >>3_a356_2ccf9dbaca6a/000000_0:0+60440271
> >> 2015-06-13 10:08:01,014 INFO [main] org.apache.hadoop.mapred.MapTask:
> >> (EQUATOR) 0 kvi 67108860(268435440)
> >> 2015-06-13 10:08:01,015 INFO [main] org.apache.hadoop.mapred.MapTask:
> >> mapreduce.task.io.sort.mb: 256
> >> 2015-06-13 10:08:01,015 INFO [main] org.apache.hadoop.mapred.MapTask:
> >>soft
> >> limit at 214748368
> >> 2015-06-13 10:08:01,015 INFO [main] org.apache.hadoop.mapred.MapTask:
> >> bufstart = 0; bufvoid = 268435456
> >> 2015-06-13 10:08:01,015 INFO [main] org.apache.hadoop.mapred.MapTask:
> >> kvstart = 67108860; length = 16777216
> >> 2015-06-13 10:08:01,023 INFO [main] org.apache.hadoop.mapred.MapTask:
> >>Map
> >> output collector class =
> >>org.apache.hadoop.mapred.MapTask$MapOutputBuffer
> >> 2015-06-13 10:08:01,048 INFO [main]
> >> org.apache.hadoop.io.compress.zlib.ZlibFactory: Successfully loaded &
> >> initialized native-zlib library
> >> 2015-06-13 10:08:01,048 INFO [main]
> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
> >> [.deflate]
> >> 2015-06-13 10:08:01,052 INFO [main]
> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
> >> [.deflate]
> >> 2015-06-13 10:08:01,052 INFO [main]
> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
> >> [.deflate]
> >> 2015-06-13 10:08:01,053 INFO [main]
> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
> >> [.deflate]
> >> 2015-06-13 10:08:01,058 INFO [main]
> >> org.apache.kylin.job.hadoop.AbstractHadoopJob: The absolute path for
> >>meta
> >> dir is
> >>
> >>/yarn/nm/usercache/biops/appcache/application_1433833901375_0139/containe
> >>r_1433833901375_0139_01_000002/meta
> >> 2015-06-13 10:08:01,059 INFO [main] org.apache.kylin.common.KylinConfig:
> >> Use
> >>
> >>KYLIN_CONF=/yarn/nm/usercache/biops/appcache/application_1433833901375_01
> >>39/container_1433833901375_0139_01_000002/meta
> >> 2015-06-13 10:08:01,084 INFO [main] org.apache.kylin.cube.CubeManager:
> >> Initializing CubeManager with config
> >> /yarn/nm/usercache/biops/filecache/19452/meta
> >> 2015-06-13 10:08:01,086 INFO [main]
> >> org.apache.kylin.common.persistence.ResourceStore: Using metadata url
> >> /yarn/nm/usercache/biops/filecache/19452/meta for resource store
> >> 2015-06-13 10:08:01,327 INFO [main]
> >>org.apache.kylin.cube.CubeDescManager:
> >> Initializing CubeDescManager with config
> >> /yarn/nm/usercache/biops/filecache/19452/meta
> >> 2015-06-13 10:08:01,327 INFO [main]
> >>org.apache.kylin.cube.CubeDescManager:
> >> Reloading Cube Metadata from folder
> >> /yarn/nm/usercache/biops/filecache/19452/meta/cube_desc
> >> 2015-06-13 10:08:22,834 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 100000
> >>records!
> >> 2015-06-13 10:08:43,225 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 200000
> >>records!
> >> 2015-06-13 10:09:03,622 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 300000
> >>records!
> >> 2015-06-13 10:09:23,999 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 400000
> >>records!
> >> 2015-06-13 10:09:44,372 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 500000
> >>records!
> >> 2015-06-13 10:10:04,780 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 600000
> >>records!
> >> 2015-06-13 10:10:25,143 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 700000
> >>records!
> >> 2015-06-13 10:10:45,512 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 800000
> >>records!
> >> 2015-06-13 10:11:05,895 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 900000
> >>records!
> >> 2015-06-13 10:11:26,284 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1000000
> >>records!
> >> 2015-06-13 10:11:46,716 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1100000
> >>records!
> >> 2015-06-13 10:12:07,174 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1200000
> >>records!
> >> 2015-06-13 10:12:27,646 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1300000
> >>records!
> >> 2015-06-13 10:12:48,127 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1400000
> >>records!
> >> 2015-06-13 10:13:08,614 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1500000
> >>records!
> >> 2015-06-13 10:13:29,148 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1600000
> >>records!
> >> 2015-06-13 10:13:49,686 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1700000
> >>records!
> >> 2015-06-13 10:14:10,167 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1800000
> >>records!
> >> 2015-06-13 10:14:30,652 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1900000
> >>records!
> >> 2015-06-13 10:14:51,199 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2000000
> >>records!
> >> 2015-06-13 10:15:11,719 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2100000
> >>records!
> >> 2015-06-13 10:15:32,221 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2200000
> >>records!
> >> 2015-06-13 10:15:52,717 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2300000
> >>records!
> >> 2015-06-13 10:16:13,252 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2400000
> >>records!
> >> 2015-06-13 10:16:33,769 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2500000
> >>records!
> >> 2015-06-13 10:16:54,263 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2600000
> >>records!
> >> 2015-06-13 10:17:14,741 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2700000
> >>records!
> >> 2015-06-13 10:17:35,213 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2800000
> >>records!
> >> 2015-06-13 10:17:55,699 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2900000
> >>records!
> >> 2015-06-13 10:18:16,204 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3000000
> >>records!
> >> 2015-06-13 10:18:36,721 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3100000
> >>records!
> >> 2015-06-13 10:18:57,249 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3200000
> >>records!
> >> 2015-06-13 10:19:17,743 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3300000
> >>records!
> >> 2015-06-13 10:19:38,275 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3400000
> >>records!
> >> 2015-06-13 10:19:58,812 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3500000
> >>records!
> >> 2015-06-13 10:20:19,312 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3600000
> >>records!
> >> 2015-06-13 10:20:39,920 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3700000
> >>records!
> >> 2015-06-13 10:21:00,518 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3800000
> >>records!
> >> 2015-06-13 10:21:21,008 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3900000
> >>records!
> >> 2015-06-13 10:21:41,525 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4000000
> >>records!
> >> 2015-06-13 10:22:02,023 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4100000
> >>records!
> >> 2015-06-13 10:22:22,534 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4200000
> >>records!
> >> 2015-06-13 10:22:43,063 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4300000
> >>records!
> >> 2015-06-13 10:23:03,558 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4400000
> >>records!
> >> 2015-06-13 10:23:24,043 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4500000
> >>records!
> >> 2015-06-13 10:23:44,537 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4600000
> >>records!
> >> 2015-06-13 10:24:05,015 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4700000
> >>records!
> >> 2015-06-13 10:24:25,490 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4800000
> >>records!
> >> 2015-06-13 10:24:45,995 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4900000
> >>records!
> >> 2015-06-13 10:25:06,487 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5000000
> >>records!
> >> 2015-06-13 10:25:26,964 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5100000
> >>records!
> >> 2015-06-13 10:25:47,524 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5200000
> >>records!
> >> 2015-06-13 10:26:08,032 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5300000
> >>records!
> >> 2015-06-13 10:26:28,514 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5400000
> >>records!
> >> 2015-06-13 10:26:49,053 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5500000
> >>records!
> >> 2015-06-13 10:26:50,360 INFO [main] org.apache.hadoop.mapred.MapTask:
> >> Spilling map output
> >> 2015-06-13 10:26:50,360 INFO [main] org.apache.hadoop.mapred.MapTask:
> >> bufstart = 0; bufend = 126646464; bufvoid = 268435456
> >> 2015-06-13 10:26:50,360 INFO [main] org.apache.hadoop.mapred.MapTask:
> >> kvstart = 67108860(268435440); kvend = 45083392(180333568); length =
> >> 22025469/16777216
> >> 2015-06-13 10:26:50,360 INFO [main] org.apache.hadoop.mapred.MapTask:
> >> (EQUATOR) 148671936 kvi 37167980(148671920)
> >> 2015-06-13 10:26:54,802 INFO [SpillThread]
> >> org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor
> >>[.snappy]
> >> 2015-06-13 10:26:54,883 INFO [SpillThread]
> >> org.apache.kylin.job.hadoop.AbstractHadoopJob: The absolute path for
> >>meta
> >> dir is
> >>
> >>/yarn/nm/usercache/biops/appcache/application_1433833901375_0139/containe
> >>r_1433833901375_0139_01_000002/meta
> >> 2015-06-13 10:27:09,614 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5600000
> >>records!
> >> 2015-06-13 10:27:30,135 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5700000
> >>records!
> >> 2015-06-13 10:27:50,657 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5800000
> >>records!
> >> 2015-06-13 10:27:59,691 INFO [SpillThread]
> >> org.apache.kylin.job.hadoop.cube.CuboidReducer: Handled 100000 records!
> >> 2015-06-13 10:28:11,185 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5900000
> >>records!
> >> 2015-06-13 10:28:31,751 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6000000
> >>records!
> >> 2015-06-13 10:28:52,303 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6100000
> >>records!
> >> 2015-06-13 10:29:03,235 INFO [SpillThread]
> >> org.apache.kylin.job.hadoop.cube.CuboidReducer: Handled 200000 records!
> >> 2015-06-13 10:29:12,873 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6200000
> >>records!
> >> 2015-06-13 10:29:33,424 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6300000
> >>records!
> >> 2015-06-13 10:29:54,168 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6400000
> >>records!
> >> 2015-06-13 10:30:14,771 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6500000
> >>records!
> >> 2015-06-13 10:30:18,479 INFO [SpillThread]
> >> org.apache.kylin.job.hadoop.cube.CuboidReducer: Handled 300000 records!
> >> 2015-06-13 10:30:35,325 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6600000
> >>records!
> >> 2015-06-13 10:30:39,831 INFO [SpillThread]
> >> org.apache.hadoop.mapred.MapTask: Finished spill 0
> >> 2015-06-13 10:30:39,831 INFO [main] org.apache.hadoop.mapred.MapTask:
> >> (RESET) equator 148671936 kv 37167980(148671920) kvi 32705668(130822672)
> >> 2015-06-13 10:30:55,807 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6700000
> >>records!
> >> 2015-06-13 10:31:16,301 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6800000
> >>records!
> >> 2015-06-13 10:31:36,804 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6900000
> >>records!
> >> 2015-06-13 10:31:57,302 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7000000
> >>records!
> >> 2015-06-13 10:32:17,790 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7100000
> >>records!
> >> 2015-06-13 10:32:38,294 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7200000
> >>records!
> >> 2015-06-13 10:32:58,830 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7300000
> >>records!
> >> 2015-06-13 10:33:19,354 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7400000
> >>records!
> >> 2015-06-13 10:33:39,866 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7500000
> >>records!
> >> 2015-06-13 10:34:00,371 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7600000
> >>records!
> >> 2015-06-13 10:34:20,872 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7700000
> >>records!
> >> 2015-06-13 10:34:41,387 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7800000
> >>records!
> >> 2015-06-13 10:35:01,884 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7900000
> >>records!
> >> 2015-06-13 10:35:22,368 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8000000
> >>records!
> >> 2015-06-13 10:35:42,868 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8100000
> >>records!
> >> 2015-06-13 10:36:03,361 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8200000
> >>records!
> >> 2015-06-13 10:36:23,981 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8300000
> >>records!
> >> 2015-06-13 10:36:44,529 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8400000
> >>records!
> >> 2015-06-13 10:37:05,034 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8500000
> >>records!
> >> 2015-06-13 10:37:25,609 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8600000
> >>records!
> >> 2015-06-13 10:37:46,152 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8700000
> >>records!
> >> 2015-06-13 10:38:06,725 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8800000
> >>records!
> >> 2015-06-13 10:38:27,296 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8900000
> >>records!
> >> 2015-06-13 10:38:47,905 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9000000
> >>records!
> >> 2015-06-13 10:39:08,430 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9100000
> >>records!
> >> 2015-06-13 10:39:28,952 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9200000
> >>records!
> >> 2015-06-13 10:39:49,460 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9300000
> >>records!
> >> 2015-06-13 10:40:09,987 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9400000
> >>records!
> >> 2015-06-13 10:40:30,551 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9500000
> >>records!
> >> 2015-06-13 10:40:51,080 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9600000
> >>records!
> >> 2015-06-13 10:41:11,579 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9700000
> >>records!
> >> 2015-06-13 10:41:32,122 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9800000
> >>records!
> >> 2015-06-13 10:41:52,623 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9900000
> >>records!
> >> 2015-06-13 10:42:13,128 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10000000
> >>records!
> >> 2015-06-13 10:42:33,628 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10100000
> >>records!
> >> 2015-06-13 10:42:54,128 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10200000
> >>records!
> >> 2015-06-13 10:43:14,629 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10300000
> >>records!
> >> 2015-06-13 10:43:35,149 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10400000
> >>records!
> >> 2015-06-13 10:43:55,764 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10500000
> >>records!
> >> 2015-06-13 10:44:16,303 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10600000
> >>records!
> >> 2015-06-13 10:44:36,857 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10700000
> >>records!
> >> 2015-06-13 10:44:57,349 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10800000
> >>records!
> >> 2015-06-13 10:45:17,840 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10900000
> >>records!
> >> 2015-06-13 10:45:38,441 INFO [main]
> >> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 11000000
> >>records!
> >> 2015-06-13 10:45:41,057 INFO [main] org.apache.hadoop.mapred.MapTask:
> >> Spilling map output
> >> 2015-06-13 10:45:41,057 INFO [main] org.apache.hadoop.mapred.MapTask:
> >> bufstart = 148671936; bufend = 6882957; bufvoid = 268435443
> >> 2015-06-13 10:45:41,057 INFO [main] org.apache.hadoop.mapred.MapTask:
> >> kvstart = 37167980(148671920); kvend = 15142512(60570048); length =
> >> 22025469/16777216
> >> 2015-06-13 10:45:41,057 INFO [main] org.apache.hadoop.mapred.MapTask:
> >> (EQUATOR) 28908413 kvi 7227096(28908384)
> >>
> >> Thanks,
> >>
> >> On Mon, Jun 15, 2015 at 3:11 PM, 周千昊 <qh...@apache.org> wrote:
> >>
> >>> Hi Mishra
> >>> It is as what you described, since the data size is 30MB only,
> >>>hadoop
> >>> will only manage to run MR in single mapper and reduce.
> >>> As for your question about why it takes so long to run on a small
> >>> dataset, can you please dig into the web page of *map reduce task
> >>>status
> >>> *to
> >>> check how much time the MR job really take, so that we can make sure if
> >>> the
> >>> time is consuming on the MR job or kylin job scheduling module
> >>>
> >>> Vineet Mishra <cl...@gmail.com>于2015年6月15日周一 下午4:51写道：
> >>>
> >>> > Shi,
> >>> >
> >>> > Hadoop is setup correctly on my cluster with the default block size
> >>>of
> >>> > 128Mb and its indeed very much running multiple Mapper/Reducer based
> >>> jobs
> >>> > for other cases.
> >>> >
> >>> > Its the only Kylin Cube building which is running through single M/R
> >>> job.
> >>> >
> >>> > Moreover to my surprise, the 4th running job which is Build Base
> >>>Cuboid
> >>> > Data shows Data size as 30Mb, is it the reason due to which single
> >>> mapper
> >>> > is getting invoked, if that being the case then even to process such
> >>>a
> >>> > small data set why is it taking around 50min.
> >>> >
> >>> > Thanks,
> >>> >
> >>> > On Mon, Jun 15, 2015 at 11:17 AM, Shi, Shaofeng <sh...@ebay.com>
> >>> wrote:
> >>> >
> >>> > > Yes you can adjust these parameters, for example give a smaller
> >>>value
> >>> for
> >>> > > kylin.job.mapreduce.default.reduce.input.mb; but it only affects
> >>>the
> >>> > > reducer number;
> >>> > >
> >>> > > I suggest you investigate why there is only 1 mapper be started;
> >>>Some
> >>> > > factors like hadoop cluster size, HDFS file block size will impact
> >>> this;
> >>> > > You can run a SQL (a query which need run MR, not simple select *)
> >>> with
> >>> > > hive -e, and then use the MR job track URL to see how many mappers
> >>>be
> >>> > > triggered; If it is still single, then the problem is in your
> >>>hadoop
> >>> > > configuration; Otherwise it may exists in Kylin, check if you put
> >>>some
> >>> > > additional parameter in conf/kylin_job_conf.xml.
> >>> > >
> >>> > >
> >>> > > On 6/15/15, 2:52 AM, "Vineet Mishra" <cl...@gmail.com>
> >>>wrote:
> >>> > >
> >>> > > >Can I have specification for these properties?
> >>> > > >
> >>> > > >KYLIN_JOB_MAPREDUCE_DEFAULT_REDUCE_COUNT_RATIO =
> >>> > > >"kylin.job.mapreduce.default.reduce.count.ratio";
> >>> > > >KYLIN_JOB_MAPREDUCE_DEFAULT_REDUCE_INPUT_MB =
> >>> > > >"kylin.job.mapreduce.default.reduce.input.mb";
> >>> > > >KYLIN_JOB_MAPREDUCE_MAX_REDUCER_NUMBER =
> >>> > > >"kylin.job.mapreduce.max.reducer.number";
> >>> > > >
> >>> > > >Thanks!
> >>> > > >
> >>> > > >On Sun, Jun 14, 2015 at 11:59 PM, Vineet Mishra <
> >>> clearmidoubt@gmail.com
> >>> > >
> >>> > > >wrote:
> >>> > > >
> >>> > > >> Hi Shi,
> >>> > > >>
> >>> > > >> Its alright!
> >>> > > >> So I was wondering my source hive Table is around 3 Gb, despite
> >>>of
> >>> my
> >>> > > >>hive
> >>> > > >> table being partitioned and holding the data around 50-70 Mb per
> >>> > > >>partition
> >>> > > >> the Mapper and Reducer getting spawned are single. The amount of
> >>> data
> >>> > > >>that
> >>> > > >> is being processed in the M/R is nothing as expected but it
> >>>takes
> >>> hell
> >>> > > >>lot
> >>> > > >> of time.
> >>> > > >>
> >>> > > >> As mentioned in the trailing mail that the job is getting very
> >>> slow,
> >>> > the
> >>> > > >> process Build Base Cuboid Data itself takes around 50mins to get
> >>> > > >> completed.
> >>> > > >>
> >>> > > >> I can tweak the reducer parameter mentioned by you but do u
> >>>think
> >>> that
> >>> > > >> will make a difference since the mapper is where the most of the
> >>> time
> >>> > is
> >>> > > >> spent.
> >>> > > >>
> >>> > > >> Can you share your thoughts for performance tuning for the cube
> >>> build!
> >>> > > >>
> >>> > > >> Thanks!
> >>> > > >>
> >>> > > >> On Sun, Jun 14, 2015 at 7:26 PM, Shi, Shaofeng
> >>><sh...@ebay.com>
> >>> > > wrote:
> >>> > > >>
> >>> > > >>> Hi, sorry, a busy weekend;
> >>> > > >>>
> >>> > > >>> Usually Kylin will request proper number of mapper and
> >>>reducers;
> >>> If
> >>> > you
> >>> > > >>> see single mapper/recudder, how much of your input and output?
> >>>If
> >>> > your
> >>> > > >>> cube is quite small, single mapper/reducer is possible;
> >>> > > >>>
> >>> > > >>> Number of mappers is decided by the FileInputFormat; But
> >>>number of
> >>> > > >>>reducer
> >>> > > >>> was set by Kylin, see:
> >>> > > >>>
> >>> > > >>>
> >>> > > >>>
> >>> > >
> >>>
> >>>
> https://github.com/apache/incubator-kylin/blob/master/job/src/main/java/
> >>> > > >>>org
> >>> > > >>> /apache/kylin/job/hadoop/cube/CuboidJob.java#L141
> >>> > > >>>
> >>> > > >>><
> >>> > >
> >>>
> https://github.com/apache/incubator-kylin/blob/master/job/src/main/java
> >>> > > >>>/org/apache/kylin/job/hadoop/cube/CuboidJob.java#L141>
> >>> > > >>>
> >>> > > >>>
> >>> > > >>>
> >>> > > >>>
> >>> > > >>> On 6/14/15, 5:25 PM, "Vineet Mishra" <cl...@gmail.com>
> >>> wrote:
> >>> > > >>>
> >>> > > >>> >Urgent call, any follow up on this?
> >>> > > >>> >
> >>> > > >>> >On Fri, Jun 12, 2015 at 6:46 PM, Vineet Mishra
> >>> > > >>><cl...@gmail.com>
> >>> > > >>> >wrote:
> >>> > > >>> >
> >>> > > >>> >>
> >>> > > >>> >> Why org.apache.kylin.job.hadoop.cube.CuboidReducer is
> >>>running
> >>> > Single
> >>> > > >>> >> Mapper/Reducer for the job. Can I have the understanding
> >>>behind
> >>> > the
> >>> > > >>> >>reason
> >>> > > >>> >> of running it as single mapper/reducer.
> >>> > > >>> >>
> >>> > > >>> >> Thanks!
> >>> > > >>> >>
> >>> > > >>> >> On Fri, Jun 12, 2015 at 6:30 PM, Vineet Mishra
> >>> > > >>><clearmidoubt@gmail.com
> >>> > > >>> >
> >>> > > >>> >> wrote:
> >>> > > >>> >>
> >>> > > >>> >>> Hi All,
> >>> > > >>> >>>
> >>> > > >>> >>> I am building a cube using Kylin and I could see that the
> >>>job
> >>> is
> >>> > > >>> >>>running
> >>> > > >>> >>> with Single Mapper and Reducer for some of the intermediate
> >>> > process
> >>> > > >>> >>>such as
> >>> > > >>> >>>
> >>> > > >>> >>> Extract Fact Table Distinct Columns
> >>> > > >>> >>> Build Dimension Dictionary
> >>> > > >>> >>> Build N-Dimension Cuboid
> >>> > > >>> >>>
> >>> > > >>> >>> I am not sure what's the reason behind running the job with
> >>> > single
> >>> > > >>> M/R,
> >>> > > >>> >>> is it really necessary or is it some default config. which
> >>> can be
> >>> > > >>> >>>tweaked,
> >>> > > >>> >>> its 70 Mins and the job status is 25% !
> >>> > > >>> >>>
> >>> > > >>> >>> Urgent Call!
> >>> > > >>> >>>
> >>> > > >>> >>> Thanks!
> >>> > > >>> >>>
> >>> > > >>> >>
> >>> > > >>> >>
> >>> > > >>>
> >>> > > >>>
> >>> > > >>
> >>> > >
> >>> > >
> >>> >
> >>>
> >>
> >>
>
>
Re: Jobs Running with Single Mapper/Reducer
Posted by "Shi, Shaofeng" <sh...@ebay.com>.
Good catch, and thanks for the sharing;
On 6/15/15, 10:20 PM, "Vineet Mishra" <cl...@gmail.com> wrote:
>Hi All,
>
>Well I got it through.
>
>Actually it was basically the pre-process which was converting my 3Gb hive
>table to 60Mb sequence file which due to block size of 128Mb was running
>in
>a single Mapper job.
>
>I changed the split size to 20Mb and it was able to spawn 3 Mapper after
>that.
>
>Anyways, thanks all for your quick response.
>
>Thanks!
>
>On Mon, Jun 15, 2015 at 3:27 PM, Vineet Mishra <cl...@gmail.com>
>wrote:
>
>> Hi,
>>
>> Sorry I misunderstood this value, actually 30Mb was the output for that
>> job. But I guess I have the got reason, (not exactly confirmed) as why
>>its
>> behaving like this, the input to this job is sequence file which is
>>60Mb in
>> size and compressed.
>>
>> No doubt when it will be passed to the mapper, it is very much certain
>>to
>> have the single mapper invoked due to the pre-configured block size.
>>Now I
>> need to check If I can override the splitSize property for the job.
>>
>> Moreover I can see that the job is running seamlessly slow, kindly find
>> the stack trace for the job mentioned below.
>>
>> 2015-06-13 10:08:00,940 INFO [main] org.apache.hadoop.mapred.MapTask:
>> Processing split: hdfs://
>>
>>dev-hadoop-namenode.com:8020/tmp/kylin-9b675c66-4ce9-4a33-a356-2ccf9dbaca
>>6a/kylin_intermediate_xyz_20150401000000_20150611000000_9b675c66_4ce9_4a3
>>3_a356_2ccf9dbaca6a/000000_0:0+60440271
>> 2015-06-13 10:08:01,014 INFO [main] org.apache.hadoop.mapred.MapTask:
>> (EQUATOR) 0 kvi 67108860(268435440)
>> 2015-06-13 10:08:01,015 INFO [main] org.apache.hadoop.mapred.MapTask:
>> mapreduce.task.io.sort.mb: 256
>> 2015-06-13 10:08:01,015 INFO [main] org.apache.hadoop.mapred.MapTask:
>>soft
>> limit at 214748368
>> 2015-06-13 10:08:01,015 INFO [main] org.apache.hadoop.mapred.MapTask:
>> bufstart = 0; bufvoid = 268435456
>> 2015-06-13 10:08:01,015 INFO [main] org.apache.hadoop.mapred.MapTask:
>> kvstart = 67108860; length = 16777216
>> 2015-06-13 10:08:01,023 INFO [main] org.apache.hadoop.mapred.MapTask:
>>Map
>> output collector class =
>>org.apache.hadoop.mapred.MapTask$MapOutputBuffer
>> 2015-06-13 10:08:01,048 INFO [main]
>> org.apache.hadoop.io.compress.zlib.ZlibFactory: Successfully loaded &
>> initialized native-zlib library
>> 2015-06-13 10:08:01,048 INFO [main]
>> org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
>> [.deflate]
>> 2015-06-13 10:08:01,052 INFO [main]
>> org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
>> [.deflate]
>> 2015-06-13 10:08:01,052 INFO [main]
>> org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
>> [.deflate]
>> 2015-06-13 10:08:01,053 INFO [main]
>> org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
>> [.deflate]
>> 2015-06-13 10:08:01,058 INFO [main]
>> org.apache.kylin.job.hadoop.AbstractHadoopJob: The absolute path for
>>meta
>> dir is
>>
>>/yarn/nm/usercache/biops/appcache/application_1433833901375_0139/containe
>>r_1433833901375_0139_01_000002/meta
>> 2015-06-13 10:08:01,059 INFO [main] org.apache.kylin.common.KylinConfig:
>> Use
>>
>>KYLIN_CONF=/yarn/nm/usercache/biops/appcache/application_1433833901375_01
>>39/container_1433833901375_0139_01_000002/meta
>> 2015-06-13 10:08:01,084 INFO [main] org.apache.kylin.cube.CubeManager:
>> Initializing CubeManager with config
>> /yarn/nm/usercache/biops/filecache/19452/meta
>> 2015-06-13 10:08:01,086 INFO [main]
>> org.apache.kylin.common.persistence.ResourceStore: Using metadata url
>> /yarn/nm/usercache/biops/filecache/19452/meta for resource store
>> 2015-06-13 10:08:01,327 INFO [main]
>>org.apache.kylin.cube.CubeDescManager:
>> Initializing CubeDescManager with config
>> /yarn/nm/usercache/biops/filecache/19452/meta
>> 2015-06-13 10:08:01,327 INFO [main]
>>org.apache.kylin.cube.CubeDescManager:
>> Reloading Cube Metadata from folder
>> /yarn/nm/usercache/biops/filecache/19452/meta/cube_desc
>> 2015-06-13 10:08:22,834 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 100000
>>records!
>> 2015-06-13 10:08:43,225 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 200000
>>records!
>> 2015-06-13 10:09:03,622 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 300000
>>records!
>> 2015-06-13 10:09:23,999 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 400000
>>records!
>> 2015-06-13 10:09:44,372 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 500000
>>records!
>> 2015-06-13 10:10:04,780 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 600000
>>records!
>> 2015-06-13 10:10:25,143 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 700000
>>records!
>> 2015-06-13 10:10:45,512 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 800000
>>records!
>> 2015-06-13 10:11:05,895 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 900000
>>records!
>> 2015-06-13 10:11:26,284 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1000000
>>records!
>> 2015-06-13 10:11:46,716 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1100000
>>records!
>> 2015-06-13 10:12:07,174 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1200000
>>records!
>> 2015-06-13 10:12:27,646 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1300000
>>records!
>> 2015-06-13 10:12:48,127 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1400000
>>records!
>> 2015-06-13 10:13:08,614 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1500000
>>records!
>> 2015-06-13 10:13:29,148 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1600000
>>records!
>> 2015-06-13 10:13:49,686 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1700000
>>records!
>> 2015-06-13 10:14:10,167 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1800000
>>records!
>> 2015-06-13 10:14:30,652 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1900000
>>records!
>> 2015-06-13 10:14:51,199 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2000000
>>records!
>> 2015-06-13 10:15:11,719 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2100000
>>records!
>> 2015-06-13 10:15:32,221 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2200000
>>records!
>> 2015-06-13 10:15:52,717 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2300000
>>records!
>> 2015-06-13 10:16:13,252 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2400000
>>records!
>> 2015-06-13 10:16:33,769 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2500000
>>records!
>> 2015-06-13 10:16:54,263 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2600000
>>records!
>> 2015-06-13 10:17:14,741 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2700000
>>records!
>> 2015-06-13 10:17:35,213 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2800000
>>records!
>> 2015-06-13 10:17:55,699 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2900000
>>records!
>> 2015-06-13 10:18:16,204 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3000000
>>records!
>> 2015-06-13 10:18:36,721 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3100000
>>records!
>> 2015-06-13 10:18:57,249 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3200000
>>records!
>> 2015-06-13 10:19:17,743 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3300000
>>records!
>> 2015-06-13 10:19:38,275 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3400000
>>records!
>> 2015-06-13 10:19:58,812 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3500000
>>records!
>> 2015-06-13 10:20:19,312 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3600000
>>records!
>> 2015-06-13 10:20:39,920 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3700000
>>records!
>> 2015-06-13 10:21:00,518 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3800000
>>records!
>> 2015-06-13 10:21:21,008 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3900000
>>records!
>> 2015-06-13 10:21:41,525 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4000000
>>records!
>> 2015-06-13 10:22:02,023 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4100000
>>records!
>> 2015-06-13 10:22:22,534 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4200000
>>records!
>> 2015-06-13 10:22:43,063 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4300000
>>records!
>> 2015-06-13 10:23:03,558 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4400000
>>records!
>> 2015-06-13 10:23:24,043 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4500000
>>records!
>> 2015-06-13 10:23:44,537 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4600000
>>records!
>> 2015-06-13 10:24:05,015 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4700000
>>records!
>> 2015-06-13 10:24:25,490 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4800000
>>records!
>> 2015-06-13 10:24:45,995 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4900000
>>records!
>> 2015-06-13 10:25:06,487 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5000000
>>records!
>> 2015-06-13 10:25:26,964 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5100000
>>records!
>> 2015-06-13 10:25:47,524 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5200000
>>records!
>> 2015-06-13 10:26:08,032 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5300000
>>records!
>> 2015-06-13 10:26:28,514 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5400000
>>records!
>> 2015-06-13 10:26:49,053 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5500000
>>records!
>> 2015-06-13 10:26:50,360 INFO [main] org.apache.hadoop.mapred.MapTask:
>> Spilling map output
>> 2015-06-13 10:26:50,360 INFO [main] org.apache.hadoop.mapred.MapTask:
>> bufstart = 0; bufend = 126646464; bufvoid = 268435456
>> 2015-06-13 10:26:50,360 INFO [main] org.apache.hadoop.mapred.MapTask:
>> kvstart = 67108860(268435440); kvend = 45083392(180333568); length =
>> 22025469/16777216
>> 2015-06-13 10:26:50,360 INFO [main] org.apache.hadoop.mapred.MapTask:
>> (EQUATOR) 148671936 kvi 37167980(148671920)
>> 2015-06-13 10:26:54,802 INFO [SpillThread]
>> org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor
>>[.snappy]
>> 2015-06-13 10:26:54,883 INFO [SpillThread]
>> org.apache.kylin.job.hadoop.AbstractHadoopJob: The absolute path for
>>meta
>> dir is
>>
>>/yarn/nm/usercache/biops/appcache/application_1433833901375_0139/containe
>>r_1433833901375_0139_01_000002/meta
>> 2015-06-13 10:27:09,614 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5600000
>>records!
>> 2015-06-13 10:27:30,135 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5700000
>>records!
>> 2015-06-13 10:27:50,657 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5800000
>>records!
>> 2015-06-13 10:27:59,691 INFO [SpillThread]
>> org.apache.kylin.job.hadoop.cube.CuboidReducer: Handled 100000 records!
>> 2015-06-13 10:28:11,185 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5900000
>>records!
>> 2015-06-13 10:28:31,751 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6000000
>>records!
>> 2015-06-13 10:28:52,303 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6100000
>>records!
>> 2015-06-13 10:29:03,235 INFO [SpillThread]
>> org.apache.kylin.job.hadoop.cube.CuboidReducer: Handled 200000 records!
>> 2015-06-13 10:29:12,873 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6200000
>>records!
>> 2015-06-13 10:29:33,424 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6300000
>>records!
>> 2015-06-13 10:29:54,168 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6400000
>>records!
>> 2015-06-13 10:30:14,771 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6500000
>>records!
>> 2015-06-13 10:30:18,479 INFO [SpillThread]
>> org.apache.kylin.job.hadoop.cube.CuboidReducer: Handled 300000 records!
>> 2015-06-13 10:30:35,325 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6600000
>>records!
>> 2015-06-13 10:30:39,831 INFO [SpillThread]
>> org.apache.hadoop.mapred.MapTask: Finished spill 0
>> 2015-06-13 10:30:39,831 INFO [main] org.apache.hadoop.mapred.MapTask:
>> (RESET) equator 148671936 kv 37167980(148671920) kvi 32705668(130822672)
>> 2015-06-13 10:30:55,807 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6700000
>>records!
>> 2015-06-13 10:31:16,301 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6800000
>>records!
>> 2015-06-13 10:31:36,804 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6900000
>>records!
>> 2015-06-13 10:31:57,302 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7000000
>>records!
>> 2015-06-13 10:32:17,790 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7100000
>>records!
>> 2015-06-13 10:32:38,294 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7200000
>>records!
>> 2015-06-13 10:32:58,830 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7300000
>>records!
>> 2015-06-13 10:33:19,354 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7400000
>>records!
>> 2015-06-13 10:33:39,866 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7500000
>>records!
>> 2015-06-13 10:34:00,371 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7600000
>>records!
>> 2015-06-13 10:34:20,872 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7700000
>>records!
>> 2015-06-13 10:34:41,387 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7800000
>>records!
>> 2015-06-13 10:35:01,884 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7900000
>>records!
>> 2015-06-13 10:35:22,368 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8000000
>>records!
>> 2015-06-13 10:35:42,868 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8100000
>>records!
>> 2015-06-13 10:36:03,361 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8200000
>>records!
>> 2015-06-13 10:36:23,981 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8300000
>>records!
>> 2015-06-13 10:36:44,529 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8400000
>>records!
>> 2015-06-13 10:37:05,034 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8500000
>>records!
>> 2015-06-13 10:37:25,609 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8600000
>>records!
>> 2015-06-13 10:37:46,152 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8700000
>>records!
>> 2015-06-13 10:38:06,725 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8800000
>>records!
>> 2015-06-13 10:38:27,296 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8900000
>>records!
>> 2015-06-13 10:38:47,905 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9000000
>>records!
>> 2015-06-13 10:39:08,430 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9100000
>>records!
>> 2015-06-13 10:39:28,952 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9200000
>>records!
>> 2015-06-13 10:39:49,460 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9300000
>>records!
>> 2015-06-13 10:40:09,987 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9400000
>>records!
>> 2015-06-13 10:40:30,551 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9500000
>>records!
>> 2015-06-13 10:40:51,080 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9600000
>>records!
>> 2015-06-13 10:41:11,579 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9700000
>>records!
>> 2015-06-13 10:41:32,122 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9800000
>>records!
>> 2015-06-13 10:41:52,623 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9900000
>>records!
>> 2015-06-13 10:42:13,128 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10000000
>>records!
>> 2015-06-13 10:42:33,628 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10100000
>>records!
>> 2015-06-13 10:42:54,128 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10200000
>>records!
>> 2015-06-13 10:43:14,629 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10300000
>>records!
>> 2015-06-13 10:43:35,149 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10400000
>>records!
>> 2015-06-13 10:43:55,764 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10500000
>>records!
>> 2015-06-13 10:44:16,303 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10600000
>>records!
>> 2015-06-13 10:44:36,857 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10700000
>>records!
>> 2015-06-13 10:44:57,349 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10800000
>>records!
>> 2015-06-13 10:45:17,840 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10900000
>>records!
>> 2015-06-13 10:45:38,441 INFO [main]
>> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 11000000
>>records!
>> 2015-06-13 10:45:41,057 INFO [main] org.apache.hadoop.mapred.MapTask:
>> Spilling map output
>> 2015-06-13 10:45:41,057 INFO [main] org.apache.hadoop.mapred.MapTask:
>> bufstart = 148671936; bufend = 6882957; bufvoid = 268435443
>> 2015-06-13 10:45:41,057 INFO [main] org.apache.hadoop.mapred.MapTask:
>> kvstart = 37167980(148671920); kvend = 15142512(60570048); length =
>> 22025469/16777216
>> 2015-06-13 10:45:41,057 INFO [main] org.apache.hadoop.mapred.MapTask:
>> (EQUATOR) 28908413 kvi 7227096(28908384)
>>
>> Thanks,
>>
>> On Mon, Jun 15, 2015 at 3:11 PM, 周千昊 <qh...@apache.org> wrote:
>>
>>> Hi Mishra
>>> It is as what you described, since the data size is 30MB only,
>>>hadoop
>>> will only manage to run MR in single mapper and reduce.
>>> As for your question about why it takes so long to run on a small
>>> dataset, can you please dig into the web page of *map reduce task
>>>status
>>> *to
>>> check how much time the MR job really take, so that we can make sure if
>>> the
>>> time is consuming on the MR job or kylin job scheduling module
>>>
>>> Vineet Mishra <cl...@gmail.com>于2015年6月15日周一 下午4:51写道：
>>>
>>> > Shi,
>>> >
>>> > Hadoop is setup correctly on my cluster with the default block size
>>>of
>>> > 128Mb and its indeed very much running multiple Mapper/Reducer based
>>> jobs
>>> > for other cases.
>>> >
>>> > Its the only Kylin Cube building which is running through single M/R
>>> job.
>>> >
>>> > Moreover to my surprise, the 4th running job which is Build Base
>>>Cuboid
>>> > Data shows Data size as 30Mb, is it the reason due to which single
>>> mapper
>>> > is getting invoked, if that being the case then even to process such
>>>a
>>> > small data set why is it taking around 50min.
>>> >
>>> > Thanks,
>>> >
>>> > On Mon, Jun 15, 2015 at 11:17 AM, Shi, Shaofeng <sh...@ebay.com>
>>> wrote:
>>> >
>>> > > Yes you can adjust these parameters, for example give a smaller
>>>value
>>> for
>>> > > kylin.job.mapreduce.default.reduce.input.mb; but it only affects
>>>the
>>> > > reducer number;
>>> > >
>>> > > I suggest you investigate why there is only 1 mapper be started;
>>>Some
>>> > > factors like hadoop cluster size, HDFS file block size will impact
>>> this;
>>> > > You can run a SQL (a query which need run MR, not simple select *)
>>> with
>>> > > hive -e, and then use the MR job track URL to see how many mappers
>>>be
>>> > > triggered; If it is still single, then the problem is in your
>>>hadoop
>>> > > configuration; Otherwise it may exists in Kylin, check if you put
>>>some
>>> > > additional parameter in conf/kylin_job_conf.xml.
>>> > >
>>> > >
>>> > > On 6/15/15, 2:52 AM, "Vineet Mishra" <cl...@gmail.com>
>>>wrote:
>>> > >
>>> > > >Can I have specification for these properties?
>>> > > >
>>> > > >KYLIN_JOB_MAPREDUCE_DEFAULT_REDUCE_COUNT_RATIO =
>>> > > >"kylin.job.mapreduce.default.reduce.count.ratio";
>>> > > >KYLIN_JOB_MAPREDUCE_DEFAULT_REDUCE_INPUT_MB =
>>> > > >"kylin.job.mapreduce.default.reduce.input.mb";
>>> > > >KYLIN_JOB_MAPREDUCE_MAX_REDUCER_NUMBER =
>>> > > >"kylin.job.mapreduce.max.reducer.number";
>>> > > >
>>> > > >Thanks!
>>> > > >
>>> > > >On Sun, Jun 14, 2015 at 11:59 PM, Vineet Mishra <
>>> clearmidoubt@gmail.com
>>> > >
>>> > > >wrote:
>>> > > >
>>> > > >> Hi Shi,
>>> > > >>
>>> > > >> Its alright!
>>> > > >> So I was wondering my source hive Table is around 3 Gb, despite
>>>of
>>> my
>>> > > >>hive
>>> > > >> table being partitioned and holding the data around 50-70 Mb per
>>> > > >>partition
>>> > > >> the Mapper and Reducer getting spawned are single. The amount of
>>> data
>>> > > >>that
>>> > > >> is being processed in the M/R is nothing as expected but it
>>>takes
>>> hell
>>> > > >>lot
>>> > > >> of time.
>>> > > >>
>>> > > >> As mentioned in the trailing mail that the job is getting very
>>> slow,
>>> > the
>>> > > >> process Build Base Cuboid Data itself takes around 50mins to get
>>> > > >> completed.
>>> > > >>
>>> > > >> I can tweak the reducer parameter mentioned by you but do u
>>>think
>>> that
>>> > > >> will make a difference since the mapper is where the most of the
>>> time
>>> > is
>>> > > >> spent.
>>> > > >>
>>> > > >> Can you share your thoughts for performance tuning for the cube
>>> build!
>>> > > >>
>>> > > >> Thanks!
>>> > > >>
>>> > > >> On Sun, Jun 14, 2015 at 7:26 PM, Shi, Shaofeng
>>><sh...@ebay.com>
>>> > > wrote:
>>> > > >>
>>> > > >>> Hi, sorry, a busy weekend;
>>> > > >>>
>>> > > >>> Usually Kylin will request proper number of mapper and
>>>reducers;
>>> If
>>> > you
>>> > > >>> see single mapper/recudder, how much of your input and output?
>>>If
>>> > your
>>> > > >>> cube is quite small, single mapper/reducer is possible;
>>> > > >>>
>>> > > >>> Number of mappers is decided by the FileInputFormat; But
>>>number of
>>> > > >>>reducer
>>> > > >>> was set by Kylin, see:
>>> > > >>>
>>> > > >>>
>>> > > >>>
>>> > >
>>>
>>>https://github.com/apache/incubator-kylin/blob/master/job/src/main/java/
>>> > > >>>org
>>> > > >>> /apache/kylin/job/hadoop/cube/CuboidJob.java#L141
>>> > > >>>
>>> > > >>><
>>> > >
>>> https://github.com/apache/incubator-kylin/blob/master/job/src/main/java
>>> > > >>>/org/apache/kylin/job/hadoop/cube/CuboidJob.java#L141>
>>> > > >>>
>>> > > >>>
>>> > > >>>
>>> > > >>>
>>> > > >>> On 6/14/15, 5:25 PM, "Vineet Mishra" <cl...@gmail.com>
>>> wrote:
>>> > > >>>
>>> > > >>> >Urgent call, any follow up on this?
>>> > > >>> >
>>> > > >>> >On Fri, Jun 12, 2015 at 6:46 PM, Vineet Mishra
>>> > > >>><cl...@gmail.com>
>>> > > >>> >wrote:
>>> > > >>> >
>>> > > >>> >>
>>> > > >>> >> Why org.apache.kylin.job.hadoop.cube.CuboidReducer is
>>>running
>>> > Single
>>> > > >>> >> Mapper/Reducer for the job. Can I have the understanding
>>>behind
>>> > the
>>> > > >>> >>reason
>>> > > >>> >> of running it as single mapper/reducer.
>>> > > >>> >>
>>> > > >>> >> Thanks!
>>> > > >>> >>
>>> > > >>> >> On Fri, Jun 12, 2015 at 6:30 PM, Vineet Mishra
>>> > > >>><clearmidoubt@gmail.com
>>> > > >>> >
>>> > > >>> >> wrote:
>>> > > >>> >>
>>> > > >>> >>> Hi All,
>>> > > >>> >>>
>>> > > >>> >>> I am building a cube using Kylin and I could see that the
>>>job
>>> is
>>> > > >>> >>>running
>>> > > >>> >>> with Single Mapper and Reducer for some of the intermediate
>>> > process
>>> > > >>> >>>such as
>>> > > >>> >>>
>>> > > >>> >>> Extract Fact Table Distinct Columns
>>> > > >>> >>> Build Dimension Dictionary
>>> > > >>> >>> Build N-Dimension Cuboid
>>> > > >>> >>>
>>> > > >>> >>> I am not sure what's the reason behind running the job with
>>> > single
>>> > > >>> M/R,
>>> > > >>> >>> is it really necessary or is it some default config. which
>>> can be
>>> > > >>> >>>tweaked,
>>> > > >>> >>> its 70 Mins and the job status is 25% !
>>> > > >>> >>>
>>> > > >>> >>> Urgent Call!
>>> > > >>> >>>
>>> > > >>> >>> Thanks!
>>> > > >>> >>>
>>> > > >>> >>
>>> > > >>> >>
>>> > > >>>
>>> > > >>>
>>> > > >>
>>> > >
>>> > >
>>> >
>>>
>>
>>
Re: Jobs Running with Single Mapper/Reducer
Posted by Vineet Mishra <cl...@gmail.com>.
Hi All,
Well I got it through.
Actually it was basically the pre-process which was converting my 3Gb hive
table to 60Mb sequence file which due to block size of 128Mb was running in
a single Mapper job.
I changed the split size to 20Mb and it was able to spawn 3 Mapper after
that.
Anyways, thanks all for your quick response.
Thanks!
On Mon, Jun 15, 2015 at 3:27 PM, Vineet Mishra <cl...@gmail.com>
wrote:
> Hi,
>
> Sorry I misunderstood this value, actually 30Mb was the output for that
> job. But I guess I have the got reason, (not exactly confirmed) as why its
> behaving like this, the input to this job is sequence file which is 60Mb in
> size and compressed.
>
> No doubt when it will be passed to the mapper, it is very much certain to
> have the single mapper invoked due to the pre-configured block size. Now I
> need to check If I can override the splitSize property for the job.
>
> Moreover I can see that the job is running seamlessly slow, kindly find
> the stack trace for the job mentioned below.
>
> 2015-06-13 10:08:00,940 INFO [main] org.apache.hadoop.mapred.MapTask:
> Processing split: hdfs://
> dev-hadoop-namenode.com:8020/tmp/kylin-9b675c66-4ce9-4a33-a356-2ccf9dbaca6a/kylin_intermediate_xyz_20150401000000_20150611000000_9b675c66_4ce9_4a33_a356_2ccf9dbaca6a/000000_0:0+60440271
> 2015-06-13 10:08:01,014 INFO [main] org.apache.hadoop.mapred.MapTask:
> (EQUATOR) 0 kvi 67108860(268435440)
> 2015-06-13 10:08:01,015 INFO [main] org.apache.hadoop.mapred.MapTask:
> mapreduce.task.io.sort.mb: 256
> 2015-06-13 10:08:01,015 INFO [main] org.apache.hadoop.mapred.MapTask: soft
> limit at 214748368
> 2015-06-13 10:08:01,015 INFO [main] org.apache.hadoop.mapred.MapTask:
> bufstart = 0; bufvoid = 268435456
> 2015-06-13 10:08:01,015 INFO [main] org.apache.hadoop.mapred.MapTask:
> kvstart = 67108860; length = 16777216
> 2015-06-13 10:08:01,023 INFO [main] org.apache.hadoop.mapred.MapTask: Map
> output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
> 2015-06-13 10:08:01,048 INFO [main]
> org.apache.hadoop.io.compress.zlib.ZlibFactory: Successfully loaded &
> initialized native-zlib library
> 2015-06-13 10:08:01,048 INFO [main]
> org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
> [.deflate]
> 2015-06-13 10:08:01,052 INFO [main]
> org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
> [.deflate]
> 2015-06-13 10:08:01,052 INFO [main]
> org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
> [.deflate]
> 2015-06-13 10:08:01,053 INFO [main]
> org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
> [.deflate]
> 2015-06-13 10:08:01,058 INFO [main]
> org.apache.kylin.job.hadoop.AbstractHadoopJob: The absolute path for meta
> dir is
> /yarn/nm/usercache/biops/appcache/application_1433833901375_0139/container_1433833901375_0139_01_000002/meta
> 2015-06-13 10:08:01,059 INFO [main] org.apache.kylin.common.KylinConfig:
> Use
> KYLIN_CONF=/yarn/nm/usercache/biops/appcache/application_1433833901375_0139/container_1433833901375_0139_01_000002/meta
> 2015-06-13 10:08:01,084 INFO [main] org.apache.kylin.cube.CubeManager:
> Initializing CubeManager with config
> /yarn/nm/usercache/biops/filecache/19452/meta
> 2015-06-13 10:08:01,086 INFO [main]
> org.apache.kylin.common.persistence.ResourceStore: Using metadata url
> /yarn/nm/usercache/biops/filecache/19452/meta for resource store
> 2015-06-13 10:08:01,327 INFO [main] org.apache.kylin.cube.CubeDescManager:
> Initializing CubeDescManager with config
> /yarn/nm/usercache/biops/filecache/19452/meta
> 2015-06-13 10:08:01,327 INFO [main] org.apache.kylin.cube.CubeDescManager:
> Reloading Cube Metadata from folder
> /yarn/nm/usercache/biops/filecache/19452/meta/cube_desc
> 2015-06-13 10:08:22,834 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 100000 records!
> 2015-06-13 10:08:43,225 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 200000 records!
> 2015-06-13 10:09:03,622 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 300000 records!
> 2015-06-13 10:09:23,999 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 400000 records!
> 2015-06-13 10:09:44,372 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 500000 records!
> 2015-06-13 10:10:04,780 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 600000 records!
> 2015-06-13 10:10:25,143 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 700000 records!
> 2015-06-13 10:10:45,512 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 800000 records!
> 2015-06-13 10:11:05,895 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 900000 records!
> 2015-06-13 10:11:26,284 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1000000 records!
> 2015-06-13 10:11:46,716 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1100000 records!
> 2015-06-13 10:12:07,174 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1200000 records!
> 2015-06-13 10:12:27,646 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1300000 records!
> 2015-06-13 10:12:48,127 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1400000 records!
> 2015-06-13 10:13:08,614 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1500000 records!
> 2015-06-13 10:13:29,148 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1600000 records!
> 2015-06-13 10:13:49,686 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1700000 records!
> 2015-06-13 10:14:10,167 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1800000 records!
> 2015-06-13 10:14:30,652 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1900000 records!
> 2015-06-13 10:14:51,199 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2000000 records!
> 2015-06-13 10:15:11,719 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2100000 records!
> 2015-06-13 10:15:32,221 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2200000 records!
> 2015-06-13 10:15:52,717 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2300000 records!
> 2015-06-13 10:16:13,252 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2400000 records!
> 2015-06-13 10:16:33,769 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2500000 records!
> 2015-06-13 10:16:54,263 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2600000 records!
> 2015-06-13 10:17:14,741 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2700000 records!
> 2015-06-13 10:17:35,213 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2800000 records!
> 2015-06-13 10:17:55,699 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2900000 records!
> 2015-06-13 10:18:16,204 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3000000 records!
> 2015-06-13 10:18:36,721 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3100000 records!
> 2015-06-13 10:18:57,249 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3200000 records!
> 2015-06-13 10:19:17,743 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3300000 records!
> 2015-06-13 10:19:38,275 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3400000 records!
> 2015-06-13 10:19:58,812 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3500000 records!
> 2015-06-13 10:20:19,312 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3600000 records!
> 2015-06-13 10:20:39,920 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3700000 records!
> 2015-06-13 10:21:00,518 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3800000 records!
> 2015-06-13 10:21:21,008 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3900000 records!
> 2015-06-13 10:21:41,525 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4000000 records!
> 2015-06-13 10:22:02,023 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4100000 records!
> 2015-06-13 10:22:22,534 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4200000 records!
> 2015-06-13 10:22:43,063 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4300000 records!
> 2015-06-13 10:23:03,558 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4400000 records!
> 2015-06-13 10:23:24,043 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4500000 records!
> 2015-06-13 10:23:44,537 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4600000 records!
> 2015-06-13 10:24:05,015 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4700000 records!
> 2015-06-13 10:24:25,490 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4800000 records!
> 2015-06-13 10:24:45,995 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4900000 records!
> 2015-06-13 10:25:06,487 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5000000 records!
> 2015-06-13 10:25:26,964 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5100000 records!
> 2015-06-13 10:25:47,524 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5200000 records!
> 2015-06-13 10:26:08,032 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5300000 records!
> 2015-06-13 10:26:28,514 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5400000 records!
> 2015-06-13 10:26:49,053 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5500000 records!
> 2015-06-13 10:26:50,360 INFO [main] org.apache.hadoop.mapred.MapTask:
> Spilling map output
> 2015-06-13 10:26:50,360 INFO [main] org.apache.hadoop.mapred.MapTask:
> bufstart = 0; bufend = 126646464; bufvoid = 268435456
> 2015-06-13 10:26:50,360 INFO [main] org.apache.hadoop.mapred.MapTask:
> kvstart = 67108860(268435440); kvend = 45083392(180333568); length =
> 22025469/16777216
> 2015-06-13 10:26:50,360 INFO [main] org.apache.hadoop.mapred.MapTask:
> (EQUATOR) 148671936 kvi 37167980(148671920)
> 2015-06-13 10:26:54,802 INFO [SpillThread]
> org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor [.snappy]
> 2015-06-13 10:26:54,883 INFO [SpillThread]
> org.apache.kylin.job.hadoop.AbstractHadoopJob: The absolute path for meta
> dir is
> /yarn/nm/usercache/biops/appcache/application_1433833901375_0139/container_1433833901375_0139_01_000002/meta
> 2015-06-13 10:27:09,614 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5600000 records!
> 2015-06-13 10:27:30,135 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5700000 records!
> 2015-06-13 10:27:50,657 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5800000 records!
> 2015-06-13 10:27:59,691 INFO [SpillThread]
> org.apache.kylin.job.hadoop.cube.CuboidReducer: Handled 100000 records!
> 2015-06-13 10:28:11,185 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5900000 records!
> 2015-06-13 10:28:31,751 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6000000 records!
> 2015-06-13 10:28:52,303 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6100000 records!
> 2015-06-13 10:29:03,235 INFO [SpillThread]
> org.apache.kylin.job.hadoop.cube.CuboidReducer: Handled 200000 records!
> 2015-06-13 10:29:12,873 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6200000 records!
> 2015-06-13 10:29:33,424 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6300000 records!
> 2015-06-13 10:29:54,168 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6400000 records!
> 2015-06-13 10:30:14,771 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6500000 records!
> 2015-06-13 10:30:18,479 INFO [SpillThread]
> org.apache.kylin.job.hadoop.cube.CuboidReducer: Handled 300000 records!
> 2015-06-13 10:30:35,325 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6600000 records!
> 2015-06-13 10:30:39,831 INFO [SpillThread]
> org.apache.hadoop.mapred.MapTask: Finished spill 0
> 2015-06-13 10:30:39,831 INFO [main] org.apache.hadoop.mapred.MapTask:
> (RESET) equator 148671936 kv 37167980(148671920) kvi 32705668(130822672)
> 2015-06-13 10:30:55,807 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6700000 records!
> 2015-06-13 10:31:16,301 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6800000 records!
> 2015-06-13 10:31:36,804 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6900000 records!
> 2015-06-13 10:31:57,302 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7000000 records!
> 2015-06-13 10:32:17,790 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7100000 records!
> 2015-06-13 10:32:38,294 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7200000 records!
> 2015-06-13 10:32:58,830 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7300000 records!
> 2015-06-13 10:33:19,354 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7400000 records!
> 2015-06-13 10:33:39,866 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7500000 records!
> 2015-06-13 10:34:00,371 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7600000 records!
> 2015-06-13 10:34:20,872 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7700000 records!
> 2015-06-13 10:34:41,387 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7800000 records!
> 2015-06-13 10:35:01,884 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7900000 records!
> 2015-06-13 10:35:22,368 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8000000 records!
> 2015-06-13 10:35:42,868 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8100000 records!
> 2015-06-13 10:36:03,361 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8200000 records!
> 2015-06-13 10:36:23,981 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8300000 records!
> 2015-06-13 10:36:44,529 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8400000 records!
> 2015-06-13 10:37:05,034 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8500000 records!
> 2015-06-13 10:37:25,609 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8600000 records!
> 2015-06-13 10:37:46,152 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8700000 records!
> 2015-06-13 10:38:06,725 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8800000 records!
> 2015-06-13 10:38:27,296 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8900000 records!
> 2015-06-13 10:38:47,905 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9000000 records!
> 2015-06-13 10:39:08,430 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9100000 records!
> 2015-06-13 10:39:28,952 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9200000 records!
> 2015-06-13 10:39:49,460 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9300000 records!
> 2015-06-13 10:40:09,987 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9400000 records!
> 2015-06-13 10:40:30,551 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9500000 records!
> 2015-06-13 10:40:51,080 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9600000 records!
> 2015-06-13 10:41:11,579 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9700000 records!
> 2015-06-13 10:41:32,122 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9800000 records!
> 2015-06-13 10:41:52,623 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9900000 records!
> 2015-06-13 10:42:13,128 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10000000 records!
> 2015-06-13 10:42:33,628 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10100000 records!
> 2015-06-13 10:42:54,128 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10200000 records!
> 2015-06-13 10:43:14,629 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10300000 records!
> 2015-06-13 10:43:35,149 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10400000 records!
> 2015-06-13 10:43:55,764 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10500000 records!
> 2015-06-13 10:44:16,303 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10600000 records!
> 2015-06-13 10:44:36,857 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10700000 records!
> 2015-06-13 10:44:57,349 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10800000 records!
> 2015-06-13 10:45:17,840 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10900000 records!
> 2015-06-13 10:45:38,441 INFO [main]
> org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 11000000 records!
> 2015-06-13 10:45:41,057 INFO [main] org.apache.hadoop.mapred.MapTask:
> Spilling map output
> 2015-06-13 10:45:41,057 INFO [main] org.apache.hadoop.mapred.MapTask:
> bufstart = 148671936; bufend = 6882957; bufvoid = 268435443
> 2015-06-13 10:45:41,057 INFO [main] org.apache.hadoop.mapred.MapTask:
> kvstart = 37167980(148671920); kvend = 15142512(60570048); length =
> 22025469/16777216
> 2015-06-13 10:45:41,057 INFO [main] org.apache.hadoop.mapred.MapTask:
> (EQUATOR) 28908413 kvi 7227096(28908384)
>
> Thanks,
>
> On Mon, Jun 15, 2015 at 3:11 PM, 周千昊 <qh...@apache.org> wrote:
>
>> Hi Mishra
>> It is as what you described, since the data size is 30MB only, hadoop
>> will only manage to run MR in single mapper and reduce.
>> As for your question about why it takes so long to run on a small
>> dataset, can you please dig into the web page of *map reduce task status
>> *to
>> check how much time the MR job really take, so that we can make sure if
>> the
>> time is consuming on the MR job or kylin job scheduling module
>>
>> Vineet Mishra <cl...@gmail.com>于2015年6月15日周一 下午4:51写道：
>>
>> > Shi,
>> >
>> > Hadoop is setup correctly on my cluster with the default block size of
>> > 128Mb and its indeed very much running multiple Mapper/Reducer based
>> jobs
>> > for other cases.
>> >
>> > Its the only Kylin Cube building which is running through single M/R
>> job.
>> >
>> > Moreover to my surprise, the 4th running job which is Build Base Cuboid
>> > Data shows Data size as 30Mb, is it the reason due to which single
>> mapper
>> > is getting invoked, if that being the case then even to process such a
>> > small data set why is it taking around 50min.
>> >
>> > Thanks,
>> >
>> > On Mon, Jun 15, 2015 at 11:17 AM, Shi, Shaofeng <sh...@ebay.com>
>> wrote:
>> >
>> > > Yes you can adjust these parameters, for example give a smaller value
>> for
>> > > kylin.job.mapreduce.default.reduce.input.mb; but it only affects the
>> > > reducer number;
>> > >
>> > > I suggest you investigate why there is only 1 mapper be started; Some
>> > > factors like hadoop cluster size, HDFS file block size will impact
>> this;
>> > > You can run a SQL (a query which need run MR, not simple select *)
>> with
>> > > hive -e, and then use the MR job track URL to see how many mappers be
>> > > triggered; If it is still single, then the problem is in your hadoop
>> > > configuration; Otherwise it may exists in Kylin, check if you put some
>> > > additional parameter in conf/kylin_job_conf.xml.
>> > >
>> > >
>> > > On 6/15/15, 2:52 AM, "Vineet Mishra" <cl...@gmail.com> wrote:
>> > >
>> > > >Can I have specification for these properties?
>> > > >
>> > > >KYLIN_JOB_MAPREDUCE_DEFAULT_REDUCE_COUNT_RATIO =
>> > > >"kylin.job.mapreduce.default.reduce.count.ratio";
>> > > >KYLIN_JOB_MAPREDUCE_DEFAULT_REDUCE_INPUT_MB =
>> > > >"kylin.job.mapreduce.default.reduce.input.mb";
>> > > >KYLIN_JOB_MAPREDUCE_MAX_REDUCER_NUMBER =
>> > > >"kylin.job.mapreduce.max.reducer.number";
>> > > >
>> > > >Thanks!
>> > > >
>> > > >On Sun, Jun 14, 2015 at 11:59 PM, Vineet Mishra <
>> clearmidoubt@gmail.com
>> > >
>> > > >wrote:
>> > > >
>> > > >> Hi Shi,
>> > > >>
>> > > >> Its alright!
>> > > >> So I was wondering my source hive Table is around 3 Gb, despite of
>> my
>> > > >>hive
>> > > >> table being partitioned and holding the data around 50-70 Mb per
>> > > >>partition
>> > > >> the Mapper and Reducer getting spawned are single. The amount of
>> data
>> > > >>that
>> > > >> is being processed in the M/R is nothing as expected but it takes
>> hell
>> > > >>lot
>> > > >> of time.
>> > > >>
>> > > >> As mentioned in the trailing mail that the job is getting very
>> slow,
>> > the
>> > > >> process Build Base Cuboid Data itself takes around 50mins to get
>> > > >> completed.
>> > > >>
>> > > >> I can tweak the reducer parameter mentioned by you but do u think
>> that
>> > > >> will make a difference since the mapper is where the most of the
>> time
>> > is
>> > > >> spent.
>> > > >>
>> > > >> Can you share your thoughts for performance tuning for the cube
>> build!
>> > > >>
>> > > >> Thanks!
>> > > >>
>> > > >> On Sun, Jun 14, 2015 at 7:26 PM, Shi, Shaofeng <sh...@ebay.com>
>> > > wrote:
>> > > >>
>> > > >>> Hi, sorry, a busy weekend;
>> > > >>>
>> > > >>> Usually Kylin will request proper number of mapper and reducers;
>> If
>> > you
>> > > >>> see single mapper/recudder, how much of your input and output? If
>> > your
>> > > >>> cube is quite small, single mapper/reducer is possible;
>> > > >>>
>> > > >>> Number of mappers is decided by the FileInputFormat; But number of
>> > > >>>reducer
>> > > >>> was set by Kylin, see:
>> > > >>>
>> > > >>>
>> > > >>>
>> > >
>> https://github.com/apache/incubator-kylin/blob/master/job/src/main/java/
>> > > >>>org
>> > > >>> /apache/kylin/job/hadoop/cube/CuboidJob.java#L141
>> > > >>>
>> > > >>><
>> > >
>> https://github.com/apache/incubator-kylin/blob/master/job/src/main/java
>> > > >>>/org/apache/kylin/job/hadoop/cube/CuboidJob.java#L141>
>> > > >>>
>> > > >>>
>> > > >>>
>> > > >>>
>> > > >>> On 6/14/15, 5:25 PM, "Vineet Mishra" <cl...@gmail.com>
>> wrote:
>> > > >>>
>> > > >>> >Urgent call, any follow up on this?
>> > > >>> >
>> > > >>> >On Fri, Jun 12, 2015 at 6:46 PM, Vineet Mishra
>> > > >>><cl...@gmail.com>
>> > > >>> >wrote:
>> > > >>> >
>> > > >>> >>
>> > > >>> >> Why org.apache.kylin.job.hadoop.cube.CuboidReducer is running
>> > Single
>> > > >>> >> Mapper/Reducer for the job. Can I have the understanding behind
>> > the
>> > > >>> >>reason
>> > > >>> >> of running it as single mapper/reducer.
>> > > >>> >>
>> > > >>> >> Thanks!
>> > > >>> >>
>> > > >>> >> On Fri, Jun 12, 2015 at 6:30 PM, Vineet Mishra
>> > > >>><clearmidoubt@gmail.com
>> > > >>> >
>> > > >>> >> wrote:
>> > > >>> >>
>> > > >>> >>> Hi All,
>> > > >>> >>>
>> > > >>> >>> I am building a cube using Kylin and I could see that the job
>> is
>> > > >>> >>>running
>> > > >>> >>> with Single Mapper and Reducer for some of the intermediate
>> > process
>> > > >>> >>>such as
>> > > >>> >>>
>> > > >>> >>> Extract Fact Table Distinct Columns
>> > > >>> >>> Build Dimension Dictionary
>> > > >>> >>> Build N-Dimension Cuboid
>> > > >>> >>>
>> > > >>> >>> I am not sure what's the reason behind running the job with
>> > single
>> > > >>> M/R,
>> > > >>> >>> is it really necessary or is it some default config. which
>> can be
>> > > >>> >>>tweaked,
>> > > >>> >>> its 70 Mins and the job status is 25% !
>> > > >>> >>>
>> > > >>> >>> Urgent Call!
>> > > >>> >>>
>> > > >>> >>> Thanks!
>> > > >>> >>>
>> > > >>> >>
>> > > >>> >>
>> > > >>>
>> > > >>>
>> > > >>
>> > >
>> > >
>> >
>>
>
>
Re: Jobs Running with Single Mapper/Reducer
Posted by Vineet Mishra <cl...@gmail.com>.
Hi,
Sorry I misunderstood this value, actually 30Mb was the output for that
job. But I guess I have the got reason, (not exactly confirmed) as why its
behaving like this, the input to this job is sequence file which is 60Mb in
size and compressed.
No doubt when it will be passed to the mapper, it is very much certain to
have the single mapper invoked due to the pre-configured block size. Now I
need to check If I can override the splitSize property for the job.
Moreover I can see that the job is running seamlessly slow, kindly find the
stack trace for the job mentioned below.
2015-06-13 10:08:00,940 INFO [main] org.apache.hadoop.mapred.MapTask:
Processing split: hdfs://
dev-hadoop-namenode.com:8020/tmp/kylin-9b675c66-4ce9-4a33-a356-2ccf9dbaca6a/kylin_intermediate_xyz_20150401000000_20150611000000_9b675c66_4ce9_4a33_a356_2ccf9dbaca6a/000000_0:0+60440271
2015-06-13 10:08:01,014 INFO [main] org.apache.hadoop.mapred.MapTask:
(EQUATOR) 0 kvi 67108860(268435440)
2015-06-13 10:08:01,015 INFO [main] org.apache.hadoop.mapred.MapTask:
mapreduce.task.io.sort.mb: 256
2015-06-13 10:08:01,015 INFO [main] org.apache.hadoop.mapred.MapTask: soft
limit at 214748368
2015-06-13 10:08:01,015 INFO [main] org.apache.hadoop.mapred.MapTask:
bufstart = 0; bufvoid = 268435456
2015-06-13 10:08:01,015 INFO [main] org.apache.hadoop.mapred.MapTask:
kvstart = 67108860; length = 16777216
2015-06-13 10:08:01,023 INFO [main] org.apache.hadoop.mapred.MapTask: Map
output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-06-13 10:08:01,048 INFO [main]
org.apache.hadoop.io.compress.zlib.ZlibFactory: Successfully loaded &
initialized native-zlib library
2015-06-13 10:08:01,048 INFO [main]
org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
[.deflate]
2015-06-13 10:08:01,052 INFO [main]
org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
[.deflate]
2015-06-13 10:08:01,052 INFO [main]
org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
[.deflate]
2015-06-13 10:08:01,053 INFO [main]
org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor
[.deflate]
2015-06-13 10:08:01,058 INFO [main]
org.apache.kylin.job.hadoop.AbstractHadoopJob: The absolute path for meta
dir is
/yarn/nm/usercache/biops/appcache/application_1433833901375_0139/container_1433833901375_0139_01_000002/meta
2015-06-13 10:08:01,059 INFO [main] org.apache.kylin.common.KylinConfig:
Use
KYLIN_CONF=/yarn/nm/usercache/biops/appcache/application_1433833901375_0139/container_1433833901375_0139_01_000002/meta
2015-06-13 10:08:01,084 INFO [main] org.apache.kylin.cube.CubeManager:
Initializing CubeManager with config
/yarn/nm/usercache/biops/filecache/19452/meta
2015-06-13 10:08:01,086 INFO [main]
org.apache.kylin.common.persistence.ResourceStore: Using metadata url
/yarn/nm/usercache/biops/filecache/19452/meta for resource store
2015-06-13 10:08:01,327 INFO [main] org.apache.kylin.cube.CubeDescManager:
Initializing CubeDescManager with config
/yarn/nm/usercache/biops/filecache/19452/meta
2015-06-13 10:08:01,327 INFO [main] org.apache.kylin.cube.CubeDescManager:
Reloading Cube Metadata from folder
/yarn/nm/usercache/biops/filecache/19452/meta/cube_desc
2015-06-13 10:08:22,834 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 100000 records!
2015-06-13 10:08:43,225 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 200000 records!
2015-06-13 10:09:03,622 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 300000 records!
2015-06-13 10:09:23,999 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 400000 records!
2015-06-13 10:09:44,372 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 500000 records!
2015-06-13 10:10:04,780 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 600000 records!
2015-06-13 10:10:25,143 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 700000 records!
2015-06-13 10:10:45,512 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 800000 records!
2015-06-13 10:11:05,895 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 900000 records!
2015-06-13 10:11:26,284 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1000000 records!
2015-06-13 10:11:46,716 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1100000 records!
2015-06-13 10:12:07,174 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1200000 records!
2015-06-13 10:12:27,646 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1300000 records!
2015-06-13 10:12:48,127 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1400000 records!
2015-06-13 10:13:08,614 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1500000 records!
2015-06-13 10:13:29,148 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1600000 records!
2015-06-13 10:13:49,686 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1700000 records!
2015-06-13 10:14:10,167 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1800000 records!
2015-06-13 10:14:30,652 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 1900000 records!
2015-06-13 10:14:51,199 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2000000 records!
2015-06-13 10:15:11,719 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2100000 records!
2015-06-13 10:15:32,221 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2200000 records!
2015-06-13 10:15:52,717 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2300000 records!
2015-06-13 10:16:13,252 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2400000 records!
2015-06-13 10:16:33,769 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2500000 records!
2015-06-13 10:16:54,263 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2600000 records!
2015-06-13 10:17:14,741 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2700000 records!
2015-06-13 10:17:35,213 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2800000 records!
2015-06-13 10:17:55,699 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 2900000 records!
2015-06-13 10:18:16,204 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3000000 records!
2015-06-13 10:18:36,721 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3100000 records!
2015-06-13 10:18:57,249 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3200000 records!
2015-06-13 10:19:17,743 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3300000 records!
2015-06-13 10:19:38,275 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3400000 records!
2015-06-13 10:19:58,812 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3500000 records!
2015-06-13 10:20:19,312 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3600000 records!
2015-06-13 10:20:39,920 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3700000 records!
2015-06-13 10:21:00,518 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3800000 records!
2015-06-13 10:21:21,008 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 3900000 records!
2015-06-13 10:21:41,525 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4000000 records!
2015-06-13 10:22:02,023 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4100000 records!
2015-06-13 10:22:22,534 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4200000 records!
2015-06-13 10:22:43,063 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4300000 records!
2015-06-13 10:23:03,558 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4400000 records!
2015-06-13 10:23:24,043 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4500000 records!
2015-06-13 10:23:44,537 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4600000 records!
2015-06-13 10:24:05,015 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4700000 records!
2015-06-13 10:24:25,490 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4800000 records!
2015-06-13 10:24:45,995 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 4900000 records!
2015-06-13 10:25:06,487 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5000000 records!
2015-06-13 10:25:26,964 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5100000 records!
2015-06-13 10:25:47,524 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5200000 records!
2015-06-13 10:26:08,032 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5300000 records!
2015-06-13 10:26:28,514 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5400000 records!
2015-06-13 10:26:49,053 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5500000 records!
2015-06-13 10:26:50,360 INFO [main] org.apache.hadoop.mapred.MapTask:
Spilling map output
2015-06-13 10:26:50,360 INFO [main] org.apache.hadoop.mapred.MapTask:
bufstart = 0; bufend = 126646464; bufvoid = 268435456
2015-06-13 10:26:50,360 INFO [main] org.apache.hadoop.mapred.MapTask:
kvstart = 67108860(268435440); kvend = 45083392(180333568); length =
22025469/16777216
2015-06-13 10:26:50,360 INFO [main] org.apache.hadoop.mapred.MapTask:
(EQUATOR) 148671936 kvi 37167980(148671920)
2015-06-13 10:26:54,802 INFO [SpillThread]
org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor [.snappy]
2015-06-13 10:26:54,883 INFO [SpillThread]
org.apache.kylin.job.hadoop.AbstractHadoopJob: The absolute path for meta
dir is
/yarn/nm/usercache/biops/appcache/application_1433833901375_0139/container_1433833901375_0139_01_000002/meta
2015-06-13 10:27:09,614 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5600000 records!
2015-06-13 10:27:30,135 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5700000 records!
2015-06-13 10:27:50,657 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5800000 records!
2015-06-13 10:27:59,691 INFO [SpillThread]
org.apache.kylin.job.hadoop.cube.CuboidReducer: Handled 100000 records!
2015-06-13 10:28:11,185 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 5900000 records!
2015-06-13 10:28:31,751 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6000000 records!
2015-06-13 10:28:52,303 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6100000 records!
2015-06-13 10:29:03,235 INFO [SpillThread]
org.apache.kylin.job.hadoop.cube.CuboidReducer: Handled 200000 records!
2015-06-13 10:29:12,873 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6200000 records!
2015-06-13 10:29:33,424 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6300000 records!
2015-06-13 10:29:54,168 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6400000 records!
2015-06-13 10:30:14,771 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6500000 records!
2015-06-13 10:30:18,479 INFO [SpillThread]
org.apache.kylin.job.hadoop.cube.CuboidReducer: Handled 300000 records!
2015-06-13 10:30:35,325 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6600000 records!
2015-06-13 10:30:39,831 INFO [SpillThread]
org.apache.hadoop.mapred.MapTask: Finished spill 0
2015-06-13 10:30:39,831 INFO [main] org.apache.hadoop.mapred.MapTask:
(RESET) equator 148671936 kv 37167980(148671920) kvi 32705668(130822672)
2015-06-13 10:30:55,807 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6700000 records!
2015-06-13 10:31:16,301 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6800000 records!
2015-06-13 10:31:36,804 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 6900000 records!
2015-06-13 10:31:57,302 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7000000 records!
2015-06-13 10:32:17,790 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7100000 records!
2015-06-13 10:32:38,294 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7200000 records!
2015-06-13 10:32:58,830 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7300000 records!
2015-06-13 10:33:19,354 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7400000 records!
2015-06-13 10:33:39,866 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7500000 records!
2015-06-13 10:34:00,371 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7600000 records!
2015-06-13 10:34:20,872 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7700000 records!
2015-06-13 10:34:41,387 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7800000 records!
2015-06-13 10:35:01,884 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 7900000 records!
2015-06-13 10:35:22,368 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8000000 records!
2015-06-13 10:35:42,868 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8100000 records!
2015-06-13 10:36:03,361 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8200000 records!
2015-06-13 10:36:23,981 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8300000 records!
2015-06-13 10:36:44,529 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8400000 records!
2015-06-13 10:37:05,034 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8500000 records!
2015-06-13 10:37:25,609 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8600000 records!
2015-06-13 10:37:46,152 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8700000 records!
2015-06-13 10:38:06,725 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8800000 records!
2015-06-13 10:38:27,296 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 8900000 records!
2015-06-13 10:38:47,905 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9000000 records!
2015-06-13 10:39:08,430 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9100000 records!
2015-06-13 10:39:28,952 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9200000 records!
2015-06-13 10:39:49,460 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9300000 records!
2015-06-13 10:40:09,987 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9400000 records!
2015-06-13 10:40:30,551 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9500000 records!
2015-06-13 10:40:51,080 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9600000 records!
2015-06-13 10:41:11,579 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9700000 records!
2015-06-13 10:41:32,122 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9800000 records!
2015-06-13 10:41:52,623 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 9900000 records!
2015-06-13 10:42:13,128 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10000000 records!
2015-06-13 10:42:33,628 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10100000 records!
2015-06-13 10:42:54,128 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10200000 records!
2015-06-13 10:43:14,629 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10300000 records!
2015-06-13 10:43:35,149 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10400000 records!
2015-06-13 10:43:55,764 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10500000 records!
2015-06-13 10:44:16,303 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10600000 records!
2015-06-13 10:44:36,857 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10700000 records!
2015-06-13 10:44:57,349 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10800000 records!
2015-06-13 10:45:17,840 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 10900000 records!
2015-06-13 10:45:38,441 INFO [main]
org.apache.kylin.job.hadoop.cube.BaseCuboidMapper: Handled 11000000 records!
2015-06-13 10:45:41,057 INFO [main] org.apache.hadoop.mapred.MapTask:
Spilling map output
2015-06-13 10:45:41,057 INFO [main] org.apache.hadoop.mapred.MapTask:
bufstart = 148671936; bufend = 6882957; bufvoid = 268435443
2015-06-13 10:45:41,057 INFO [main] org.apache.hadoop.mapred.MapTask:
kvstart = 37167980(148671920); kvend = 15142512(60570048); length =
22025469/16777216
2015-06-13 10:45:41,057 INFO [main] org.apache.hadoop.mapred.MapTask:
(EQUATOR) 28908413 kvi 7227096(28908384)
Thanks,
On Mon, Jun 15, 2015 at 3:11 PM, 周千昊 <qh...@apache.org> wrote:
> Hi Mishra
> It is as what you described, since the data size is 30MB only, hadoop
> will only manage to run MR in single mapper and reduce.
> As for your question about why it takes so long to run on a small
> dataset, can you please dig into the web page of *map reduce task status
> *to
> check how much time the MR job really take, so that we can make sure if the
> time is consuming on the MR job or kylin job scheduling module
>
> Vineet Mishra <cl...@gmail.com>于2015年6月15日周一 下午4:51写道：
>
> > Shi,
> >
> > Hadoop is setup correctly on my cluster with the default block size of
> > 128Mb and its indeed very much running multiple Mapper/Reducer based jobs
> > for other cases.
> >
> > Its the only Kylin Cube building which is running through single M/R job.
> >
> > Moreover to my surprise, the 4th running job which is Build Base Cuboid
> > Data shows Data size as 30Mb, is it the reason due to which single mapper
> > is getting invoked, if that being the case then even to process such a
> > small data set why is it taking around 50min.
> >
> > Thanks,
> >
> > On Mon, Jun 15, 2015 at 11:17 AM, Shi, Shaofeng <sh...@ebay.com>
> wrote:
> >
> > > Yes you can adjust these parameters, for example give a smaller value
> for
> > > kylin.job.mapreduce.default.reduce.input.mb; but it only affects the
> > > reducer number;
> > >
> > > I suggest you investigate why there is only 1 mapper be started; Some
> > > factors like hadoop cluster size, HDFS file block size will impact
> this;
> > > You can run a SQL (a query which need run MR, not simple select *) with
> > > hive -e, and then use the MR job track URL to see how many mappers be
> > > triggered; If it is still single, then the problem is in your hadoop
> > > configuration; Otherwise it may exists in Kylin, check if you put some
> > > additional parameter in conf/kylin_job_conf.xml.
> > >
> > >
> > > On 6/15/15, 2:52 AM, "Vineet Mishra" <cl...@gmail.com> wrote:
> > >
> > > >Can I have specification for these properties?
> > > >
> > > >KYLIN_JOB_MAPREDUCE_DEFAULT_REDUCE_COUNT_RATIO =
> > > >"kylin.job.mapreduce.default.reduce.count.ratio";
> > > >KYLIN_JOB_MAPREDUCE_DEFAULT_REDUCE_INPUT_MB =
> > > >"kylin.job.mapreduce.default.reduce.input.mb";
> > > >KYLIN_JOB_MAPREDUCE_MAX_REDUCER_NUMBER =
> > > >"kylin.job.mapreduce.max.reducer.number";
> > > >
> > > >Thanks!
> > > >
> > > >On Sun, Jun 14, 2015 at 11:59 PM, Vineet Mishra <
> clearmidoubt@gmail.com
> > >
> > > >wrote:
> > > >
> > > >> Hi Shi,
> > > >>
> > > >> Its alright!
> > > >> So I was wondering my source hive Table is around 3 Gb, despite of
> my
> > > >>hive
> > > >> table being partitioned and holding the data around 50-70 Mb per
> > > >>partition
> > > >> the Mapper and Reducer getting spawned are single. The amount of
> data
> > > >>that
> > > >> is being processed in the M/R is nothing as expected but it takes
> hell
> > > >>lot
> > > >> of time.
> > > >>
> > > >> As mentioned in the trailing mail that the job is getting very slow,
> > the
> > > >> process Build Base Cuboid Data itself takes around 50mins to get
> > > >> completed.
> > > >>
> > > >> I can tweak the reducer parameter mentioned by you but do u think
> that
> > > >> will make a difference since the mapper is where the most of the
> time
> > is
> > > >> spent.
> > > >>
> > > >> Can you share your thoughts for performance tuning for the cube
> build!
> > > >>
> > > >> Thanks!
> > > >>
> > > >> On Sun, Jun 14, 2015 at 7:26 PM, Shi, Shaofeng <sh...@ebay.com>
> > > wrote:
> > > >>
> > > >>> Hi, sorry, a busy weekend;
> > > >>>
> > > >>> Usually Kylin will request proper number of mapper and reducers; If
> > you
> > > >>> see single mapper/recudder, how much of your input and output? If
> > your
> > > >>> cube is quite small, single mapper/reducer is possible;
> > > >>>
> > > >>> Number of mappers is decided by the FileInputFormat; But number of
> > > >>>reducer
> > > >>> was set by Kylin, see:
> > > >>>
> > > >>>
> > > >>>
> > >
> https://github.com/apache/incubator-kylin/blob/master/job/src/main/java/
> > > >>>org
> > > >>> /apache/kylin/job/hadoop/cube/CuboidJob.java#L141
> > > >>>
> > > >>><
> > >
> https://github.com/apache/incubator-kylin/blob/master/job/src/main/java
> > > >>>/org/apache/kylin/job/hadoop/cube/CuboidJob.java#L141>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>> On 6/14/15, 5:25 PM, "Vineet Mishra" <cl...@gmail.com>
> wrote:
> > > >>>
> > > >>> >Urgent call, any follow up on this?
> > > >>> >
> > > >>> >On Fri, Jun 12, 2015 at 6:46 PM, Vineet Mishra
> > > >>><cl...@gmail.com>
> > > >>> >wrote:
> > > >>> >
> > > >>> >>
> > > >>> >> Why org.apache.kylin.job.hadoop.cube.CuboidReducer is running
> > Single
> > > >>> >> Mapper/Reducer for the job. Can I have the understanding behind
> > the
> > > >>> >>reason
> > > >>> >> of running it as single mapper/reducer.
> > > >>> >>
> > > >>> >> Thanks!
> > > >>> >>
> > > >>> >> On Fri, Jun 12, 2015 at 6:30 PM, Vineet Mishra
> > > >>><clearmidoubt@gmail.com
> > > >>> >
> > > >>> >> wrote:
> > > >>> >>
> > > >>> >>> Hi All,
> > > >>> >>>
> > > >>> >>> I am building a cube using Kylin and I could see that the job
> is
> > > >>> >>>running
> > > >>> >>> with Single Mapper and Reducer for some of the intermediate
> > process
> > > >>> >>>such as
> > > >>> >>>
> > > >>> >>> Extract Fact Table Distinct Columns
> > > >>> >>> Build Dimension Dictionary
> > > >>> >>> Build N-Dimension Cuboid
> > > >>> >>>
> > > >>> >>> I am not sure what's the reason behind running the job with
> > single
> > > >>> M/R,
> > > >>> >>> is it really necessary or is it some default config. which can
> be
> > > >>> >>>tweaked,
> > > >>> >>> its 70 Mins and the job status is 25% !
> > > >>> >>>
> > > >>> >>> Urgent Call!
> > > >>> >>>
> > > >>> >>> Thanks!
> > > >>> >>>
> > > >>> >>
> > > >>> >>
> > > >>>
> > > >>>
> > > >>
> > >
> > >
> >
>
Re: Jobs Running with Single Mapper/Reducer
Posted by 周千昊 <qh...@apache.org>.
Hi Mishra
It is as what you described, since the data size is 30MB only, hadoop
will only manage to run MR in single mapper and reduce.
As for your question about why it takes so long to run on a small
dataset, can you please dig into the web page of *map reduce task status *to
check how much time the MR job really take, so that we can make sure if the
time is consuming on the MR job or kylin job scheduling module
Vineet Mishra <cl...@gmail.com>于2015年6月15日周一 下午4:51写道：
> Shi,
>
> Hadoop is setup correctly on my cluster with the default block size of
> 128Mb and its indeed very much running multiple Mapper/Reducer based jobs
> for other cases.
>
> Its the only Kylin Cube building which is running through single M/R job.
>
> Moreover to my surprise, the 4th running job which is Build Base Cuboid
> Data shows Data size as 30Mb, is it the reason due to which single mapper
> is getting invoked, if that being the case then even to process such a
> small data set why is it taking around 50min.
>
> Thanks,
>
> On Mon, Jun 15, 2015 at 11:17 AM, Shi, Shaofeng <sh...@ebay.com> wrote:
>
> > Yes you can adjust these parameters, for example give a smaller value for
> > kylin.job.mapreduce.default.reduce.input.mb; but it only affects the
> > reducer number;
> >
> > I suggest you investigate why there is only 1 mapper be started; Some
> > factors like hadoop cluster size, HDFS file block size will impact this;
> > You can run a SQL (a query which need run MR, not simple select *) with
> > hive -e, and then use the MR job track URL to see how many mappers be
> > triggered; If it is still single, then the problem is in your hadoop
> > configuration; Otherwise it may exists in Kylin, check if you put some
> > additional parameter in conf/kylin_job_conf.xml.
> >
> >
> > On 6/15/15, 2:52 AM, "Vineet Mishra" <cl...@gmail.com> wrote:
> >
> > >Can I have specification for these properties?
> > >
> > >KYLIN_JOB_MAPREDUCE_DEFAULT_REDUCE_COUNT_RATIO =
> > >"kylin.job.mapreduce.default.reduce.count.ratio";
> > >KYLIN_JOB_MAPREDUCE_DEFAULT_REDUCE_INPUT_MB =
> > >"kylin.job.mapreduce.default.reduce.input.mb";
> > >KYLIN_JOB_MAPREDUCE_MAX_REDUCER_NUMBER =
> > >"kylin.job.mapreduce.max.reducer.number";
> > >
> > >Thanks!
> > >
> > >On Sun, Jun 14, 2015 at 11:59 PM, Vineet Mishra <clearmidoubt@gmail.com
> >
> > >wrote:
> > >
> > >> Hi Shi,
> > >>
> > >> Its alright!
> > >> So I was wondering my source hive Table is around 3 Gb, despite of my
> > >>hive
> > >> table being partitioned and holding the data around 50-70 Mb per
> > >>partition
> > >> the Mapper and Reducer getting spawned are single. The amount of data
> > >>that
> > >> is being processed in the M/R is nothing as expected but it takes hell
> > >>lot
> > >> of time.
> > >>
> > >> As mentioned in the trailing mail that the job is getting very slow,
> the
> > >> process Build Base Cuboid Data itself takes around 50mins to get
> > >> completed.
> > >>
> > >> I can tweak the reducer parameter mentioned by you but do u think that
> > >> will make a difference since the mapper is where the most of the time
> is
> > >> spent.
> > >>
> > >> Can you share your thoughts for performance tuning for the cube build!
> > >>
> > >> Thanks!
> > >>
> > >> On Sun, Jun 14, 2015 at 7:26 PM, Shi, Shaofeng <sh...@ebay.com>
> > wrote:
> > >>
> > >>> Hi, sorry, a busy weekend;
> > >>>
> > >>> Usually Kylin will request proper number of mapper and reducers; If
> you
> > >>> see single mapper/recudder, how much of your input and output? If
> your
> > >>> cube is quite small, single mapper/reducer is possible;
> > >>>
> > >>> Number of mappers is decided by the FileInputFormat; But number of
> > >>>reducer
> > >>> was set by Kylin, see:
> > >>>
> > >>>
> > >>>
> > https://github.com/apache/incubator-kylin/blob/master/job/src/main/java/
> > >>>org
> > >>> /apache/kylin/job/hadoop/cube/CuboidJob.java#L141
> > >>>
> > >>><
> > https://github.com/apache/incubator-kylin/blob/master/job/src/main/java
> > >>>/org/apache/kylin/job/hadoop/cube/CuboidJob.java#L141>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> On 6/14/15, 5:25 PM, "Vineet Mishra" <cl...@gmail.com> wrote:
> > >>>
> > >>> >Urgent call, any follow up on this?
> > >>> >
> > >>> >On Fri, Jun 12, 2015 at 6:46 PM, Vineet Mishra
> > >>><cl...@gmail.com>
> > >>> >wrote:
> > >>> >
> > >>> >>
> > >>> >> Why org.apache.kylin.job.hadoop.cube.CuboidReducer is running
> Single
> > >>> >> Mapper/Reducer for the job. Can I have the understanding behind
> the
> > >>> >>reason
> > >>> >> of running it as single mapper/reducer.
> > >>> >>
> > >>> >> Thanks!
> > >>> >>
> > >>> >> On Fri, Jun 12, 2015 at 6:30 PM, Vineet Mishra
> > >>><clearmidoubt@gmail.com
> > >>> >
> > >>> >> wrote:
> > >>> >>
> > >>> >>> Hi All,
> > >>> >>>
> > >>> >>> I am building a cube using Kylin and I could see that the job is
> > >>> >>>running
> > >>> >>> with Single Mapper and Reducer for some of the intermediate
> process
> > >>> >>>such as
> > >>> >>>
> > >>> >>> Extract Fact Table Distinct Columns
> > >>> >>> Build Dimension Dictionary
> > >>> >>> Build N-Dimension Cuboid
> > >>> >>>
> > >>> >>> I am not sure what's the reason behind running the job with
> single
> > >>> M/R,
> > >>> >>> is it really necessary or is it some default config. which can be
> > >>> >>>tweaked,
> > >>> >>> its 70 Mins and the job status is 25% !
> > >>> >>>
> > >>> >>> Urgent Call!
> > >>> >>>
> > >>> >>> Thanks!
> > >>> >>>
> > >>> >>
> > >>> >>
> > >>>
> > >>>
> > >>
> >
> >
>
Re: Jobs Running with Single Mapper/Reducer
Posted by Vineet Mishra <cl...@gmail.com>.
Shi,
Hadoop is setup correctly on my cluster with the default block size of
128Mb and its indeed very much running multiple Mapper/Reducer based jobs
for other cases.
Its the only Kylin Cube building which is running through single M/R job.
Moreover to my surprise, the 4th running job which is Build Base Cuboid
Data shows Data size as 30Mb, is it the reason due to which single mapper
is getting invoked, if that being the case then even to process such a
small data set why is it taking around 50min.
Thanks,
On Mon, Jun 15, 2015 at 11:17 AM, Shi, Shaofeng <sh...@ebay.com> wrote:
> Yes you can adjust these parameters, for example give a smaller value for
> kylin.job.mapreduce.default.reduce.input.mb; but it only affects the
> reducer number;
>
> I suggest you investigate why there is only 1 mapper be started; Some
> factors like hadoop cluster size, HDFS file block size will impact this;
> You can run a SQL (a query which need run MR, not simple select *) with
> hive -e, and then use the MR job track URL to see how many mappers be
> triggered; If it is still single, then the problem is in your hadoop
> configuration; Otherwise it may exists in Kylin, check if you put some
> additional parameter in conf/kylin_job_conf.xml.
>
>
> On 6/15/15, 2:52 AM, "Vineet Mishra" <cl...@gmail.com> wrote:
>
> >Can I have specification for these properties?
> >
> >KYLIN_JOB_MAPREDUCE_DEFAULT_REDUCE_COUNT_RATIO =
> >"kylin.job.mapreduce.default.reduce.count.ratio";
> >KYLIN_JOB_MAPREDUCE_DEFAULT_REDUCE_INPUT_MB =
> >"kylin.job.mapreduce.default.reduce.input.mb";
> >KYLIN_JOB_MAPREDUCE_MAX_REDUCER_NUMBER =
> >"kylin.job.mapreduce.max.reducer.number";
> >
> >Thanks!
> >
> >On Sun, Jun 14, 2015 at 11:59 PM, Vineet Mishra <cl...@gmail.com>
> >wrote:
> >
> >> Hi Shi,
> >>
> >> Its alright!
> >> So I was wondering my source hive Table is around 3 Gb, despite of my
> >>hive
> >> table being partitioned and holding the data around 50-70 Mb per
> >>partition
> >> the Mapper and Reducer getting spawned are single. The amount of data
> >>that
> >> is being processed in the M/R is nothing as expected but it takes hell
> >>lot
> >> of time.
> >>
> >> As mentioned in the trailing mail that the job is getting very slow, the
> >> process Build Base Cuboid Data itself takes around 50mins to get
> >> completed.
> >>
> >> I can tweak the reducer parameter mentioned by you but do u think that
> >> will make a difference since the mapper is where the most of the time is
> >> spent.
> >>
> >> Can you share your thoughts for performance tuning for the cube build!
> >>
> >> Thanks!
> >>
> >> On Sun, Jun 14, 2015 at 7:26 PM, Shi, Shaofeng <sh...@ebay.com>
> wrote:
> >>
> >>> Hi, sorry, a busy weekend;
> >>>
> >>> Usually Kylin will request proper number of mapper and reducers; If you
> >>> see single mapper/recudder, how much of your input and output? If your
> >>> cube is quite small, single mapper/reducer is possible;
> >>>
> >>> Number of mappers is decided by the FileInputFormat; But number of
> >>>reducer
> >>> was set by Kylin, see:
> >>>
> >>>
> >>>
> https://github.com/apache/incubator-kylin/blob/master/job/src/main/java/
> >>>org
> >>> /apache/kylin/job/hadoop/cube/CuboidJob.java#L141
> >>>
> >>><
> https://github.com/apache/incubator-kylin/blob/master/job/src/main/java
> >>>/org/apache/kylin/job/hadoop/cube/CuboidJob.java#L141>
> >>>
> >>>
> >>>
> >>>
> >>> On 6/14/15, 5:25 PM, "Vineet Mishra" <cl...@gmail.com> wrote:
> >>>
> >>> >Urgent call, any follow up on this?
> >>> >
> >>> >On Fri, Jun 12, 2015 at 6:46 PM, Vineet Mishra
> >>><cl...@gmail.com>
> >>> >wrote:
> >>> >
> >>> >>
> >>> >> Why org.apache.kylin.job.hadoop.cube.CuboidReducer is running Single
> >>> >> Mapper/Reducer for the job. Can I have the understanding behind the
> >>> >>reason
> >>> >> of running it as single mapper/reducer.
> >>> >>
> >>> >> Thanks!
> >>> >>
> >>> >> On Fri, Jun 12, 2015 at 6:30 PM, Vineet Mishra
> >>><clearmidoubt@gmail.com
> >>> >
> >>> >> wrote:
> >>> >>
> >>> >>> Hi All,
> >>> >>>
> >>> >>> I am building a cube using Kylin and I could see that the job is
> >>> >>>running
> >>> >>> with Single Mapper and Reducer for some of the intermediate process
> >>> >>>such as
> >>> >>>
> >>> >>> Extract Fact Table Distinct Columns
> >>> >>> Build Dimension Dictionary
> >>> >>> Build N-Dimension Cuboid
> >>> >>>
> >>> >>> I am not sure what's the reason behind running the job with single
> >>> M/R,
> >>> >>> is it really necessary or is it some default config. which can be
> >>> >>>tweaked,
> >>> >>> its 70 Mins and the job status is 25% !
> >>> >>>
> >>> >>> Urgent Call!
> >>> >>>
> >>> >>> Thanks!
> >>> >>>
> >>> >>
> >>> >>
> >>>
> >>>
> >>
>
>
Re: Jobs Running with Single Mapper/Reducer
Posted by "Shi, Shaofeng" <sh...@ebay.com>.
Yes you can adjust these parameters, for example give a smaller value for
kylin.job.mapreduce.default.reduce.input.mb; but it only affects the
reducer number;
I suggest you investigate why there is only 1 mapper be started; Some
factors like hadoop cluster size, HDFS file block size will impact this;
You can run a SQL (a query which need run MR, not simple select *) with
hive -e, and then use the MR job track URL to see how many mappers be
triggered; If it is still single, then the problem is in your hadoop
configuration; Otherwise it may exists in Kylin, check if you put some
additional parameter in conf/kylin_job_conf.xml.
On 6/15/15, 2:52 AM, "Vineet Mishra" <cl...@gmail.com> wrote:
>Can I have specification for these properties?
>
>KYLIN_JOB_MAPREDUCE_DEFAULT_REDUCE_COUNT_RATIO =
>"kylin.job.mapreduce.default.reduce.count.ratio";
>KYLIN_JOB_MAPREDUCE_DEFAULT_REDUCE_INPUT_MB =
>"kylin.job.mapreduce.default.reduce.input.mb";
>KYLIN_JOB_MAPREDUCE_MAX_REDUCER_NUMBER =
>"kylin.job.mapreduce.max.reducer.number";
>
>Thanks!
>
>On Sun, Jun 14, 2015 at 11:59 PM, Vineet Mishra <cl...@gmail.com>
>wrote:
>
>> Hi Shi,
>>
>> Its alright!
>> So I was wondering my source hive Table is around 3 Gb, despite of my
>>hive
>> table being partitioned and holding the data around 50-70 Mb per
>>partition
>> the Mapper and Reducer getting spawned are single. The amount of data
>>that
>> is being processed in the M/R is nothing as expected but it takes hell
>>lot
>> of time.
>>
>> As mentioned in the trailing mail that the job is getting very slow, the
>> process Build Base Cuboid Data itself takes around 50mins to get
>> completed.
>>
>> I can tweak the reducer parameter mentioned by you but do u think that
>> will make a difference since the mapper is where the most of the time is
>> spent.
>>
>> Can you share your thoughts for performance tuning for the cube build!
>>
>> Thanks!
>>
>> On Sun, Jun 14, 2015 at 7:26 PM, Shi, Shaofeng <sh...@ebay.com> wrote:
>>
>>> Hi, sorry, a busy weekend;
>>>
>>> Usually Kylin will request proper number of mapper and reducers; If you
>>> see single mapper/recudder, how much of your input and output? If your
>>> cube is quite small, single mapper/reducer is possible;
>>>
>>> Number of mappers is decided by the FileInputFormat; But number of
>>>reducer
>>> was set by Kylin, see:
>>>
>>>
>>>https://github.com/apache/incubator-kylin/blob/master/job/src/main/java/
>>>org
>>> /apache/kylin/job/hadoop/cube/CuboidJob.java#L141
>>>
>>><https://github.com/apache/incubator-kylin/blob/master/job/src/main/java
>>>/org/apache/kylin/job/hadoop/cube/CuboidJob.java#L141>
>>>
>>>
>>>
>>>
>>> On 6/14/15, 5:25 PM, "Vineet Mishra" <cl...@gmail.com> wrote:
>>>
>>> >Urgent call, any follow up on this?
>>> >
>>> >On Fri, Jun 12, 2015 at 6:46 PM, Vineet Mishra
>>><cl...@gmail.com>
>>> >wrote:
>>> >
>>> >>
>>> >> Why org.apache.kylin.job.hadoop.cube.CuboidReducer is running Single
>>> >> Mapper/Reducer for the job. Can I have the understanding behind the
>>> >>reason
>>> >> of running it as single mapper/reducer.
>>> >>
>>> >> Thanks!
>>> >>
>>> >> On Fri, Jun 12, 2015 at 6:30 PM, Vineet Mishra
>>><clearmidoubt@gmail.com
>>> >
>>> >> wrote:
>>> >>
>>> >>> Hi All,
>>> >>>
>>> >>> I am building a cube using Kylin and I could see that the job is
>>> >>>running
>>> >>> with Single Mapper and Reducer for some of the intermediate process
>>> >>>such as
>>> >>>
>>> >>> Extract Fact Table Distinct Columns
>>> >>> Build Dimension Dictionary
>>> >>> Build N-Dimension Cuboid
>>> >>>
>>> >>> I am not sure what's the reason behind running the job with single
>>> M/R,
>>> >>> is it really necessary or is it some default config. which can be
>>> >>>tweaked,
>>> >>> its 70 Mins and the job status is 25% !
>>> >>>
>>> >>> Urgent Call!
>>> >>>
>>> >>> Thanks!
>>> >>>
>>> >>
>>> >>
>>>
>>>
>>
Re: Jobs Running with Single Mapper/Reducer
Posted by Vineet Mishra <cl...@gmail.com>.
Can I have specification for these properties?
KYLIN_JOB_MAPREDUCE_DEFAULT_REDUCE_COUNT_RATIO =
"kylin.job.mapreduce.default.reduce.count.ratio";
KYLIN_JOB_MAPREDUCE_DEFAULT_REDUCE_INPUT_MB =
"kylin.job.mapreduce.default.reduce.input.mb";
KYLIN_JOB_MAPREDUCE_MAX_REDUCER_NUMBER =
"kylin.job.mapreduce.max.reducer.number";
Thanks!
On Sun, Jun 14, 2015 at 11:59 PM, Vineet Mishra <cl...@gmail.com>
wrote:
> Hi Shi,
>
> Its alright!
> So I was wondering my source hive Table is around 3 Gb, despite of my hive
> table being partitioned and holding the data around 50-70 Mb per partition
> the Mapper and Reducer getting spawned are single. The amount of data that
> is being processed in the M/R is nothing as expected but it takes hell lot
> of time.
>
> As mentioned in the trailing mail that the job is getting very slow, the
> process Build Base Cuboid Data itself takes around 50mins to get
> completed.
>
> I can tweak the reducer parameter mentioned by you but do u think that
> will make a difference since the mapper is where the most of the time is
> spent.
>
> Can you share your thoughts for performance tuning for the cube build!
>
> Thanks!
>
> On Sun, Jun 14, 2015 at 7:26 PM, Shi, Shaofeng <sh...@ebay.com> wrote:
>
>> Hi, sorry, a busy weekend;
>>
>> Usually Kylin will request proper number of mapper and reducers; If you
>> see single mapper/recudder, how much of your input and output? If your
>> cube is quite small, single mapper/reducer is possible;
>>
>> Number of mappers is decided by the FileInputFormat; But number of reducer
>> was set by Kylin, see:
>>
>> https://github.com/apache/incubator-kylin/blob/master/job/src/main/java/org
>> /apache/kylin/job/hadoop/cube/CuboidJob.java#L141
>> <https://github.com/apache/incubator-kylin/blob/master/job/src/main/java/org/apache/kylin/job/hadoop/cube/CuboidJob.java#L141>
>>
>>
>>
>>
>> On 6/14/15, 5:25 PM, "Vineet Mishra" <cl...@gmail.com> wrote:
>>
>> >Urgent call, any follow up on this?
>> >
>> >On Fri, Jun 12, 2015 at 6:46 PM, Vineet Mishra <cl...@gmail.com>
>> >wrote:
>> >
>> >>
>> >> Why org.apache.kylin.job.hadoop.cube.CuboidReducer is running Single
>> >> Mapper/Reducer for the job. Can I have the understanding behind the
>> >>reason
>> >> of running it as single mapper/reducer.
>> >>
>> >> Thanks!
>> >>
>> >> On Fri, Jun 12, 2015 at 6:30 PM, Vineet Mishra <clearmidoubt@gmail.com
>> >
>> >> wrote:
>> >>
>> >>> Hi All,
>> >>>
>> >>> I am building a cube using Kylin and I could see that the job is
>> >>>running
>> >>> with Single Mapper and Reducer for some of the intermediate process
>> >>>such as
>> >>>
>> >>> Extract Fact Table Distinct Columns
>> >>> Build Dimension Dictionary
>> >>> Build N-Dimension Cuboid
>> >>>
>> >>> I am not sure what's the reason behind running the job with single
>> M/R,
>> >>> is it really necessary or is it some default config. which can be
>> >>>tweaked,
>> >>> its 70 Mins and the job status is 25% !
>> >>>
>> >>> Urgent Call!
>> >>>
>> >>> Thanks!
>> >>>
>> >>
>> >>
>>
>>
>
Re: Jobs Running with Single Mapper/Reducer
Posted by Vineet Mishra <cl...@gmail.com>.
Hi Shi,
Its alright!
So I was wondering my source hive Table is around 3 Gb, despite of my hive
table being partitioned and holding the data around 50-70 Mb per partition
the Mapper and Reducer getting spawned are single. The amount of data that
is being processed in the M/R is nothing as expected but it takes hell lot
of time.
As mentioned in the trailing mail that the job is getting very slow, the
process Build Base Cuboid Data itself takes around 50mins to get completed.
I can tweak the reducer parameter mentioned by you but do u think that will
make a difference since the mapper is where the most of the time is spent.
Can you share your thoughts for performance tuning for the cube build!
Thanks!
On Sun, Jun 14, 2015 at 7:26 PM, Shi, Shaofeng <sh...@ebay.com> wrote:
> Hi, sorry, a busy weekend;
>
> Usually Kylin will request proper number of mapper and reducers; If you
> see single mapper/recudder, how much of your input and output? If your
> cube is quite small, single mapper/reducer is possible;
>
> Number of mappers is decided by the FileInputFormat; But number of reducer
> was set by Kylin, see:
> https://github.com/apache/incubator-kylin/blob/master/job/src/main/java/org
> /apache/kylin/job/hadoop/cube/CuboidJob.java#L141
>
>
>
>
> On 6/14/15, 5:25 PM, "Vineet Mishra" <cl...@gmail.com> wrote:
>
> >Urgent call, any follow up on this?
> >
> >On Fri, Jun 12, 2015 at 6:46 PM, Vineet Mishra <cl...@gmail.com>
> >wrote:
> >
> >>
> >> Why org.apache.kylin.job.hadoop.cube.CuboidReducer is running Single
> >> Mapper/Reducer for the job. Can I have the understanding behind the
> >>reason
> >> of running it as single mapper/reducer.
> >>
> >> Thanks!
> >>
> >> On Fri, Jun 12, 2015 at 6:30 PM, Vineet Mishra <cl...@gmail.com>
> >> wrote:
> >>
> >>> Hi All,
> >>>
> >>> I am building a cube using Kylin and I could see that the job is
> >>>running
> >>> with Single Mapper and Reducer for some of the intermediate process
> >>>such as
> >>>
> >>> Extract Fact Table Distinct Columns
> >>> Build Dimension Dictionary
> >>> Build N-Dimension Cuboid
> >>>
> >>> I am not sure what's the reason behind running the job with single M/R,
> >>> is it really necessary or is it some default config. which can be
> >>>tweaked,
> >>> its 70 Mins and the job status is 25% !
> >>>
> >>> Urgent Call!
> >>>
> >>> Thanks!
> >>>
> >>
> >>
>
>
Re: Jobs Running with Single Mapper/Reducer
Posted by "Shi, Shaofeng" <sh...@ebay.com>.
Hi, sorry, a busy weekend;
Usually Kylin will request proper number of mapper and reducers; If you
see single mapper/recudder, how much of your input and output? If your
cube is quite small, single mapper/reducer is possible;
Number of mappers is decided by the FileInputFormat; But number of reducer
was set by Kylin, see:
https://github.com/apache/incubator-kylin/blob/master/job/src/main/java/org
/apache/kylin/job/hadoop/cube/CuboidJob.java#L141
On 6/14/15, 5:25 PM, "Vineet Mishra" <cl...@gmail.com> wrote:
>Urgent call, any follow up on this?
>
>On Fri, Jun 12, 2015 at 6:46 PM, Vineet Mishra <cl...@gmail.com>
>wrote:
>
>>
>> Why org.apache.kylin.job.hadoop.cube.CuboidReducer is running Single
>> Mapper/Reducer for the job. Can I have the understanding behind the
>>reason
>> of running it as single mapper/reducer.
>>
>> Thanks!
>>
>> On Fri, Jun 12, 2015 at 6:30 PM, Vineet Mishra <cl...@gmail.com>
>> wrote:
>>
>>> Hi All,
>>>
>>> I am building a cube using Kylin and I could see that the job is
>>>running
>>> with Single Mapper and Reducer for some of the intermediate process
>>>such as
>>>
>>> Extract Fact Table Distinct Columns
>>> Build Dimension Dictionary
>>> Build N-Dimension Cuboid
>>>
>>> I am not sure what's the reason behind running the job with single M/R,
>>> is it really necessary or is it some default config. which can be
>>>tweaked,
>>> its 70 Mins and the job status is 25% !
>>>
>>> Urgent Call!
>>>
>>> Thanks!
>>>
>>
>>
Re: Jobs Running with Single Mapper/Reducer
Posted by Vineet Mishra <cl...@gmail.com>.
Urgent call, any follow up on this?
On Fri, Jun 12, 2015 at 6:46 PM, Vineet Mishra <cl...@gmail.com>
wrote:
>
> Why org.apache.kylin.job.hadoop.cube.CuboidReducer is running Single
> Mapper/Reducer for the job. Can I have the understanding behind the reason
> of running it as single mapper/reducer.
>
> Thanks!
>
> On Fri, Jun 12, 2015 at 6:30 PM, Vineet Mishra <cl...@gmail.com>
> wrote:
>
>> Hi All,
>>
>> I am building a cube using Kylin and I could see that the job is running
>> with Single Mapper and Reducer for some of the intermediate process such as
>>
>> Extract Fact Table Distinct Columns
>> Build Dimension Dictionary
>> Build N-Dimension Cuboid
>>
>> I am not sure what's the reason behind running the job with single M/R,
>> is it really necessary or is it some default config. which can be tweaked,
>> its 70 Mins and the job status is 25% !
>>
>> Urgent Call!
>>
>> Thanks!
>>
>
>
Re: Jobs Running with Single Mapper/Reducer
Posted by Vineet Mishra <cl...@gmail.com>.
Why org.apache.kylin.job.hadoop.cube.CuboidReducer is running Single
Mapper/Reducer for the job. Can I have the understanding behind the reason
of running it as single mapper/reducer.
Thanks!
On Fri, Jun 12, 2015 at 6:30 PM, Vineet Mishra <cl...@gmail.com>
wrote:
> Hi All,
>
> I am building a cube using Kylin and I could see that the job is running
> with Single Mapper and Reducer for some of the intermediate process such as
>
> Extract Fact Table Distinct Columns
> Build Dimension Dictionary
> Build N-Dimension Cuboid
>
> I am not sure what's the reason behind running the job with single M/R, is
> it really necessary or is it some default config. which can be tweaked, its
> 70 Mins and the job status is 25% !
>
> Urgent Call!
>
> Thanks!
>