You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by Abhilash L L <ab...@infoworks.io> on 2015/11/27 06:45:01 UTC
ObjectGraphWalker 1000 limit issue
Warn while query execution
[http-bio-7071-exec-4]:[2015-11-26
07:22:32,164][WARN][net.sf.ehcache.pool.sizeof.ObjectGraphWalker.checkMaxDepth(ObjectGraphWalker.java:209)]
- The configured limit of 1,000 object references was reached while
attempting to calculate the size of the object graph. Severe performance
degradation could occur if the sizing operation continues. This can be
avoided by setting the CacheManger or Cache <sizeOfPolicy> elements
maxDepthExceededBehavior to "abort" or adding stop points with
@IgnoreSizeOf annotations. If performance degradation is NOT an issue at
the configured limit, raise the limit value using the CacheManager or Cache
<sizeOfPolicy> elements maxDepth attribute. For more information, see the
Ehcache configuration documentation.
I am observing in the tomcat logs for certain queries which are expected to
return high number of records.. And then there is no response for the
query request
Whats the suggested corrective action ?
Regards,
Abhilash
Re: ObjectGraphWalker 1000 limit issue
Posted by Abhilash L L <ab...@infoworks.io>.
Thanks for the clarification
Regards,
Abhilash
On Fri, Nov 27, 2015 at 12:21 PM, Xiaoyu Wang <wa...@jd.com> wrote:
> I create a jira:https://issues.apache.org/jira/browse/KYLIN-1174
>
> If need , I change the code SQLResponse.java to ignore the WARN.
> @liyang
>
>
> 在 2015年11月27日 14:44, Li Yang 写道:
>
>> This is a known point of improvement. Kylin caches query result so that
>> the
>> same query if comes again can benefit. Currently ehcache is used, that's
>> why you see this warn, is because ehcache crawling a big object graph to
>> estimate its size. By improvement, I mean the cache could hold
>> serialization form of result set rather than its object form.
>>
>> Anyway, for now, just ignore the warn, it's no harm.
>>
>> On Fri, Nov 27, 2015 at 2:24 PM, Luke Han <lu...@gmail.com> wrote:
>>
>> Why not open an JIRA and submit patch for such idea;-)
>>>
>>> Thanks.
>>>
>>>
>>> Best Regards!
>>> ---------------------
>>>
>>> Luke Han
>>>
>>> On Fri, Nov 27, 2015 at 2:11 PM, Abhilash L L <ab...@infoworks.io>
>>> wrote:
>>>
>>> Shouldnt this be like a config for an end user to change ?
>>>>
>>>> 1000 default is really low i feel
>>>>
>>>>
>>>> am on 0.7.2
>>>>
>>>> Regards,
>>>> Abhilash
>>>>
>>>> On Fri, Nov 27, 2015 at 11:28 AM, Xiaoyu Wang <wa...@jd.com>
>>>>
>>> wrote:
>>>
>>>> In My env, I change the code in SQLResponse.java
>>>>>
>>>>> @IgnoreSizeOf private List<List<String>>results;||
>>>>>
>>>>> add the @IgnoreSizeOf annotations to resolve this problem.
>>>>>
>>>>> 在 2015年11月27日 13:45, Abhilash L L 写道:
>>>>>
>>>>> Warn while query execution
>>>>>>
>>>>>> [http-bio-7071-exec-4]:[2015-11-26
>>>>>>
>>>>>>
>>>>>>
>>> 07:22:32,164][WARN][net.sf.ehcache.pool.sizeof.ObjectGraphWalker.checkMaxDepth(ObjectGraphWalker.java:209)]
>>>
>>>> - The configured limit of 1,000 object references was reached while
>>>>>> attempting to calculate the size of the object graph. Severe
>>>>>>
>>>>> performance
>>>
>>>> degradation could occur if the sizing operation continues. This can be
>>>>>> avoided by setting the CacheManger or Cache <sizeOfPolicy> elements
>>>>>> maxDepthExceededBehavior to "abort" or adding stop points with
>>>>>> @IgnoreSizeOf annotations. If performance degradation is NOT an issue
>>>>>>
>>>>> at
>>>
>>>> the configured limit, raise the limit value using the CacheManager or
>>>>>> Cache
>>>>>> <sizeOfPolicy> elements maxDepth attribute. For more information, see
>>>>>>
>>>>> the
>>>>
>>>>> Ehcache configuration documentation.
>>>>>>
>>>>>> I am observing in the tomcat logs for certain queries which are
>>>>>>
>>>>> expected
>>>
>>>> to
>>>>>> return high number of records.. And then there is no response for
>>>>>>
>>>>> the
>>>
>>>> query request
>>>>>>
>>>>>> Whats the suggested corrective action ?
>>>>>>
>>>>>> Regards,
>>>>>> Abhilash
>>>>>>
>>>>>>
>>>>>>
>
Re: ObjectGraphWalker 1000 limit issue
Posted by Xiaoyu Wang <wa...@jd.com>.
I create a jira:https://issues.apache.org/jira/browse/KYLIN-1174
If need , I change the code SQLResponse.java to ignore the WARN.
@liyang
在 2015年11月27日 14:44, Li Yang 写道:
> This is a known point of improvement. Kylin caches query result so that the
> same query if comes again can benefit. Currently ehcache is used, that's
> why you see this warn, is because ehcache crawling a big object graph to
> estimate its size. By improvement, I mean the cache could hold
> serialization form of result set rather than its object form.
>
> Anyway, for now, just ignore the warn, it's no harm.
>
> On Fri, Nov 27, 2015 at 2:24 PM, Luke Han <lu...@gmail.com> wrote:
>
>> Why not open an JIRA and submit patch for such idea;-)
>>
>> Thanks.
>>
>>
>> Best Regards!
>> ---------------------
>>
>> Luke Han
>>
>> On Fri, Nov 27, 2015 at 2:11 PM, Abhilash L L <ab...@infoworks.io>
>> wrote:
>>
>>> Shouldnt this be like a config for an end user to change ?
>>>
>>> 1000 default is really low i feel
>>>
>>>
>>> am on 0.7.2
>>>
>>> Regards,
>>> Abhilash
>>>
>>> On Fri, Nov 27, 2015 at 11:28 AM, Xiaoyu Wang <wa...@jd.com>
>> wrote:
>>>> In My env, I change the code in SQLResponse.java
>>>>
>>>> @IgnoreSizeOf private List<List<String>>results;||
>>>>
>>>> add the @IgnoreSizeOf annotations to resolve this problem.
>>>>
>>>> 在 2015年11月27日 13:45, Abhilash L L 写道:
>>>>
>>>>> Warn while query execution
>>>>>
>>>>> [http-bio-7071-exec-4]:[2015-11-26
>>>>>
>>>>>
>> 07:22:32,164][WARN][net.sf.ehcache.pool.sizeof.ObjectGraphWalker.checkMaxDepth(ObjectGraphWalker.java:209)]
>>>>> - The configured limit of 1,000 object references was reached while
>>>>> attempting to calculate the size of the object graph. Severe
>> performance
>>>>> degradation could occur if the sizing operation continues. This can be
>>>>> avoided by setting the CacheManger or Cache <sizeOfPolicy> elements
>>>>> maxDepthExceededBehavior to "abort" or adding stop points with
>>>>> @IgnoreSizeOf annotations. If performance degradation is NOT an issue
>> at
>>>>> the configured limit, raise the limit value using the CacheManager or
>>>>> Cache
>>>>> <sizeOfPolicy> elements maxDepth attribute. For more information, see
>>> the
>>>>> Ehcache configuration documentation.
>>>>>
>>>>> I am observing in the tomcat logs for certain queries which are
>> expected
>>>>> to
>>>>> return high number of records.. And then there is no response for
>> the
>>>>> query request
>>>>>
>>>>> Whats the suggested corrective action ?
>>>>>
>>>>> Regards,
>>>>> Abhilash
>>>>>
>>>>>
Re: ObjectGraphWalker 1000 limit issue
Posted by Li Yang <li...@apache.org>.
This is a known point of improvement. Kylin caches query result so that the
same query if comes again can benefit. Currently ehcache is used, that's
why you see this warn, is because ehcache crawling a big object graph to
estimate its size. By improvement, I mean the cache could hold
serialization form of result set rather than its object form.
Anyway, for now, just ignore the warn, it's no harm.
On Fri, Nov 27, 2015 at 2:24 PM, Luke Han <lu...@gmail.com> wrote:
> Why not open an JIRA and submit patch for such idea;-)
>
> Thanks.
>
>
> Best Regards!
> ---------------------
>
> Luke Han
>
> On Fri, Nov 27, 2015 at 2:11 PM, Abhilash L L <ab...@infoworks.io>
> wrote:
>
> > Shouldnt this be like a config for an end user to change ?
> >
> > 1000 default is really low i feel
> >
> >
> > am on 0.7.2
> >
> > Regards,
> > Abhilash
> >
> > On Fri, Nov 27, 2015 at 11:28 AM, Xiaoyu Wang <wa...@jd.com>
> wrote:
> >
> > > In My env, I change the code in SQLResponse.java
> > >
> > > @IgnoreSizeOf private List<List<String>>results;||
> > >
> > > add the @IgnoreSizeOf annotations to resolve this problem.
> > >
> > > 在 2015年11月27日 13:45, Abhilash L L 写道:
> > >
> > >> Warn while query execution
> > >>
> > >> [http-bio-7071-exec-4]:[2015-11-26
> > >>
> > >>
> >
> 07:22:32,164][WARN][net.sf.ehcache.pool.sizeof.ObjectGraphWalker.checkMaxDepth(ObjectGraphWalker.java:209)]
> > >> - The configured limit of 1,000 object references was reached while
> > >> attempting to calculate the size of the object graph. Severe
> performance
> > >> degradation could occur if the sizing operation continues. This can be
> > >> avoided by setting the CacheManger or Cache <sizeOfPolicy> elements
> > >> maxDepthExceededBehavior to "abort" or adding stop points with
> > >> @IgnoreSizeOf annotations. If performance degradation is NOT an issue
> at
> > >> the configured limit, raise the limit value using the CacheManager or
> > >> Cache
> > >> <sizeOfPolicy> elements maxDepth attribute. For more information, see
> > the
> > >> Ehcache configuration documentation.
> > >>
> > >> I am observing in the tomcat logs for certain queries which are
> expected
> > >> to
> > >> return high number of records.. And then there is no response for
> the
> > >> query request
> > >>
> > >> Whats the suggested corrective action ?
> > >>
> > >> Regards,
> > >> Abhilash
> > >>
> > >>
> >
>
Re: ObjectGraphWalker 1000 limit issue
Posted by Luke Han <lu...@gmail.com>.
Why not open an JIRA and submit patch for such idea;-)
Thanks.
Best Regards!
---------------------
Luke Han
On Fri, Nov 27, 2015 at 2:11 PM, Abhilash L L <ab...@infoworks.io> wrote:
> Shouldnt this be like a config for an end user to change ?
>
> 1000 default is really low i feel
>
>
> am on 0.7.2
>
> Regards,
> Abhilash
>
> On Fri, Nov 27, 2015 at 11:28 AM, Xiaoyu Wang <wa...@jd.com> wrote:
>
> > In My env, I change the code in SQLResponse.java
> >
> > @IgnoreSizeOf private List<List<String>>results;||
> >
> > add the @IgnoreSizeOf annotations to resolve this problem.
> >
> > 在 2015年11月27日 13:45, Abhilash L L 写道:
> >
> >> Warn while query execution
> >>
> >> [http-bio-7071-exec-4]:[2015-11-26
> >>
> >>
> 07:22:32,164][WARN][net.sf.ehcache.pool.sizeof.ObjectGraphWalker.checkMaxDepth(ObjectGraphWalker.java:209)]
> >> - The configured limit of 1,000 object references was reached while
> >> attempting to calculate the size of the object graph. Severe performance
> >> degradation could occur if the sizing operation continues. This can be
> >> avoided by setting the CacheManger or Cache <sizeOfPolicy> elements
> >> maxDepthExceededBehavior to "abort" or adding stop points with
> >> @IgnoreSizeOf annotations. If performance degradation is NOT an issue at
> >> the configured limit, raise the limit value using the CacheManager or
> >> Cache
> >> <sizeOfPolicy> elements maxDepth attribute. For more information, see
> the
> >> Ehcache configuration documentation.
> >>
> >> I am observing in the tomcat logs for certain queries which are expected
> >> to
> >> return high number of records.. And then there is no response for the
> >> query request
> >>
> >> Whats the suggested corrective action ?
> >>
> >> Regards,
> >> Abhilash
> >>
> >>
>
Re: ObjectGraphWalker 1000 limit issue
Posted by Xiaoyu Wang <wa...@jd.com>.
This is a WARN, It doesn't matter.
You can also change the
$KYLIN_HOME/tomcat/webapps/kylin/WEB-INF/classes/ehcache.xml add
<sizeOfPolicy maxDepth="*100*" maxDepthExceededBehavior="*continue*"/>
configuration maxDepth property you want.
see:
http://www.ehcache.org/ehcache.xml
在 2015年11月27日 14:11, Abhilash L L 写道:
> Shouldnt this be like a config for an end user to change ?
>
> 1000 default is really low i feel
>
>
> am on 0.7.2
>
> Regards,
> Abhilash
>
> On Fri, Nov 27, 2015 at 11:28 AM, Xiaoyu Wang <wa...@jd.com> wrote:
>
>> In My env, I change the code in SQLResponse.java
>>
>> @IgnoreSizeOf private List<List<String>>results;||
>>
>> add the @IgnoreSizeOf annotations to resolve this problem.
>>
>> 在 2015年11月27日 13:45, Abhilash L L 写道:
>>
>>> Warn while query execution
>>>
>>> [http-bio-7071-exec-4]:[2015-11-26
>>>
>>> 07:22:32,164][WARN][net.sf.ehcache.pool.sizeof.ObjectGraphWalker.checkMaxDepth(ObjectGraphWalker.java:209)]
>>> - The configured limit of 1,000 object references was reached while
>>> attempting to calculate the size of the object graph. Severe performance
>>> degradation could occur if the sizing operation continues. This can be
>>> avoided by setting the CacheManger or Cache <sizeOfPolicy> elements
>>> maxDepthExceededBehavior to "abort" or adding stop points with
>>> @IgnoreSizeOf annotations. If performance degradation is NOT an issue at
>>> the configured limit, raise the limit value using the CacheManager or
>>> Cache
>>> <sizeOfPolicy> elements maxDepth attribute. For more information, see the
>>> Ehcache configuration documentation.
>>>
>>> I am observing in the tomcat logs for certain queries which are expected
>>> to
>>> return high number of records.. And then there is no response for the
>>> query request
>>>
>>> Whats the suggested corrective action ?
>>>
>>> Regards,
>>> Abhilash
>>>
>>>
Re: ObjectGraphWalker 1000 limit issue
Posted by Abhilash L L <ab...@infoworks.io>.
Shouldnt this be like a config for an end user to change ?
1000 default is really low i feel
am on 0.7.2
Regards,
Abhilash
On Fri, Nov 27, 2015 at 11:28 AM, Xiaoyu Wang <wa...@jd.com> wrote:
> In My env, I change the code in SQLResponse.java
>
> @IgnoreSizeOf private List<List<String>>results;||
>
> add the @IgnoreSizeOf annotations to resolve this problem.
>
> 在 2015年11月27日 13:45, Abhilash L L 写道:
>
>> Warn while query execution
>>
>> [http-bio-7071-exec-4]:[2015-11-26
>>
>> 07:22:32,164][WARN][net.sf.ehcache.pool.sizeof.ObjectGraphWalker.checkMaxDepth(ObjectGraphWalker.java:209)]
>> - The configured limit of 1,000 object references was reached while
>> attempting to calculate the size of the object graph. Severe performance
>> degradation could occur if the sizing operation continues. This can be
>> avoided by setting the CacheManger or Cache <sizeOfPolicy> elements
>> maxDepthExceededBehavior to "abort" or adding stop points with
>> @IgnoreSizeOf annotations. If performance degradation is NOT an issue at
>> the configured limit, raise the limit value using the CacheManager or
>> Cache
>> <sizeOfPolicy> elements maxDepth attribute. For more information, see the
>> Ehcache configuration documentation.
>>
>> I am observing in the tomcat logs for certain queries which are expected
>> to
>> return high number of records.. And then there is no response for the
>> query request
>>
>> Whats the suggested corrective action ?
>>
>> Regards,
>> Abhilash
>>
>>
Re: ObjectGraphWalker 1000 limit issue
Posted by Xiaoyu Wang <wa...@jd.com>.
In My env, I change the code in SQLResponse.java
@IgnoreSizeOf private List<List<String>>results;||
add the @IgnoreSizeOf annotations to resolve this problem.
在 2015年11月27日 13:45, Abhilash L L 写道:
> Warn while query execution
>
> [http-bio-7071-exec-4]:[2015-11-26
> 07:22:32,164][WARN][net.sf.ehcache.pool.sizeof.ObjectGraphWalker.checkMaxDepth(ObjectGraphWalker.java:209)]
> - The configured limit of 1,000 object references was reached while
> attempting to calculate the size of the object graph. Severe performance
> degradation could occur if the sizing operation continues. This can be
> avoided by setting the CacheManger or Cache <sizeOfPolicy> elements
> maxDepthExceededBehavior to "abort" or adding stop points with
> @IgnoreSizeOf annotations. If performance degradation is NOT an issue at
> the configured limit, raise the limit value using the CacheManager or Cache
> <sizeOfPolicy> elements maxDepth attribute. For more information, see the
> Ehcache configuration documentation.
>
> I am observing in the tomcat logs for certain queries which are expected to
> return high number of records.. And then there is no response for the
> query request
>
> Whats the suggested corrective action ?
>
> Regards,
> Abhilash
>