You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by 沙漠火狐 <27...@qq.com> on 2016/11/02 06:52:49 UTC
The coprocessor thread stopped itself due to scan timeout or scan threshold(check region server log)
Hi
my Kylin version is 1.5.4.1 for CDH5.7/5.8
when I query in kylin, it return a error like
{cellset: null, rowTotalsLists: null, colTotalsLists: null, runtime: null,…}cellset:nullcolTotalsLists:nullerror:"IOException: org.apache.commons.httpclient.methods.PostMethod@4d54f318 failed, error code 500 and response: {"url":"http://zeus001.jp:7070/kylin/api/query","exception":"Error while executing SQL \"select \"DIM_DATE\".\"DATE_ID\" as \"c0\", \"V_DIM_CATE_LEVEL2\".\"CATE_LEVEL1_ID\" as \"c1\", \"V_DIM_BRAND\".\"BRAND_RANK\" as \"c2\", \"DIM_GOODS_TYPE\".\"GOODS_TYPE_ID\" as \"c3\", count(distinct \"FCT_ORDR_PATH_OLAP\".\"GU_ID\") as \"m0\", count(distinct \"FCT_ORDR_PATH_OLAP\".\"PAY_USER_ID\") as \"m1\" from \"FCT_ORDR_PATH_OLAP\" as \"FCT_ORDR_PATH_OLAP\" join \"DIM_DATE\" as \"DIM_DATE\" on \"FCT_ORDR_PATH_OLAP\".\"DATE_ID\" = \"DIM_DATE\".\"DATE_ID\" join \"V_DIM_CATE_LEVEL2\" as \"V_DIM_CATE_LEVEL2\" on \"FCT_ORDR_PATH_OLAP\".\"CATE_LEVEL2_ID\" = \"V_DIM_CATE_LEVEL2\".\"CATE_LEVEL2_ID\" join \"V_DIM_BRAND\" as \"V_DIM_BRAND\" on \"FCT_ORDR_PATH_OLAP\".\"BRAND_ID\" = \"V_DIM_BRAND\".\"BRAND_ID\" join \"DIM_GOODS_TYPE\" as \"DIM_GOODS_TYPE\" on \"FCT_ORDR_PATH_OLAP\".\"GOODS_TYPE_ID\" = \"DIM_GOODS_TYPE\".\"GOODS_TYPE_ID\" where \"DIM_DATE\".\"DATE_ID\" in ('2016-10-28', '2016-10-29', '2016-10-30', '2016-10-31') group by \"DIM_DATE\".\"DATE_ID\", \"V_DIM_CATE_LEVEL2\".\"CATE_LEVEL1_ID\", \"V_DIM_BRAND\".\"BRAND_RANK\", \"DIM_GOODS_TYPE\".\"GOODS_TYPE_ID\"\":
<sub-thread for GTScanRequest 1d11f21d> The coprocessor thread stopped itself due to scan timeout or scan threshold(check region server log), failing current query..."}"height:nullleftOffset:0query:nullrowTotalsLists:nullruntime:nulltopOffset:0width:null
and the HttpRequest only 11s, and I set the hbase.rpc.timeout = 180s it's the same !
I want Konw what the reason of the error ? thanks!
Re: The coprocessor thread stopped itself due to scan timeout or scan
threshold(check region server log)
Posted by ShaoFeng Shi <sh...@apache.org>.
Kylin will proctect the coprocessor from taking too much memory on HBase,
so it will estimate a threshold for each query; but the estimation might be
a little pessimistic (especially for 'count distinct'); You can set the
following parameters in conf/kylin.properties to a bigger value (this is
the default setting):
*kylin.query.coprocessor.mem.gb <http://kylin.query.coprocessor.mem.gb>=3.0*
*kylin.query.mem.budget=*3221225472
Besides, you may need check the log in region server, and considering to
optimize the query to reduce the scan numbers; In future release Kylin will
optimize this logic.
2016-11-02 14:52 GMT+08:00 沙漠火狐 <27...@qq.com>:
> Hi
> my Kylin version is 1.5.4.1 for CDH5.7/5.8
>
>
> when I query in kylin, it return a error like
>
>
> {cellset: null, rowTotalsLists: null, colTotalsLists: null, runtime:
> null,…}cellset:nullcolTotalsLists:nullerror:"IOException:
> org.apache.commons.httpclient.methods.PostMethod@4d54f318 failed, error
> code 500 and response: {"url":"http://zeus001.jp:
> 7070/kylin/api/query","exception":"Error while executing SQL \"select
> \"DIM_DATE\".\"DATE_ID\" as \"c0\", \"V_DIM_CATE_LEVEL2\".\"CATE_LEVEL1_ID\"
> as \"c1\", \"V_DIM_BRAND\".\"BRAND_RANK\" as \"c2\",
> \"DIM_GOODS_TYPE\".\"GOODS_TYPE_ID\" as \"c3\", count(distinct
> \"FCT_ORDR_PATH_OLAP\".\"GU_ID\") as \"m0\", count(distinct
> \"FCT_ORDR_PATH_OLAP\".\"PAY_USER_ID\") as \"m1\" from
> \"FCT_ORDR_PATH_OLAP\" as \"FCT_ORDR_PATH_OLAP\" join \"DIM_DATE\" as
> \"DIM_DATE\" on \"FCT_ORDR_PATH_OLAP\".\"DATE_ID\" =
> \"DIM_DATE\".\"DATE_ID\" join \"V_DIM_CATE_LEVEL2\" as
> \"V_DIM_CATE_LEVEL2\" on \"FCT_ORDR_PATH_OLAP\".\"CATE_LEVEL2_ID\"
> = \"V_DIM_CATE_LEVEL2\".\"CATE_LEVEL2_ID\" join \"V_DIM_BRAND\" as
> \"V_DIM_BRAND\" on \"FCT_ORDR_PATH_OLAP\".\"BRAND_ID\" =
> \"V_DIM_BRAND\".\"BRAND_ID\" join \"DIM_GOODS_TYPE\" as \"DIM_GOODS_TYPE\"
> on \"FCT_ORDR_PATH_OLAP\".\"GOODS_TYPE_ID\" = \"DIM_GOODS_TYPE\".\"GOODS_TYPE_ID\"
> where \"DIM_DATE\".\"DATE_ID\" in ('2016-10-28', '2016-10-29',
> '2016-10-30', '2016-10-31') group by \"DIM_DATE\".\"DATE_ID\",
> \"V_DIM_CATE_LEVEL2\".\"CATE_LEVEL1_ID\", \"V_DIM_BRAND\".\"BRAND_RANK\",
> \"DIM_GOODS_TYPE\".\"GOODS_TYPE_ID\"\":
> <sub-thread for GTScanRequest 1d11f21d> The coprocessor thread
> stopped itself due to scan timeout or scan threshold(check region server
> log), failing current query..."}"height:nullleftOffset:0query:
> nullrowTotalsLists:nullruntime:nulltopOffset:0width:null
> and the HttpRequest only 11s, and I set the hbase.rpc.timeout = 180s
> it's the same !
> I want Konw what the reason of the error ?
>
> thanks!
--
Best regards,
Shaofeng Shi 史少锋