You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by GitBox <gi...@apache.org> on 2021/02/05 09:38:41 UTC
[GitHub] [kylin] helenzeng0503 edited a comment on pull request #1569: KYLIN-4896 Optimize the process of writing big resource files into HDFS.
helenzeng0503 edited a comment on pull request #1569:
URL: https://github.com/apache/kylin/pull/1569#issuecomment-773915722
## Test Evidence
- First, we configure hbase.client.keyvalue.maxsize to 1KB.
<img width="347" alt="截屏2021-02-05 下午5 03 31" src="https://user-images.githubusercontent.com/66761090/107015227-7ac08900-67d7-11eb-82ad-c2ad7656a0ed.png">
- Then we restart kylin.
Check the setting of the hbase.client.keyvalue.maxsize and kvSizeLimit in log after kylin is up. We should see that "hbase.client.keyvalue.maxsize is 1024, kvSizeLimit is set to 10485760"
<img width="642" alt="截屏2021-02-05 下午5 05 12" src="https://user-images.githubusercontent.com/66761090/107015594-e0147a00-67d7-11eb-9b40-7ca3c0dbeed9.png">
- Then we build a cube with size larger than 1KB. We can see that kylin_sales_cube has size 6432>1024.
<img width="519" alt="截屏2021-02-05 下午5 33 24" src="https://user-images.githubusercontent.com/66761090/107015912-46010180-67d8-11eb-99f1-2f024c4881bb.png">
- Then we check HDFS and make sure that kylin_sales_cube.json is not written in HDFS.
<img width="827" alt="截屏2021-02-05 下午5 10 50" src="https://user-images.githubusercontent.com/66761090/107016183-9a0be600-67d8-11eb-904e-e5c83183e17f.png">
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org