You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@echarts.apache.org by GitBox <gi...@apache.org> on 2022/09/28 13:38:54 UTC
[GitHub] [echarts] MeetzhDing opened a new issue, #17711: [Bug] dataset DataStore['_initDataFromProvider'] cost too many memory when using encode
MeetzhDing opened a new issue, #17711:
URL: https://github.com/apache/echarts/issues/17711
### Version
5.4.0
### Link to Minimal Reproduction
https://codepen.io/meetzhding/pen/YzLeNbj
### Steps to Reproduce
store all data in dataset.source, then draw it in line chart using encode.
```js
var container = document.getElementById('main');
var chart = echarts.init(container);
function genTimeSeriesDataset() {
const dataset = {
source: {},
dimensions: [],
};
const timePointSize = 3 * 60 * 60
const lineCount = 100;
dataset.dimensions.push({
name: 'time1',
type: 'time',
displayName: 'time1',
});
dataset.source['time1'] = []
const timeEnd = +new Date().setMilliseconds(0);
const timeStart = timeEnd - timePointSize * 1000;
for(let i = 0; i<timePointSize; i++) {
dataset.source['time1'].push(timeStart + i * 1000)
}
for(let i=1;i<=lineCount;i++) {
const dimensionName = `number${i}`
dataset.dimensions.push({
name: dimensionName,
type: 'number',
displayName: dimensionName,
});
dataset.source[dimensionName] = []
for(let i = 0; i<timePointSize; i++) {
dataset.source[dimensionName].push(Math.floor(Math.random()*100))
}
}
return dataset
}
const dataset = genTimeSeriesDataset();
console.time('render');
chart.setOption({
xAxis: {
type: 'time',
},
yAxis: {
type: 'value'
},
dataset: dataset,
series: dataset.dimensions.slice(1).map(dimension => {
return {
type: 'line',
name: dimension.name,
animation: false,
showSymbol: false,
encode: {
x: 'time1',
y: dimension.name
},
stack: 'main',
lineStyle: {
width: 0.5
}
}
})
})
console.timeEnd('render');
```
### Current Behavior
<img width="2048" alt="image" src="https://user-images.githubusercontent.com/12808432/192789705-a0da3ff8-1ff6-4602-93d4-54058b009cf7.png">
Raw Data Size:
```js
const timePointSize = 3 * 60 * 60
const lineCount = 100
rawMem = timePointSize * (1 + lineCount) * 8Byte = 6240000Bytes ~= 6M
```
Page Memory Cost：= 800M+
---
In file `src/data/DataStore.ts`, _initDataFromProvider will copy the raw dataset.source, then the memory cost is equal to `lineCount * rawDatasetSize`
https://github.com/apache/echarts/blob/213f7ebd8b5f8c6dbb6ae1d11c3a7b97505dc798/src/data/DataStore.ts#L374-L433
### Expected Behavior
Reduce memory cost.
When using encode, it should copy the raw dataset mentioned in encode only, it will save lots of memory.
Or DataStore should use raw dataset field directly, then copy raw data in transform step only.
### Environment
```markdown
- OS: MacOS
- Browser: Chrome
- Framework: any
```
### Any additional comments?
this question may be related to https://github.com/apache/echarts/issues/11907
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@echarts.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@echarts.apache.org
For additional commands, e-mail: commits-help@echarts.apache.org
[GitHub] [echarts] troublezzb commented on issue #17711: [Bug] dataset DataStore['_initDataFromProvider'] cost too many memory when using encode
Posted by GitBox <gi...@apache.org>.
troublezzb commented on issue #17711:
URL: https://github.com/apache/echarts/issues/17711#issuecomment-1335230127
I have a same bug (may be).
1. Open [https://echarts.apache.org/examples/zh/editor.html?c=dynamic-data2](url) ，copy this code and run:
```javascript
const speedLength = 30;
let source = [];
//初始化数据
const timeStamp = new Date().getTime() - speedLength * 1000;
source = new Array(speedLength).fill(0).map((value, i) => {
return [timeStamp + i * 1000, 0, 0, 0, 0, 0, 0, 0, 0];
});
option = {
dataset: {
dimensions: ['Time', 'Total', 'Sim1', 'Sim2', 'Sim3', 'Sim4', 'Sim5', 'Sim6', 'Wan1'],
source: []
},
tooltip: { trigger: 'axis' },
xAxis: { type: 'time' },
yAxis: { type: 'value' },
//多个系列
series: new Array(8).fill(0).map((val, i) => {
return {
type: 'line',
encode: { x: 0, y: i + 1, itemId: 0, seriesName: i + 1 }
};
})
};
//定时器
let timeId = setInterval(() => {
// window.console.log(window.performance.memory)
const ulLength = source.length - speedLength;
if (ulLength >= 0) source.splice(0, ulLength + 1);
source = [...source, [new Date().getTime(), 0, 0, 0, 0, 0, 0, 0, 0]];
myChart.setOption({ dataset: { source: [...source] } });
}, 1000);
```
2. After 2 minutes, you will see "JS heap size (used)" is more and more higher (about 60kB/s).
> PS1: It is not this bug before v5.0.0 .
> PS2: It is not this bug when I use "series.data" instead of "dataset.source".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@echarts.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@echarts.apache.org
For additional commands, e-mail: commits-help@echarts.apache.org
[GitHub] [echarts] MeetzhDing commented on issue #17711: [Bug] dataset DataStore['_initDataFromProvider'] cost too many memory when using encode
Posted by GitBox <gi...@apache.org>.
MeetzhDing commented on issue #17711:
URL: https://github.com/apache/echarts/issues/17711#issuecomment-1288396052
@pissang @susiwen8 @100pah
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@echarts.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@echarts.apache.org
For additional commands, e-mail: commits-help@echarts.apache.org