You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by "Wang-0122 (via GitHub)" <gi...@apache.org> on 2025/01/13 10:35:14 UTC
[I] [Bug] 导出tsfile总是出现内存不足的问题 [iotdb]
Wang-0122 opened a new issue, #14691:
URL: https://github.com/apache/iotdb/issues/14691
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/iotdb/issues) and found nothing similar.
### Version
Cannot dump result because: 301: There is not enough memory to execute current fragment instance, current remaining free memory is 44350172B, estimated memory usage for current fragment instance is 848419096B
内存是足够的, 是不是要修改什么配置的, 2c8g的机器
### Describe the bug and provide the minimal reproduce step
1
### What did you expect to see?
1
### What did you see instead?
1
### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] 导出tsfile总是出现内存不足的问题 [iotdb]
Posted by "Wang-0122 (via GitHub)" <gi...@apache.org>.
Wang-0122 commented on issue #14691:
URL: https://github.com/apache/iotdb/issues/14691#issuecomment-2588982841
> 可以修改datanode-env.sh 的 memory-size 如果 是 windows 则是 datanode-env.bat
8G内存的机器, MEMORY_SIZE=6G 已经改到6G了, 还是不行
参考了这个链接做了些修改, 也无济于事
https://iotdb.apache.org/zh/UserGuide/latest/FAQ/Frequently-asked-questions.html#%E9%A2%84%E4%BC%B0%E5%86%85%E5%AD%98%E4%B8%8D%E8%B6%B3%E6%8A%A5%E9%94%99%E5%A6%82%E4%BD%95%E5%A4%84%E7%90%86
[root@ip-10-0-137-177 apache-iotdb-1.3.3-all-bin]# tools/export-tsfile.sh -h 127.0.0.1 -p 6667 -u root -pw root -t ./backup -q "select * from root.vGco.** where time >= 1736734200000 and time <= 1736734260000"
------------------------------------------
Starting IoTDB Client Export Script
------------------------------------------
Cannot dump result because: 301: There is not enough memory to execute current fragment instance, current remaining free memory is 308727591B, estimated memory usage for current fragment instance is 888118024B
只导出1分钟的数据也不行, 需要888118024B = 800M 内存
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] 导出tsfile总是出现内存不足的问题 [iotdb]
Posted by "CritasWang (via GitHub)" <gi...@apache.org>.
CritasWang commented on issue #14691:
URL: https://github.com/apache/iotdb/issues/14691#issuecomment-2586765699
可以修改datanode-env.sh 的 memory-size
如果 是 windows 则是 datanode-env.bat
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] 导出tsfile总是出现内存不足的问题 [iotdb]
Posted by "CritasWang (via GitHub)" <gi...@apache.org>.
CritasWang commented on issue #14691:
URL: https://github.com/apache/iotdb/issues/14691#issuecomment-2817498463
> > 可以修改datanode-env.sh 的 memory-size 如果 是 windows 则是 datanode-env.bat
>
> 8G内存的机器, MEMORY_SIZE=6G 已经改到6G了, 还是不行 参考了这个链接做了些修改, 也无济于事 https://iotdb.apache.org/zh/UserGuide/latest/FAQ/Frequently-asked-questions.html#%E9%A2%84%E4%BC%B0%E5%86%85%E5%AD%98%E4%B8%8D%E8%B6%B3%E6%8A%A5%E9%94%99%E5%A6%82%E4%BD%95%E5%A4%84%E7%90%86
>
> ## [root@ip-10-0-137-177 apache-[iotdb-1](https://issues.apache.org/jira/browse/IOTDB-1).3.3-all-bin]# tools/export-tsfile.sh -h 127.0.0.1 -p 6667 -u root -pw root -t ./backup -q "select * from root.vGco.** where time >= 1736734200000 and time <= 1736734260000"
> ## Starting IoTDB Client Export Script
> Cannot dump result because: 301: There is not enough memory to execute current fragment instance, current remaining free memory is 308727591B, estimated memory usage for current fragment instance is 888118024B
>
> 只导出1分钟的数据也不行, 需要888118024B = 800M 内存
这个操作中会将root.vGco.** 内所有的测点按时间对齐组成一个超级大宽表，因此预估的资源会比较多。
建议使用 export-tsfile.sh 时 select 语句中仅包含单个 device，不同的 device 分开来导出
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] 导出tsfile总是出现内存不足的问题 [iotdb]
Posted by "TheSeasonForHer (via GitHub)" <gi...@apache.org>.
TheSeasonForHer commented on issue #14691:
URL: https://github.com/apache/iotdb/issues/14691#issuecomment-2651193095
1、maybe you do not restart service after you edit the varible[参考1](https://www.timecho.com/docs/zh/UserGuide/latest/Deployment-and-Maintenance/Stand-Alone-Deployment_timecho.html#%E5%8F%82%E6%95%B0%E9%85%8D%E7%BD%AE)
2、you can consider that use ‘‘align by device adding” in the sql or slimit[参考2](https://www.timecho.com/docs/zh/UserGuide/latest/FAQ/Frequently-asked-questions.html#%E9%A2%84%E4%BC%B0%E5%86%85%E5%AD%98%E4%B8%8D%E8%B6%B3%E6%8A%A5%E9%94%99%E5%A6%82%E4%BD%95%E5%A4%84%E7%90%86)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] 导出tsfile总是出现内存不足的问题 [iotdb]
Posted by "Wang-0122 (via GitHub)" <gi...@apache.org>.
Wang-0122 commented on issue #14691:
URL: https://github.com/apache/iotdb/issues/14691#issuecomment-2586743989
版本: 1.3.3, 1个database, 大部分类型都有
集群环境为 3C3D
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org