You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by victory_zgh <vi...@163.com> on 2016/01/07 11:55:34 UTC
Add Tachyon bolt for persisting state
Hi，
Storm has implemented HDFS bolt to store state data，however，in particular，writing data to HDFS is slower than Tachyon.
Tachyon is a distributed memory file system(http://www.tachyon-project.org/), which performace is very good ,and now gets more and more attention.
Tachyon is memory-centric,and the speed is fast. If state data is stored by Tachyon, the performance of saving state may become better，at the same time ,the state query performance may become better too.
I think, The work of adding tachyon-bolt is very interesting, and in some use case it can improve performance a lot.
Thank you!
Best regards
2016-01-07
Guanghui.Zhu
Department of Computer Science and Technology
State Key Laboratory for Novel Software Technology
Nanjing University
Phone: +86 13770681551
Email: victory_zgh@163.com
ResearchLab Homepage: http://pasa-bigdata.nju.edu.cn
Re: Add Tachyon bolt for persisting state
Posted by Bobby Evans <ev...@yahoo-inc.com.INVALID>.
That is great idea. If you want to file a JIRA for it and put up a pull request we would be happy to review it and help bring it into Storm. But with that too you should be able to setup your jars to use the Hadoop FileSystem compatibility that is a part of Tachyon to do it with the current code.
- Bobby
On Thursday, January 7, 2016 7:37 AM, victory_zgh <vi...@163.com> wrote:
Hi，
Storm has implemented HDFS bolt to store state data，however，in particular，writing data to HDFS is slower than Tachyon.
Tachyon is a distributed memory file system(http://www.tachyon-project.org/), which performace is very good ,and now gets more and more attention.
Tachyon is memory-centric,and the speed is fast. If state data is stored by Tachyon, the performance of saving state may become better，at the same time ,the state query performance may become better too.
I think, The work of adding tachyon-bolt is very interesting, and in some use case it can improve performance a lot.
Thank you!
Best regards
2016-01-07
Guanghui.Zhu
Department of Computer Science and Technology
State Key Laboratory for Novel Software Technology
Nanjing University
Phone: +86 13770681551
Email: victory_zgh@163.com
ResearchLab Homepage: http://pasa-bigdata.nju.edu.cn