You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Brink <jq...@163.com> on 2009/10/28 05:58:52 UTC
How to store tree structure in Cassandra
Hi All,
For a DMS, I want to replace MySQL with Cassandra to store file/folder nodes. Current I use adjacency list model to stores nodes hierarchy. The shortage of the adjacency list model is the expensive traversal cost. While I want to navigate the entire workspace or folder in hierarchy, I have to execute multiple queries (for each sub folder). In addition, special care must be taken when deleting nodes because of the potential for orphaning an entire sub-tree in the process (delete a folder and all of its children may be orphaned). How to design to store files/folders'node as tree structure in Cassandra?
Thanks,
-Brink
Re:Re: How to store tree structure in Cassandra
Posted by Brink <jq...@163.com>.
I cannot get the tree structure storing detail design by clicking the following link. Could you show me by other ways? If storing the tree structure in Cassandra is not good solution, can we have a server and store it in the server's memory?
Regards,
-Brink
在2009-10-30，Vijay <vi...@gmail.com> 写道：
earlier, I was working on something similar to that... Some one has done some work based on Hbase....
try this: http://dev.day.com/discussion-groups/content/lists/jackrabbit-dev/2009-02/2009-02-19_Jackrabbit_on_Hadoop_Marcel_Reutegger.html
similar thing can be done on cassandra, if you looking at replacing the persistence manager....
For remove instead of making it orphan you might want to remove it, unless you want to write a map reduce job.
Regards,
</VJ>
On Wed, Oct 28, 2009 at 12:35 AM, Mark Robson <ma...@gmail.com> wrote:
2009/10/28 Brink <jq...@163.com>
Hi All,
For a DMS, I want to replace MySQL with Cassandra to store file/folder nodes. Current I use adjacency list model to stores nodes hierarchy. The shortage of the adjacency list model is the expensive traversal cost. While I want to navigate the entire workspace or folder in hierarchy, I have to execute multiple queries (for each sub folder). In addition, special care must be taken when deleting nodes because of the potential for orphaning an entire sub-tree in the process (delete a folder and all of its children may be orphaned). How to design to store files/folders'node as tree structure in Cassandra?
Can you not use the entire path as a key? That would enable a recursive directory scan to become a simple range scan (which is supported provided you use the OrderedPartitioner)
The downside is that if you want to scan the immediate children you get a lot of additional junk you have to discard.
Storing redundant information in Cassandra (or any such database) is not an unusual solution - if you want very high scalability as well as good performance in competing cases (e.g. store the same thing under several keys)
Mark
Re: How to store tree structure in Cassandra
Posted by Vijay <vi...@gmail.com>.
earlier, I was working on something similar to that... Some one has done
some work based on Hbase....
try this:
http://dev.day.com/discussion-groups/content/lists/jackrabbit-dev/2009-02/2009-02-19_Jackrabbit_on_Hadoop_Marcel_Reutegger.html
similar thing can be done on cassandra, if you looking at replacing the
persistence manager....
For remove instead of making it orphan you might want to remove it, unless
you want to write a map reduce job.
Regards,
</VJ>
On Wed, Oct 28, 2009 at 12:35 AM, Mark Robson <ma...@gmail.com> wrote:
>
>
> 2009/10/28 Brink <jq...@163.com>
>
> Hi All,
>>
>> For a DMS, I want to replace MySQL with Cassandra to store file/folder
>> nodes. Current I use adjacency list model to stores nodes hierarchy. The
>> shortage of the adjacency list model is the expensive traversal cost. While
>> I want to navigate the entire workspace or folder in hierarchy, I have to
>> execute multiple queries (for each sub folder). In addition, special care
>> must be taken when deleting nodes because of the potential for orphaning an
>> entire sub-tree in the process (delete a folder and all of its children may
>> be orphaned). How to design to store files/folders'node as tree structure in
>> Cassandra?
>>
>
> Can you not use the entire path as a key? That would enable a recursive
> directory scan to become a simple range scan (which is supported provided
> you use the OrderedPartitioner)
>
> The downside is that if you want to scan the immediate children you get a
> lot of additional junk you have to discard.
>
> Storing redundant information in Cassandra (or any such database) is not an
> unusual solution - if you want very high scalability as well as good
> performance in competing cases (e.g. store the same thing under several
> keys)
>
> Mark
>
Re: Any alternative to Thrift
Posted by Sameer Charles <sa...@towards.ch>.
Great! Yes I am trying to embed Cassandra, this application will be
distributed on each node in a cluster together with Cassandra.
I will do some tests on Thrift vs. StorageProxy, if there is not much
difference in performance then I might stick with Thrift.
Thanks for your help.
Regards,
Sameer Charles
ROME - It's just a start
http://romecms.com/about/
On Oct 28, 2009, at 6:26 PM, Jonathan Ellis wrote:
> If you're embedding it as part of a distributed cluster, use the
> static methods in StorageProxy. If the local jvm is the only node,
> you can skip the messaging layer and use Table objects directly (see
> TableTest.java for examples of using RowMutation / getColumnFamily).
>
> -Jonathan
>
> On Wed, Oct 28, 2009 at 11:22 AM, Sameer Charles
> <sa...@towards.ch> wrote:
>> Hi all,
>>
>> Is Thrift the only way to talk to Cassandra? what if I embed
>> Cassandra
>> within my app (single JVM), can I use some native
>> Cassandra API to communicate?
>>
>> Any help will be appreciated.
>>
>> Thanks,
>> Sameer Charles
>>
Re: Any alternative to Thrift
Posted by Jonathan Ellis <jb...@gmail.com>.
If you're embedding it as part of a distributed cluster, use the
static methods in StorageProxy. If the local jvm is the only node,
you can skip the messaging layer and use Table objects directly (see
TableTest.java for examples of using RowMutation / getColumnFamily).
-Jonathan
On Wed, Oct 28, 2009 at 11:22 AM, Sameer Charles
<sa...@towards.ch> wrote:
> Hi all,
>
> Is Thrift the only way to talk to Cassandra? what if I embed Cassandra
> within my app (single JVM), can I use some native
> Cassandra API to communicate?
>
> Any help will be appreciated.
>
> Thanks,
> Sameer Charles
>
Any alternative to Thrift
Posted by Sameer Charles <sa...@towards.ch>.
Hi all,
Is Thrift the only way to talk to Cassandra? what if I embed Cassandra
within my app (single JVM), can I use some native
Cassandra API to communicate?
Any help will be appreciated.
Thanks,
Sameer Charles
Re: How to store tree structure in Cassandra
Posted by Mark Robson <ma...@gmail.com>.
2009/10/28 Brink <jq...@163.com>
> Hi All,
>
> For a DMS, I want to replace MySQL with Cassandra to store file/folder
> nodes. Current I use adjacency list model to stores nodes hierarchy. The
> shortage of the adjacency list model is the expensive traversal cost. While
> I want to navigate the entire workspace or folder in hierarchy, I have to
> execute multiple queries (for each sub folder). In addition, special care
> must be taken when deleting nodes because of the potential for orphaning an
> entire sub-tree in the process (delete a folder and all of its children may
> be orphaned). How to design to store files/folders'node as tree structure in
> Cassandra?
>
Can you not use the entire path as a key? That would enable a recursive
directory scan to become a simple range scan (which is supported provided
you use the OrderedPartitioner)
The downside is that if you want to scan the immediate children you get a
lot of additional junk you have to discard.
Storing redundant information in Cassandra (or any such database) is not an
unusual solution - if you want very high scalability as well as good
performance in competing cases (e.g. store the same thing under several
keys)
Mark