You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Vijay (JIRA)" <ji...@apache.org> on 2011/02/15 20:15:58 UTC
[jira] Created: (CASSANDRA-2167) Add a deserialize(ByteBuffer)
method to ColumnFamilySerializer
Add a deserialize(ByteBuffer) method to ColumnFamilySerializer
--------------------------------------------------------------
Key: CASSANDRA-2167
URL: https://issues.apache.org/jira/browse/CASSANDRA-2167
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Vijay
Priority: Minor
by adding deserialize bytebuffer in the columnFamilySerializer we might be able to avoid copying of the bytes[] while deserializing the CF.
This can be done using....
ByteBuffer buff = ByteBuffer.wrap(bb.array(), bb.position(), bb.position()+ length);
This is an improvement on "https://issues.apache.org/jira/browse/CASSANDRA-1969" Plz see jonathan's comment.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2167) Add a deserialize(ByteBuffer)
method to ColumnFamilySerializer
Posted by "Vijay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vijay updated CASSANDRA-2167:
-----------------------------
Description:
by adding deserialize bytebuffer in the columnFamilySerializer we might be able to avoid copying of the bytes[] while deserializing the CF.
This can be done using....
ByteBuffer buff = ByteBuffer.wrap(bb.array(), bb.position(), length);
This is an improvement on "https://issues.apache.org/jira/browse/CASSANDRA-1969" Plz see jonathan's comment.
was:
by adding deserialize bytebuffer in the columnFamilySerializer we might be able to avoid copying of the bytes[] while deserializing the CF.
This can be done using....
ByteBuffer buff = ByteBuffer.wrap(bb.array(), bb.position(), bb.position()+ length);
This is an improvement on "https://issues.apache.org/jira/browse/CASSANDRA-1969" Plz see jonathan's comment.
> Add a deserialize(ByteBuffer) method to ColumnFamilySerializer
> --------------------------------------------------------------
>
> Key: CASSANDRA-2167
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2167
> Project: Cassandra
> Issue Type: Sub-task
> Components: Core
> Reporter: Vijay
> Priority: Minor
>
> by adding deserialize bytebuffer in the columnFamilySerializer we might be able to avoid copying of the bytes[] while deserializing the CF.
> This can be done using....
> ByteBuffer buff = ByteBuffer.wrap(bb.array(), bb.position(), length);
> This is an improvement on "https://issues.apache.org/jira/browse/CASSANDRA-1969" Plz see jonathan's comment.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2167) Add a deserialize(ByteBuffer)
method to ColumnFamilySerializer
Posted by "Vijay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13018537#comment-13018537 ]
Vijay commented on CASSANDRA-2167:
----------------------------------
Initial assumption was that, it will be handled with by the Memory.free.... seems like it is not.... so we should add some kind of locks before it frees up the memory? as a separate thread (mem.free)? or is it too much overhead? if we agree on lock... i can add some code to do so.
> Add a deserialize(ByteBuffer) method to ColumnFamilySerializer
> --------------------------------------------------------------
>
> Key: CASSANDRA-2167
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2167
> Project: Cassandra
> Issue Type: Sub-task
> Components: Core
> Affects Versions: 0.8
> Reporter: Vijay
> Assignee: Vijay
> Priority: Minor
> Fix For: 1.0
>
> Attachments: 0001-Reduce-Byte-Copy-Streams.txt, 0002-Reduce-Byte-Copy-BBUtil.txt, 0003-Reduce-BB-Copy.txt
>
>
> by adding deserialize bytebuffer in the columnFamilySerializer we might be able to avoid copying of the bytes[] while deserializing the CF.
> This can be done using....
> ByteBuffer buff = ByteBuffer.wrap(bb.array(), bb.position(), length);
> This is an improvement on "https://issues.apache.org/jira/browse/CASSANDRA-1969" Plz see jonathan's comment.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2167) Add a deserialize(ByteBuffer)
method to ColumnFamilySerializer
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13018453#comment-13018453 ]
Jonathan Ellis commented on CASSANDRA-2167:
-------------------------------------------
Revisiting this, I don't think it's actually a valid optimization -- if we free the backing Memory object before shipping the read buffers off over the network, we'll segfault. No?
> Add a deserialize(ByteBuffer) method to ColumnFamilySerializer
> --------------------------------------------------------------
>
> Key: CASSANDRA-2167
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2167
> Project: Cassandra
> Issue Type: Sub-task
> Components: Core
> Affects Versions: 0.8
> Reporter: Vijay
> Assignee: Vijay
> Priority: Minor
> Fix For: 0.8
>
> Attachments: 0001-Reduce-Byte-Copy-Streams.txt, 0002-Reduce-Byte-Copy-BBUtil.txt, 0003-Reduce-BB-Copy.txt
>
>
> by adding deserialize bytebuffer in the columnFamilySerializer we might be able to avoid copying of the bytes[] while deserializing the CF.
> This can be done using....
> ByteBuffer buff = ByteBuffer.wrap(bb.array(), bb.position(), length);
> This is an improvement on "https://issues.apache.org/jira/browse/CASSANDRA-1969" Plz see jonathan's comment.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2167) Add a deserialize(ByteBuffer)
method to ColumnFamilySerializer
Posted by "Vijay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vijay updated CASSANDRA-2167:
-----------------------------
Attachment: 0003-Reduce-BB-Copy.txt
0001-Reduce-Byte-Copy-Streams.txt
0002-Reduce-Byte-Copy-BBUtil.txt
0001-Reduce-Byte-Copy-Streams - Introduce New memory Input and Output stream which will insert the bytes directly into the Native memory and read from it to avoid copies of data.
0002-Reduce-Byte-Copy-BBUtil - Remove the Old bytebuffer Input and Output streams
0003-Reduce-BB-Copy - update the SerializedCache to use the new MIS and MOS
> Add a deserialize(ByteBuffer) method to ColumnFamilySerializer
> --------------------------------------------------------------
>
> Key: CASSANDRA-2167
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2167
> Project: Cassandra
> Issue Type: Sub-task
> Components: Core
> Reporter: Vijay
> Priority: Minor
> Attachments: 0001-Reduce-Byte-Copy-Streams.txt, 0002-Reduce-Byte-Copy-BBUtil.txt, 0003-Reduce-BB-Copy.txt
>
>
> by adding deserialize bytebuffer in the columnFamilySerializer we might be able to avoid copying of the bytes[] while deserializing the CF.
> This can be done using....
> ByteBuffer buff = ByteBuffer.wrap(bb.array(), bb.position(), length);
> This is an improvement on "https://issues.apache.org/jira/browse/CASSANDRA-1969" Plz see jonathan's comment.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2167) Add a deserialize(ByteBuffer)
method to ColumnFamilySerializer
Posted by "Vijay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12997916#comment-12997916 ]
Vijay commented on CASSANDRA-2167:
----------------------------------
Initial Tests show better performance with this approach... BB.array is not allowed for DBB and it wasn't clean and this approach seems to fit better with the existing approaches.
> Add a deserialize(ByteBuffer) method to ColumnFamilySerializer
> --------------------------------------------------------------
>
> Key: CASSANDRA-2167
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2167
> Project: Cassandra
> Issue Type: Sub-task
> Components: Core
> Reporter: Vijay
> Priority: Minor
> Attachments: 0001-Reduce-Byte-Copy-Streams.txt, 0002-Reduce-Byte-Copy-BBUtil.txt, 0003-Reduce-BB-Copy.txt
>
>
> by adding deserialize bytebuffer in the columnFamilySerializer we might be able to avoid copying of the bytes[] while deserializing the CF.
> This can be done using....
> ByteBuffer buff = ByteBuffer.wrap(bb.array(), bb.position(), length);
> This is an improvement on "https://issues.apache.org/jira/browse/CASSANDRA-1969" Plz see jonathan's comment.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira