You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Mike Smith (Created) (JIRA)" <ji...@apache.org> on 2011/11/02 17:15:32 UTC
[jira] [Created] (CASSANDRA-3446) Problem SliceByNamesReadCommand
on super column family after flush operation
Problem SliceByNamesReadCommand on super column family after flush operation
----------------------------------------------------------------------------
Key: CASSANDRA-3446
URL: https://issues.apache.org/jira/browse/CASSANDRA-3446
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.0.1, 1.0.0
Environment: Linux iam 3.0.0-12-generic #20-Ubuntu SMP Fri Oct 7 14:56:25 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
java version "1.6.0_23"
OpenJDK Runtime Environment (IcedTea6 1.11pre) (6b23~pre10-0ubuntu5)
OpenJDK 64-Bit Server VM (build 20.0-b11, mixed mode)
hector-core; 1.0-1
Reporter: Mike Smith
Fix For: 0.7.6
I'm having a problem with doing a multiget_slice on a super column family
after its first flush. Updates to the column values work properly, but
trying to retrieve the updated values using a multiget_slice operation fail
to get the updated values. Instead they return the values from before the
flush. The problem is not apparent with standard column families.
I've seen this problem in Cassandra v1.0.0 and v1.0.1. The problem
is not present in Cassandra v0.7.6.
Steps to reproduce:
1. Create one or more super column entries
2. Verify the sub column values can be updated and that you can retrieve
the new values
3. Use nodetool to flush the column family or restart cassandra
4. Update the sub column values
5. Verify they have been updated using cassandra-cli
6. Verify you *DO NOT* get the updated values when doing a
multiget_slice; instead you get the old values from before the flush
You can get the most recent value by doing a flush followed by a major
compaction. However, future updates are not retrieved properly either.
With debug turned on, it looks like the multiget_slice query uses the
following command/consistency level:
SliceByNamesReadCommand(table='test_cassandra', key=666f6f,
columnParent='QueryPath(columnFamilyName='test', superColumnName='null',
columnName='null')', columns=[foo,])/QUORUM.
Cassandra-cli uses the following command/consistency level for a get_slice:
SliceFromReadCommand(table='test_cassandra', key='666f6f',
column_parent='QueryPath(columnFamilyName='test', superColumnName='null',
columnName='null')', start='', finish='', reversed=false,
count=1000000)/QUORUM
Notice the test program gets 'bar2' for the column values and cassandra-cli
gets 'bar3' for the column values:
tcpdump from test program using hector-core:1.0-1
16:46:07.424562 IP iam.47158 > iam.9160: Flags [P.], seq 55:138, ack 30,
win 257, options [nop,nop,TS val 27474096 ecr 27474095], length 83
E....#@.@.PK.........6#.....].8......{.....
..8...8.........multiget_slice................foo..........test................foo.........
16:46:07.424575 IP iam.9160 > iam.47158: Flags [.], ack 138, win 256,
options [nop,nop,TS val 27474096 ecr 27474096], length 0
E..4..@.@.<.........#..6].8..........(.....
..8...8.
16:46:07.428771 IP iam.9160 > iam.47158: Flags [P.], seq 30:173, ack 138,
win 256, options [nop,nop,TS val 27474097 ecr 27474096], length 143
@.@.<&........#..6].8................
............foo...............foo...............foo1.......bar2
........6h........foo2.......bar2
........I.....
tcpdump of cassandra-cli:
16:30:55.945123 IP iam.47134 > iam.9160: Flags [P.], seq 370:479, ack 5310,
win 387, options [nop,nop,TS val 27246226 ecr 27241207], length 109
E.....@.@.9q..........#..n.X\
.............
................get_range_slices..............test.........................................................d.........
16:30:55.945152 IP iam.9160 > iam.47134: Flags [.], ack 479, win 256,
options [nop,nop,TS val 27246226 ecr 27246226], length 0
E..4..@.@.".........#...\
...n.......(.....
........
16:30:55.949245 IP iam.9160 > iam.47134: Flags [P.], seq 5310:5461, ack
479, win 256, options [nop,nop,TS val 27246227 ecr 27246226], length 151
E.....@.@."V........#...\
...n.............
....................get_range_slices...................foo..................foo...............foo1.......bar3
........&.........foo2.......bar3
........: .....
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3446) Problem SliceByNamesReadCommand
on super column family after flush operation
Posted by "Nicola Orrù (Commented JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13145684#comment-13145684 ]
Nicola Orrù commented on CASSANDRA-3446:
----------------------------------------
I had a similar problem and spent a fair amount of time tracking it down. It appears related to a problem in collating data from Memtables and SStables, but only when the query involves SuperColumns
I may have found a fix. It did solve the problem for me but I haven't tested extensively for regressions or concurrency issues.
{code}
package org.apache.cassandra.db;
public class TreeMapBackedSortedColumns extends TreeMap<ByteBuffer, IColumn> implements ISortedColumns
/*
* If we find an old column that has the same name
* the ask it to resolve itself else add the new column
*/
public void addColumn(IColumn column, Allocator allocator)
{
ByteBuffer name = column.name();
IColumn oldColumn = put(name, column);
if (oldColumn != null)
{
if (oldColumn instanceof SuperColumn)
{
assert column instanceof SuperColumn;
((SuperColumn) oldColumn).putColumn((SuperColumn)column, allocator);
// we need to restore the old value here or things won't work! --norru@scee.net
put(name, oldColumn); // <--- here it is
}
else
{
// calculate reconciled col from old (existing) col and new col
IColumn reconciledColumn = column.reconcile(oldColumn, allocator);
put(name, reconciledColumn);
}
}
}
{code}
Let me know if you need a proper patch. It's an one liner so it might be easier for you to add the line yourself.
Let me know if you need more details.
> Problem SliceByNamesReadCommand on super column family after flush operation
> ----------------------------------------------------------------------------
>
> Key: CASSANDRA-3446
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3446
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0, 1.0.1
> Environment: Linux iam 3.0.0-12-generic #20-Ubuntu SMP Fri Oct 7 14:56:25 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
> java version "1.6.0_23"
> OpenJDK Runtime Environment (IcedTea6 1.11pre) (6b23~pre10-0ubuntu5)
> OpenJDK 64-Bit Server VM (build 20.0-b11, mixed mode)
> hector-core; 1.0-1
> Reporter: Mike Smith
> Fix For: 0.7.6, 0.7.10
>
> Attachments: 3446-test.patch
>
>
> I'm having a problem with doing a multiget_slice on a super column family
> after its first flush. Updates to the column values work properly, but
> trying to retrieve the updated values using a multiget_slice operation fail
> to get the updated values. Instead they return the values from before the
> flush. The problem is not apparent with standard column families.
> I've seen this problem in Cassandra v1.0.0 and v1.0.1. The problem
> is not present in Cassandra v0.7.6.
> Steps to reproduce:
> 1. Create one or more super column entries
> 2. Verify the sub column values can be updated and that you can retrieve
> the new values
> 3. Use nodetool to flush the column family or restart cassandra
> 4. Update the sub column values
> 5. Verify they have been updated using cassandra-cli
> 6. Verify you *DO NOT* get the updated values when doing a
> multiget_slice; instead you get the old values from before the flush
> You can get the most recent value by doing a flush followed by a major
> compaction. However, future updates are not retrieved properly either.
> With debug turned on, it looks like the multiget_slice query uses the
> following command/consistency level:
> SliceByNamesReadCommand(table='test_cassandra', key=666f6f,
> columnParent='QueryPath(columnFamilyName='test', superColumnName='null',
> columnName='null')', columns=[foo,])/QUORUM.
> Cassandra-cli uses the following command/consistency level for a get_slice:
> SliceFromReadCommand(table='test_cassandra', key='666f6f',
> column_parent='QueryPath(columnFamilyName='test', superColumnName='null',
> columnName='null')', start='', finish='', reversed=false,
> count=1000000)/QUORUM
> Notice the test program gets 'bar2' for the column values and cassandra-cli
> gets 'bar3' for the column values:
> tcpdump from test program using hector-core:1.0-1
> 16:46:07.424562 IP iam.47158 > iam.9160: Flags [P.], seq 55:138, ack 30,
> win 257, options [nop,nop,TS val 27474096 ecr 27474095], length 83
> E....#@.@.PK.........6#.....].8......{.....
> ..8...8.........multiget_slice................foo..........test................foo.........
> 16:46:07.424575 IP iam.9160 > iam.47158: Flags [.], ack 138, win 256,
> options [nop,nop,TS val 27474096 ecr 27474096], length 0
> E..4..@.@.<.........#..6].8..........(.....
> ..8...8.
> 16:46:07.428771 IP iam.9160 > iam.47158: Flags [P.], seq 30:173, ack 138,
> win 256, options [nop,nop,TS val 27474097 ecr 27474096], length 143
> @.@.<&........#..6].8................
> ............foo...............foo...............foo1.......bar2
> ........6h........foo2.......bar2
> ........I.....
> tcpdump of cassandra-cli:
> 16:30:55.945123 IP iam.47134 > iam.9160: Flags [P.], seq 370:479, ack 5310,
> win 387, options [nop,nop,TS val 27246226 ecr 27241207], length 109
> E.....@.@.9q..........#..n.X\
> .............
> ................get_range_slices..............test.........................................................d.........
> 16:30:55.945152 IP iam.9160 > iam.47134: Flags [.], ack 479, win 256,
> options [nop,nop,TS val 27246226 ecr 27246226], length 0
> E..4..@.@.".........#...\
> ...n.......(.....
> ........
> 16:30:55.949245 IP iam.9160 > iam.47134: Flags [P.], seq 5310:5461, ack
> 479, win 256, options [nop,nop,TS val 27246227 ecr 27246226], length 151
> E.....@.@."V........#...\
> ...n.............
> ....................get_range_slices...................foo..................foo...............foo1.......bar3
> ........&.........foo2.......bar3
> ........: .....
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3446) Problem SliceByNamesReadCommand
on super column family after flush operation
Posted by "Mike Smith (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mike Smith updated CASSANDRA-3446:
----------------------------------
Fix Version/s: 0.7.10
> Problem SliceByNamesReadCommand on super column family after flush operation
> ----------------------------------------------------------------------------
>
> Key: CASSANDRA-3446
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3446
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0, 1.0.1
> Environment: Linux iam 3.0.0-12-generic #20-Ubuntu SMP Fri Oct 7 14:56:25 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
> java version "1.6.0_23"
> OpenJDK Runtime Environment (IcedTea6 1.11pre) (6b23~pre10-0ubuntu5)
> OpenJDK 64-Bit Server VM (build 20.0-b11, mixed mode)
> hector-core; 1.0-1
> Reporter: Mike Smith
> Fix For: 0.7.6, 0.7.10
>
>
> I'm having a problem with doing a multiget_slice on a super column family
> after its first flush. Updates to the column values work properly, but
> trying to retrieve the updated values using a multiget_slice operation fail
> to get the updated values. Instead they return the values from before the
> flush. The problem is not apparent with standard column families.
> I've seen this problem in Cassandra v1.0.0 and v1.0.1. The problem
> is not present in Cassandra v0.7.6.
> Steps to reproduce:
> 1. Create one or more super column entries
> 2. Verify the sub column values can be updated and that you can retrieve
> the new values
> 3. Use nodetool to flush the column family or restart cassandra
> 4. Update the sub column values
> 5. Verify they have been updated using cassandra-cli
> 6. Verify you *DO NOT* get the updated values when doing a
> multiget_slice; instead you get the old values from before the flush
> You can get the most recent value by doing a flush followed by a major
> compaction. However, future updates are not retrieved properly either.
> With debug turned on, it looks like the multiget_slice query uses the
> following command/consistency level:
> SliceByNamesReadCommand(table='test_cassandra', key=666f6f,
> columnParent='QueryPath(columnFamilyName='test', superColumnName='null',
> columnName='null')', columns=[foo,])/QUORUM.
> Cassandra-cli uses the following command/consistency level for a get_slice:
> SliceFromReadCommand(table='test_cassandra', key='666f6f',
> column_parent='QueryPath(columnFamilyName='test', superColumnName='null',
> columnName='null')', start='', finish='', reversed=false,
> count=1000000)/QUORUM
> Notice the test program gets 'bar2' for the column values and cassandra-cli
> gets 'bar3' for the column values:
> tcpdump from test program using hector-core:1.0-1
> 16:46:07.424562 IP iam.47158 > iam.9160: Flags [P.], seq 55:138, ack 30,
> win 257, options [nop,nop,TS val 27474096 ecr 27474095], length 83
> E....#@.@.PK.........6#.....].8......{.....
> ..8...8.........multiget_slice................foo..........test................foo.........
> 16:46:07.424575 IP iam.9160 > iam.47158: Flags [.], ack 138, win 256,
> options [nop,nop,TS val 27474096 ecr 27474096], length 0
> E..4..@.@.<.........#..6].8..........(.....
> ..8...8.
> 16:46:07.428771 IP iam.9160 > iam.47158: Flags [P.], seq 30:173, ack 138,
> win 256, options [nop,nop,TS val 27474097 ecr 27474096], length 143
> @.@.<&........#..6].8................
> ............foo...............foo...............foo1.......bar2
> ........6h........foo2.......bar2
> ........I.....
> tcpdump of cassandra-cli:
> 16:30:55.945123 IP iam.47134 > iam.9160: Flags [P.], seq 370:479, ack 5310,
> win 387, options [nop,nop,TS val 27246226 ecr 27241207], length 109
> E.....@.@.9q..........#..n.X\
> .............
> ................get_range_slices..............test.........................................................d.........
> 16:30:55.945152 IP iam.9160 > iam.47134: Flags [.], ack 479, win 256,
> options [nop,nop,TS val 27246226 ecr 27246226], length 0
> E..4..@.@.".........#...\
> ...n.......(.....
> ........
> 16:30:55.949245 IP iam.9160 > iam.47134: Flags [P.], seq 5310:5461, ack
> 479, win 256, options [nop,nop,TS val 27246227 ecr 27246226], length 151
> E.....@.@."V........#...\
> ...n.............
> ....................get_range_slices...................foo..................foo...............foo1.......bar3
> ........&.........foo2.......bar3
> ........: .....
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3446) Problem SliceByNamesReadCommand
on super column family after flush operation
Posted by "Sylvain Lebresne (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13146184#comment-13146184 ]
Sylvain Lebresne commented on CASSANDRA-3446:
---------------------------------------------
+1, good catches.
> Problem SliceByNamesReadCommand on super column family after flush operation
> ----------------------------------------------------------------------------
>
> Key: CASSANDRA-3446
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3446
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: Linux iam 3.0.0-12-generic #20-Ubuntu SMP Fri Oct 7 14:56:25 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
> java version "1.6.0_23"
> OpenJDK Runtime Environment (IcedTea6 1.11pre) (6b23~pre10-0ubuntu5)
> OpenJDK 64-Bit Server VM (build 20.0-b11, mixed mode)
> hector-core; 1.0-1
> Reporter: Mike Smith
> Assignee: Jonathan Ellis
> Labels: supercolumns
> Fix For: 1.0.3
>
> Attachments: 0001-fix-addColumn-and-collation-supercolumn-bugs.patch, 0002-r-m-minTimestamp-method.patch, 3446-test.patch
>
>
> I'm having a problem with doing a multiget_slice on a super column family
> after its first flush. Updates to the column values work properly, but
> trying to retrieve the updated values using a multiget_slice operation fail
> to get the updated values. Instead they return the values from before the
> flush. The problem is not apparent with standard column families.
> I've seen this problem in Cassandra v1.0.0 and v1.0.1. The problem
> is not present in Cassandra v0.7.6.
> Steps to reproduce:
> 1. Create one or more super column entries
> 2. Verify the sub column values can be updated and that you can retrieve
> the new values
> 3. Use nodetool to flush the column family or restart cassandra
> 4. Update the sub column values
> 5. Verify they have been updated using cassandra-cli
> 6. Verify you *DO NOT* get the updated values when doing a
> multiget_slice; instead you get the old values from before the flush
> You can get the most recent value by doing a flush followed by a major
> compaction. However, future updates are not retrieved properly either.
> With debug turned on, it looks like the multiget_slice query uses the
> following command/consistency level:
> SliceByNamesReadCommand(table='test_cassandra', key=666f6f,
> columnParent='QueryPath(columnFamilyName='test', superColumnName='null',
> columnName='null')', columns=[foo,])/QUORUM.
> Cassandra-cli uses the following command/consistency level for a get_slice:
> SliceFromReadCommand(table='test_cassandra', key='666f6f',
> column_parent='QueryPath(columnFamilyName='test', superColumnName='null',
> columnName='null')', start='', finish='', reversed=false,
> count=1000000)/QUORUM
> Notice the test program gets 'bar2' for the column values and cassandra-cli
> gets 'bar3' for the column values:
> tcpdump from test program using hector-core:1.0-1
> 16:46:07.424562 IP iam.47158 > iam.9160: Flags [P.], seq 55:138, ack 30,
> win 257, options [nop,nop,TS val 27474096 ecr 27474095], length 83
> E....#@.@.PK.........6#.....].8......{.....
> ..8...8.........multiget_slice................foo..........test................foo.........
> 16:46:07.424575 IP iam.9160 > iam.47158: Flags [.], ack 138, win 256,
> options [nop,nop,TS val 27474096 ecr 27474096], length 0
> E..4..@.@.<.........#..6].8..........(.....
> ..8...8.
> 16:46:07.428771 IP iam.9160 > iam.47158: Flags [P.], seq 30:173, ack 138,
> win 256, options [nop,nop,TS val 27474097 ecr 27474096], length 143
> @.@.<&........#..6].8................
> ............foo...............foo...............foo1.......bar2
> ........6h........foo2.......bar2
> ........I.....
> tcpdump of cassandra-cli:
> 16:30:55.945123 IP iam.47134 > iam.9160: Flags [P.], seq 370:479, ack 5310,
> win 387, options [nop,nop,TS val 27246226 ecr 27241207], length 109
> E.....@.@.9q..........#..n.X\
> .............
> ................get_range_slices..............test.........................................................d.........
> 16:30:55.945152 IP iam.9160 > iam.47134: Flags [.], ack 479, win 256,
> options [nop,nop,TS val 27246226 ecr 27246226], length 0
> E..4..@.@.".........#...\
> ...n.......(.....
> ........
> 16:30:55.949245 IP iam.9160 > iam.47134: Flags [P.], seq 5310:5461, ack
> 479, win 256, options [nop,nop,TS val 27246227 ecr 27246226], length 151
> E.....@.@."V........#...\
> ...n.............
> ....................get_range_slices...................foo..................foo...............foo1.......bar3
> ........&.........foo2.......bar3
> ........: .....
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3446) Problem SliceByNamesReadCommand
on super column family after flush operation
Posted by "Rick Branson (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rick Branson updated CASSANDRA-3446:
------------------------------------
Attachment: 3446-test.patch
Adds a unit test to ColumnFamilyStoreTest to reproduce the issue.
> Problem SliceByNamesReadCommand on super column family after flush operation
> ----------------------------------------------------------------------------
>
> Key: CASSANDRA-3446
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3446
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0, 1.0.1
> Environment: Linux iam 3.0.0-12-generic #20-Ubuntu SMP Fri Oct 7 14:56:25 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
> java version "1.6.0_23"
> OpenJDK Runtime Environment (IcedTea6 1.11pre) (6b23~pre10-0ubuntu5)
> OpenJDK 64-Bit Server VM (build 20.0-b11, mixed mode)
> hector-core; 1.0-1
> Reporter: Mike Smith
> Fix For: 0.7.6, 0.7.10
>
> Attachments: 3446-test.patch
>
>
> I'm having a problem with doing a multiget_slice on a super column family
> after its first flush. Updates to the column values work properly, but
> trying to retrieve the updated values using a multiget_slice operation fail
> to get the updated values. Instead they return the values from before the
> flush. The problem is not apparent with standard column families.
> I've seen this problem in Cassandra v1.0.0 and v1.0.1. The problem
> is not present in Cassandra v0.7.6.
> Steps to reproduce:
> 1. Create one or more super column entries
> 2. Verify the sub column values can be updated and that you can retrieve
> the new values
> 3. Use nodetool to flush the column family or restart cassandra
> 4. Update the sub column values
> 5. Verify they have been updated using cassandra-cli
> 6. Verify you *DO NOT* get the updated values when doing a
> multiget_slice; instead you get the old values from before the flush
> You can get the most recent value by doing a flush followed by a major
> compaction. However, future updates are not retrieved properly either.
> With debug turned on, it looks like the multiget_slice query uses the
> following command/consistency level:
> SliceByNamesReadCommand(table='test_cassandra', key=666f6f,
> columnParent='QueryPath(columnFamilyName='test', superColumnName='null',
> columnName='null')', columns=[foo,])/QUORUM.
> Cassandra-cli uses the following command/consistency level for a get_slice:
> SliceFromReadCommand(table='test_cassandra', key='666f6f',
> column_parent='QueryPath(columnFamilyName='test', superColumnName='null',
> columnName='null')', start='', finish='', reversed=false,
> count=1000000)/QUORUM
> Notice the test program gets 'bar2' for the column values and cassandra-cli
> gets 'bar3' for the column values:
> tcpdump from test program using hector-core:1.0-1
> 16:46:07.424562 IP iam.47158 > iam.9160: Flags [P.], seq 55:138, ack 30,
> win 257, options [nop,nop,TS val 27474096 ecr 27474095], length 83
> E....#@.@.PK.........6#.....].8......{.....
> ..8...8.........multiget_slice................foo..........test................foo.........
> 16:46:07.424575 IP iam.9160 > iam.47158: Flags [.], ack 138, win 256,
> options [nop,nop,TS val 27474096 ecr 27474096], length 0
> E..4..@.@.<.........#..6].8..........(.....
> ..8...8.
> 16:46:07.428771 IP iam.9160 > iam.47158: Flags [P.], seq 30:173, ack 138,
> win 256, options [nop,nop,TS val 27474097 ecr 27474096], length 143
> @.@.<&........#..6].8................
> ............foo...............foo...............foo1.......bar2
> ........6h........foo2.......bar2
> ........I.....
> tcpdump of cassandra-cli:
> 16:30:55.945123 IP iam.47134 > iam.9160: Flags [P.], seq 370:479, ack 5310,
> win 387, options [nop,nop,TS val 27246226 ecr 27241207], length 109
> E.....@.@.9q..........#..n.X\
> .............
> ................get_range_slices..............test.........................................................d.........
> 16:30:55.945152 IP iam.9160 > iam.47134: Flags [.], ack 479, win 256,
> options [nop,nop,TS val 27246226 ecr 27246226], length 0
> E..4..@.@.".........#...\
> ...n.......(.....
> ........
> 16:30:55.949245 IP iam.9160 > iam.47134: Flags [P.], seq 5310:5461, ack
> 479, win 256, options [nop,nop,TS val 27246227 ecr 27246226], length 151
> E.....@.@."V........#...\
> ...n.............
> ....................get_range_slices...................foo..................foo...............foo1.......bar3
> ........&.........foo2.......bar3
> ........: .....
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3446) Problem SliceByNamesReadCommand
on super column family after flush operation
Posted by "Mike Smith (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13142462#comment-13142462 ]
Mike Smith commented on CASSANDRA-3446:
---------------------------------------
create keyspace test_cassandra;
use test_cassandra;
create column family test1 with column_type = 'Super' and comparator =
'AsciiType';
create column family test2 with column_type = 'Standard' and comparator =
'AsciiType';
> Problem SliceByNamesReadCommand on super column family after flush operation
> ----------------------------------------------------------------------------
>
> Key: CASSANDRA-3446
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3446
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0, 1.0.1
> Environment: Linux iam 3.0.0-12-generic #20-Ubuntu SMP Fri Oct 7 14:56:25 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
> java version "1.6.0_23"
> OpenJDK Runtime Environment (IcedTea6 1.11pre) (6b23~pre10-0ubuntu5)
> OpenJDK 64-Bit Server VM (build 20.0-b11, mixed mode)
> hector-core; 1.0-1
> Reporter: Mike Smith
> Fix For: 0.7.6
>
>
> I'm having a problem with doing a multiget_slice on a super column family
> after its first flush. Updates to the column values work properly, but
> trying to retrieve the updated values using a multiget_slice operation fail
> to get the updated values. Instead they return the values from before the
> flush. The problem is not apparent with standard column families.
> I've seen this problem in Cassandra v1.0.0 and v1.0.1. The problem
> is not present in Cassandra v0.7.6.
> Steps to reproduce:
> 1. Create one or more super column entries
> 2. Verify the sub column values can be updated and that you can retrieve
> the new values
> 3. Use nodetool to flush the column family or restart cassandra
> 4. Update the sub column values
> 5. Verify they have been updated using cassandra-cli
> 6. Verify you *DO NOT* get the updated values when doing a
> multiget_slice; instead you get the old values from before the flush
> You can get the most recent value by doing a flush followed by a major
> compaction. However, future updates are not retrieved properly either.
> With debug turned on, it looks like the multiget_slice query uses the
> following command/consistency level:
> SliceByNamesReadCommand(table='test_cassandra', key=666f6f,
> columnParent='QueryPath(columnFamilyName='test', superColumnName='null',
> columnName='null')', columns=[foo,])/QUORUM.
> Cassandra-cli uses the following command/consistency level for a get_slice:
> SliceFromReadCommand(table='test_cassandra', key='666f6f',
> column_parent='QueryPath(columnFamilyName='test', superColumnName='null',
> columnName='null')', start='', finish='', reversed=false,
> count=1000000)/QUORUM
> Notice the test program gets 'bar2' for the column values and cassandra-cli
> gets 'bar3' for the column values:
> tcpdump from test program using hector-core:1.0-1
> 16:46:07.424562 IP iam.47158 > iam.9160: Flags [P.], seq 55:138, ack 30,
> win 257, options [nop,nop,TS val 27474096 ecr 27474095], length 83
> E....#@.@.PK.........6#.....].8......{.....
> ..8...8.........multiget_slice................foo..........test................foo.........
> 16:46:07.424575 IP iam.9160 > iam.47158: Flags [.], ack 138, win 256,
> options [nop,nop,TS val 27474096 ecr 27474096], length 0
> E..4..@.@.<.........#..6].8..........(.....
> ..8...8.
> 16:46:07.428771 IP iam.9160 > iam.47158: Flags [P.], seq 30:173, ack 138,
> win 256, options [nop,nop,TS val 27474097 ecr 27474096], length 143
> @.@.<&........#..6].8................
> ............foo...............foo...............foo1.......bar2
> ........6h........foo2.......bar2
> ........I.....
> tcpdump of cassandra-cli:
> 16:30:55.945123 IP iam.47134 > iam.9160: Flags [P.], seq 370:479, ack 5310,
> win 387, options [nop,nop,TS val 27246226 ecr 27241207], length 109
> E.....@.@.9q..........#..n.X\
> .............
> ................get_range_slices..............test.........................................................d.........
> 16:30:55.945152 IP iam.9160 > iam.47134: Flags [.], ack 479, win 256,
> options [nop,nop,TS val 27246226 ecr 27246226], length 0
> E..4..@.@.".........#...\
> ...n.......(.....
> ........
> 16:30:55.949245 IP iam.9160 > iam.47134: Flags [P.], seq 5310:5461, ack
> 479, win 256, options [nop,nop,TS val 27246227 ecr 27246226], length 151
> E.....@.@."V........#...\
> ...n.............
> ....................get_range_slices...................foo..................foo...............foo1.......bar3
> ........&.........foo2.......bar3
> ........: .....
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-3446) Problem
SliceByNamesReadCommand on super column family after flush operation
Posted by "Nicola Orrù (Issue Comment Edited JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13145684#comment-13145684 ]
Nicola Orrù edited comment on CASSANDRA-3446 at 11/7/11 6:38 PM:
-----------------------------------------------------------------
I had a similar problem and spent a fair amount of time tracking it down. It appears related to a problem in collating data from Memtables and SStables, but only when the query involves SuperColumns
I may have found a fix. It did solve the problem for me but I haven't tested extensively for regressions or concurrency issues.
{code}
package org.apache.cassandra.db;
public class TreeMapBackedSortedColumns extends TreeMap<ByteBuffer, IColumn> implements ISortedColumns
/*
* If we find an old column that has the same name
* the ask it to resolve itself else add the new column
*/
public void addColumn(IColumn column, Allocator allocator)
{
ByteBuffer name = column.name();
IColumn oldColumn = put(name, column);
if (oldColumn != null)
{
if (oldColumn instanceof SuperColumn)
{
assert column instanceof SuperColumn;
((SuperColumn) oldColumn).putColumn((SuperColumn)column, allocator);
// we need to restore the old value here or things won't work! --norru@scee.net
put(name, oldColumn); // <--- here it is
}
else
{
// calculate reconciled col from old (existing) col and new col
IColumn reconciledColumn = column.reconcile(oldColumn, allocator);
put(name, reconciledColumn);
}
}
}
{code}
Let me know if you need a proper patch. It's an one liner so it might be easier for you to add the line yourself.
Also let me know if you need more details. Cheers!
was (Author: norru):
I had a similar problem and spent a fair amount of time tracking it down. It appears related to a problem in collating data from Memtables and SStables, but only when the query involves SuperColumns
I may have found a fix. It did solve the problem for me but I haven't tested extensively for regressions or concurrency issues.
{code}
package org.apache.cassandra.db;
public class TreeMapBackedSortedColumns extends TreeMap<ByteBuffer, IColumn> implements ISortedColumns
/*
* If we find an old column that has the same name
* the ask it to resolve itself else add the new column
*/
public void addColumn(IColumn column, Allocator allocator)
{
ByteBuffer name = column.name();
IColumn oldColumn = put(name, column);
if (oldColumn != null)
{
if (oldColumn instanceof SuperColumn)
{
assert column instanceof SuperColumn;
((SuperColumn) oldColumn).putColumn((SuperColumn)column, allocator);
// we need to restore the old value here or things won't work! --norru@scee.net
put(name, oldColumn); // <--- here it is
}
else
{
// calculate reconciled col from old (existing) col and new col
IColumn reconciledColumn = column.reconcile(oldColumn, allocator);
put(name, reconciledColumn);
}
}
}
{code}
Let me know if you need a proper patch. It's an one liner so it might be easier for you to add the line yourself.
Let me know if you need more details.
> Problem SliceByNamesReadCommand on super column family after flush operation
> ----------------------------------------------------------------------------
>
> Key: CASSANDRA-3446
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3446
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0, 1.0.1
> Environment: Linux iam 3.0.0-12-generic #20-Ubuntu SMP Fri Oct 7 14:56:25 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
> java version "1.6.0_23"
> OpenJDK Runtime Environment (IcedTea6 1.11pre) (6b23~pre10-0ubuntu5)
> OpenJDK 64-Bit Server VM (build 20.0-b11, mixed mode)
> hector-core; 1.0-1
> Reporter: Mike Smith
> Fix For: 0.7.6, 0.7.10
>
> Attachments: 3446-test.patch
>
>
> I'm having a problem with doing a multiget_slice on a super column family
> after its first flush. Updates to the column values work properly, but
> trying to retrieve the updated values using a multiget_slice operation fail
> to get the updated values. Instead they return the values from before the
> flush. The problem is not apparent with standard column families.
> I've seen this problem in Cassandra v1.0.0 and v1.0.1. The problem
> is not present in Cassandra v0.7.6.
> Steps to reproduce:
> 1. Create one or more super column entries
> 2. Verify the sub column values can be updated and that you can retrieve
> the new values
> 3. Use nodetool to flush the column family or restart cassandra
> 4. Update the sub column values
> 5. Verify they have been updated using cassandra-cli
> 6. Verify you *DO NOT* get the updated values when doing a
> multiget_slice; instead you get the old values from before the flush
> You can get the most recent value by doing a flush followed by a major
> compaction. However, future updates are not retrieved properly either.
> With debug turned on, it looks like the multiget_slice query uses the
> following command/consistency level:
> SliceByNamesReadCommand(table='test_cassandra', key=666f6f,
> columnParent='QueryPath(columnFamilyName='test', superColumnName='null',
> columnName='null')', columns=[foo,])/QUORUM.
> Cassandra-cli uses the following command/consistency level for a get_slice:
> SliceFromReadCommand(table='test_cassandra', key='666f6f',
> column_parent='QueryPath(columnFamilyName='test', superColumnName='null',
> columnName='null')', start='', finish='', reversed=false,
> count=1000000)/QUORUM
> Notice the test program gets 'bar2' for the column values and cassandra-cli
> gets 'bar3' for the column values:
> tcpdump from test program using hector-core:1.0-1
> 16:46:07.424562 IP iam.47158 > iam.9160: Flags [P.], seq 55:138, ack 30,
> win 257, options [nop,nop,TS val 27474096 ecr 27474095], length 83
> E....#@.@.PK.........6#.....].8......{.....
> ..8...8.........multiget_slice................foo..........test................foo.........
> 16:46:07.424575 IP iam.9160 > iam.47158: Flags [.], ack 138, win 256,
> options [nop,nop,TS val 27474096 ecr 27474096], length 0
> E..4..@.@.<.........#..6].8..........(.....
> ..8...8.
> 16:46:07.428771 IP iam.9160 > iam.47158: Flags [P.], seq 30:173, ack 138,
> win 256, options [nop,nop,TS val 27474097 ecr 27474096], length 143
> @.@.<&........#..6].8................
> ............foo...............foo...............foo1.......bar2
> ........6h........foo2.......bar2
> ........I.....
> tcpdump of cassandra-cli:
> 16:30:55.945123 IP iam.47134 > iam.9160: Flags [P.], seq 370:479, ack 5310,
> win 387, options [nop,nop,TS val 27246226 ecr 27241207], length 109
> E.....@.@.9q..........#..n.X\
> .............
> ................get_range_slices..............test.........................................................d.........
> 16:30:55.945152 IP iam.9160 > iam.47134: Flags [.], ack 479, win 256,
> options [nop,nop,TS val 27246226 ecr 27246226], length 0
> E..4..@.@.".........#...\
> ...n.......(.....
> ........
> 16:30:55.949245 IP iam.9160 > iam.47134: Flags [P.], seq 5310:5461, ack
> 479, win 256, options [nop,nop,TS val 27246227 ecr 27246226], length 151
> E.....@.@."V........#...\
> ...n.............
> ....................get_range_slices...................foo..................foo...............foo1.......bar3
> ........&.........foo2.......bar3
> ........: .....
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3446) Problem SliceByNamesReadCommand
on super column family after flush operation
Posted by "Mike Smith (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13142833#comment-13142833 ]
Mike Smith commented on CASSANDRA-3446:
---------------------------------------
Yes, single node cluster.
On Wed, Nov 2, 2011 at 9:43 PM, Jonathan Ellis (Commented) (JIRA) <
> Problem SliceByNamesReadCommand on super column family after flush operation
> ----------------------------------------------------------------------------
>
> Key: CASSANDRA-3446
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3446
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0, 1.0.1
> Environment: Linux iam 3.0.0-12-generic #20-Ubuntu SMP Fri Oct 7 14:56:25 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
> java version "1.6.0_23"
> OpenJDK Runtime Environment (IcedTea6 1.11pre) (6b23~pre10-0ubuntu5)
> OpenJDK 64-Bit Server VM (build 20.0-b11, mixed mode)
> hector-core; 1.0-1
> Reporter: Mike Smith
> Fix For: 0.7.6, 0.7.10
>
>
> I'm having a problem with doing a multiget_slice on a super column family
> after its first flush. Updates to the column values work properly, but
> trying to retrieve the updated values using a multiget_slice operation fail
> to get the updated values. Instead they return the values from before the
> flush. The problem is not apparent with standard column families.
> I've seen this problem in Cassandra v1.0.0 and v1.0.1. The problem
> is not present in Cassandra v0.7.6.
> Steps to reproduce:
> 1. Create one or more super column entries
> 2. Verify the sub column values can be updated and that you can retrieve
> the new values
> 3. Use nodetool to flush the column family or restart cassandra
> 4. Update the sub column values
> 5. Verify they have been updated using cassandra-cli
> 6. Verify you *DO NOT* get the updated values when doing a
> multiget_slice; instead you get the old values from before the flush
> You can get the most recent value by doing a flush followed by a major
> compaction. However, future updates are not retrieved properly either.
> With debug turned on, it looks like the multiget_slice query uses the
> following command/consistency level:
> SliceByNamesReadCommand(table='test_cassandra', key=666f6f,
> columnParent='QueryPath(columnFamilyName='test', superColumnName='null',
> columnName='null')', columns=[foo,])/QUORUM.
> Cassandra-cli uses the following command/consistency level for a get_slice:
> SliceFromReadCommand(table='test_cassandra', key='666f6f',
> column_parent='QueryPath(columnFamilyName='test', superColumnName='null',
> columnName='null')', start='', finish='', reversed=false,
> count=1000000)/QUORUM
> Notice the test program gets 'bar2' for the column values and cassandra-cli
> gets 'bar3' for the column values:
> tcpdump from test program using hector-core:1.0-1
> 16:46:07.424562 IP iam.47158 > iam.9160: Flags [P.], seq 55:138, ack 30,
> win 257, options [nop,nop,TS val 27474096 ecr 27474095], length 83
> E....#@.@.PK.........6#.....].8......{.....
> ..8...8.........multiget_slice................foo..........test................foo.........
> 16:46:07.424575 IP iam.9160 > iam.47158: Flags [.], ack 138, win 256,
> options [nop,nop,TS val 27474096 ecr 27474096], length 0
> E..4..@.@.<.........#..6].8..........(.....
> ..8...8.
> 16:46:07.428771 IP iam.9160 > iam.47158: Flags [P.], seq 30:173, ack 138,
> win 256, options [nop,nop,TS val 27474097 ecr 27474096], length 143
> @.@.<&........#..6].8................
> ............foo...............foo...............foo1.......bar2
> ........6h........foo2.......bar2
> ........I.....
> tcpdump of cassandra-cli:
> 16:30:55.945123 IP iam.47134 > iam.9160: Flags [P.], seq 370:479, ack 5310,
> win 387, options [nop,nop,TS val 27246226 ecr 27241207], length 109
> E.....@.@.9q..........#..n.X\
> .............
> ................get_range_slices..............test.........................................................d.........
> 16:30:55.945152 IP iam.9160 > iam.47134: Flags [.], ack 479, win 256,
> options [nop,nop,TS val 27246226 ecr 27246226], length 0
> E..4..@.@.".........#...\
> ...n.......(.....
> ........
> 16:30:55.949245 IP iam.9160 > iam.47134: Flags [P.], seq 5310:5461, ack
> 479, win 256, options [nop,nop,TS val 27246227 ecr 27246226], length 151
> E.....@.@."V........#...\
> ...n.............
> ....................get_range_slices...................foo..................foo...............foo1.......bar3
> ........&.........foo2.......bar3
> ........: .....
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3446) Problem SliceByNamesReadCommand
on super column family after flush operation
Posted by "Jonathan Ellis (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13142401#comment-13142401 ]
Jonathan Ellis commented on CASSANDRA-3446:
-------------------------------------------
Can you post your supercolumn create statement?
> Problem SliceByNamesReadCommand on super column family after flush operation
> ----------------------------------------------------------------------------
>
> Key: CASSANDRA-3446
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3446
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0, 1.0.1
> Environment: Linux iam 3.0.0-12-generic #20-Ubuntu SMP Fri Oct 7 14:56:25 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
> java version "1.6.0_23"
> OpenJDK Runtime Environment (IcedTea6 1.11pre) (6b23~pre10-0ubuntu5)
> OpenJDK 64-Bit Server VM (build 20.0-b11, mixed mode)
> hector-core; 1.0-1
> Reporter: Mike Smith
> Fix For: 0.7.6
>
>
> I'm having a problem with doing a multiget_slice on a super column family
> after its first flush. Updates to the column values work properly, but
> trying to retrieve the updated values using a multiget_slice operation fail
> to get the updated values. Instead they return the values from before the
> flush. The problem is not apparent with standard column families.
> I've seen this problem in Cassandra v1.0.0 and v1.0.1. The problem
> is not present in Cassandra v0.7.6.
> Steps to reproduce:
> 1. Create one or more super column entries
> 2. Verify the sub column values can be updated and that you can retrieve
> the new values
> 3. Use nodetool to flush the column family or restart cassandra
> 4. Update the sub column values
> 5. Verify they have been updated using cassandra-cli
> 6. Verify you *DO NOT* get the updated values when doing a
> multiget_slice; instead you get the old values from before the flush
> You can get the most recent value by doing a flush followed by a major
> compaction. However, future updates are not retrieved properly either.
> With debug turned on, it looks like the multiget_slice query uses the
> following command/consistency level:
> SliceByNamesReadCommand(table='test_cassandra', key=666f6f,
> columnParent='QueryPath(columnFamilyName='test', superColumnName='null',
> columnName='null')', columns=[foo,])/QUORUM.
> Cassandra-cli uses the following command/consistency level for a get_slice:
> SliceFromReadCommand(table='test_cassandra', key='666f6f',
> column_parent='QueryPath(columnFamilyName='test', superColumnName='null',
> columnName='null')', start='', finish='', reversed=false,
> count=1000000)/QUORUM
> Notice the test program gets 'bar2' for the column values and cassandra-cli
> gets 'bar3' for the column values:
> tcpdump from test program using hector-core:1.0-1
> 16:46:07.424562 IP iam.47158 > iam.9160: Flags [P.], seq 55:138, ack 30,
> win 257, options [nop,nop,TS val 27474096 ecr 27474095], length 83
> E....#@.@.PK.........6#.....].8......{.....
> ..8...8.........multiget_slice................foo..........test................foo.........
> 16:46:07.424575 IP iam.9160 > iam.47158: Flags [.], ack 138, win 256,
> options [nop,nop,TS val 27474096 ecr 27474096], length 0
> E..4..@.@.<.........#..6].8..........(.....
> ..8...8.
> 16:46:07.428771 IP iam.9160 > iam.47158: Flags [P.], seq 30:173, ack 138,
> win 256, options [nop,nop,TS val 27474097 ecr 27474096], length 143
> @.@.<&........#..6].8................
> ............foo...............foo...............foo1.......bar2
> ........6h........foo2.......bar2
> ........I.....
> tcpdump of cassandra-cli:
> 16:30:55.945123 IP iam.47134 > iam.9160: Flags [P.], seq 370:479, ack 5310,
> win 387, options [nop,nop,TS val 27246226 ecr 27241207], length 109
> E.....@.@.9q..........#..n.X\
> .............
> ................get_range_slices..............test.........................................................d.........
> 16:30:55.945152 IP iam.9160 > iam.47134: Flags [.], ack 479, win 256,
> options [nop,nop,TS val 27246226 ecr 27246226], length 0
> E..4..@.@.".........#...\
> ...n.......(.....
> ........
> 16:30:55.949245 IP iam.9160 > iam.47134: Flags [P.], seq 5310:5461, ack
> 479, win 256, options [nop,nop,TS val 27246227 ecr 27246226], length 151
> E.....@.@."V........#...\
> ...n.............
> ....................get_range_slices...................foo..................foo...............foo1.......bar3
> ........&.........foo2.......bar3
> ........: .....
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3446) Problem SliceByNamesReadCommand
on super column family after flush operation
Posted by "Mike Smith (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13145864#comment-13145864 ]
Mike Smith commented on CASSANDRA-3446:
---------------------------------------
I applied the patches to 1.0.2 and all looks good. Thanks a lot for the quick turn around and keep up the good work!!
> Problem SliceByNamesReadCommand on super column family after flush operation
> ----------------------------------------------------------------------------
>
> Key: CASSANDRA-3446
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3446
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: Linux iam 3.0.0-12-generic #20-Ubuntu SMP Fri Oct 7 14:56:25 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
> java version "1.6.0_23"
> OpenJDK Runtime Environment (IcedTea6 1.11pre) (6b23~pre10-0ubuntu5)
> OpenJDK 64-Bit Server VM (build 20.0-b11, mixed mode)
> hector-core; 1.0-1
> Reporter: Mike Smith
> Assignee: Jonathan Ellis
> Labels: supercolumns
> Fix For: 1.0.3
>
> Attachments: 0001-fix-addColumn-and-collation-supercolumn-bugs.patch, 0002-r-m-minTimestamp-method.patch, 3446-test.patch
>
>
> I'm having a problem with doing a multiget_slice on a super column family
> after its first flush. Updates to the column values work properly, but
> trying to retrieve the updated values using a multiget_slice operation fail
> to get the updated values. Instead they return the values from before the
> flush. The problem is not apparent with standard column families.
> I've seen this problem in Cassandra v1.0.0 and v1.0.1. The problem
> is not present in Cassandra v0.7.6.
> Steps to reproduce:
> 1. Create one or more super column entries
> 2. Verify the sub column values can be updated and that you can retrieve
> the new values
> 3. Use nodetool to flush the column family or restart cassandra
> 4. Update the sub column values
> 5. Verify they have been updated using cassandra-cli
> 6. Verify you *DO NOT* get the updated values when doing a
> multiget_slice; instead you get the old values from before the flush
> You can get the most recent value by doing a flush followed by a major
> compaction. However, future updates are not retrieved properly either.
> With debug turned on, it looks like the multiget_slice query uses the
> following command/consistency level:
> SliceByNamesReadCommand(table='test_cassandra', key=666f6f,
> columnParent='QueryPath(columnFamilyName='test', superColumnName='null',
> columnName='null')', columns=[foo,])/QUORUM.
> Cassandra-cli uses the following command/consistency level for a get_slice:
> SliceFromReadCommand(table='test_cassandra', key='666f6f',
> column_parent='QueryPath(columnFamilyName='test', superColumnName='null',
> columnName='null')', start='', finish='', reversed=false,
> count=1000000)/QUORUM
> Notice the test program gets 'bar2' for the column values and cassandra-cli
> gets 'bar3' for the column values:
> tcpdump from test program using hector-core:1.0-1
> 16:46:07.424562 IP iam.47158 > iam.9160: Flags [P.], seq 55:138, ack 30,
> win 257, options [nop,nop,TS val 27474096 ecr 27474095], length 83
> E....#@.@.PK.........6#.....].8......{.....
> ..8...8.........multiget_slice................foo..........test................foo.........
> 16:46:07.424575 IP iam.9160 > iam.47158: Flags [.], ack 138, win 256,
> options [nop,nop,TS val 27474096 ecr 27474096], length 0
> E..4..@.@.<.........#..6].8..........(.....
> ..8...8.
> 16:46:07.428771 IP iam.9160 > iam.47158: Flags [P.], seq 30:173, ack 138,
> win 256, options [nop,nop,TS val 27474097 ecr 27474096], length 143
> @.@.<&........#..6].8................
> ............foo...............foo...............foo1.......bar2
> ........6h........foo2.......bar2
> ........I.....
> tcpdump of cassandra-cli:
> 16:30:55.945123 IP iam.47134 > iam.9160: Flags [P.], seq 370:479, ack 5310,
> win 387, options [nop,nop,TS val 27246226 ecr 27241207], length 109
> E.....@.@.9q..........#..n.X\
> .............
> ................get_range_slices..............test.........................................................d.........
> 16:30:55.945152 IP iam.9160 > iam.47134: Flags [.], ack 479, win 256,
> options [nop,nop,TS val 27246226 ecr 27246226], length 0
> E..4..@.@.".........#...\
> ...n.......(.....
> ........
> 16:30:55.949245 IP iam.9160 > iam.47134: Flags [P.], seq 5310:5461, ack
> 479, win 256, options [nop,nop,TS val 27246227 ecr 27246226], length 151
> E.....@.@."V........#...\
> ...n.............
> ....................get_range_slices...................foo..................foo...............foo1.......bar3
> ........&.........foo2.......bar3
> ........: .....
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-3446) Problem
SliceByNamesReadCommand on super column family after flush operation
Posted by "Jonathan Ellis (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13142401#comment-13142401 ]
Jonathan Ellis edited comment on CASSANDRA-3446 at 11/2/11 6:40 PM:
--------------------------------------------------------------------
Can you post your columnfamily create statement?
was (Author: jbellis):
Can you post your supercolumn create statement?
> Problem SliceByNamesReadCommand on super column family after flush operation
> ----------------------------------------------------------------------------
>
> Key: CASSANDRA-3446
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3446
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0, 1.0.1
> Environment: Linux iam 3.0.0-12-generic #20-Ubuntu SMP Fri Oct 7 14:56:25 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
> java version "1.6.0_23"
> OpenJDK Runtime Environment (IcedTea6 1.11pre) (6b23~pre10-0ubuntu5)
> OpenJDK 64-Bit Server VM (build 20.0-b11, mixed mode)
> hector-core; 1.0-1
> Reporter: Mike Smith
> Fix For: 0.7.6
>
>
> I'm having a problem with doing a multiget_slice on a super column family
> after its first flush. Updates to the column values work properly, but
> trying to retrieve the updated values using a multiget_slice operation fail
> to get the updated values. Instead they return the values from before the
> flush. The problem is not apparent with standard column families.
> I've seen this problem in Cassandra v1.0.0 and v1.0.1. The problem
> is not present in Cassandra v0.7.6.
> Steps to reproduce:
> 1. Create one or more super column entries
> 2. Verify the sub column values can be updated and that you can retrieve
> the new values
> 3. Use nodetool to flush the column family or restart cassandra
> 4. Update the sub column values
> 5. Verify they have been updated using cassandra-cli
> 6. Verify you *DO NOT* get the updated values when doing a
> multiget_slice; instead you get the old values from before the flush
> You can get the most recent value by doing a flush followed by a major
> compaction. However, future updates are not retrieved properly either.
> With debug turned on, it looks like the multiget_slice query uses the
> following command/consistency level:
> SliceByNamesReadCommand(table='test_cassandra', key=666f6f,
> columnParent='QueryPath(columnFamilyName='test', superColumnName='null',
> columnName='null')', columns=[foo,])/QUORUM.
> Cassandra-cli uses the following command/consistency level for a get_slice:
> SliceFromReadCommand(table='test_cassandra', key='666f6f',
> column_parent='QueryPath(columnFamilyName='test', superColumnName='null',
> columnName='null')', start='', finish='', reversed=false,
> count=1000000)/QUORUM
> Notice the test program gets 'bar2' for the column values and cassandra-cli
> gets 'bar3' for the column values:
> tcpdump from test program using hector-core:1.0-1
> 16:46:07.424562 IP iam.47158 > iam.9160: Flags [P.], seq 55:138, ack 30,
> win 257, options [nop,nop,TS val 27474096 ecr 27474095], length 83
> E....#@.@.PK.........6#.....].8......{.....
> ..8...8.........multiget_slice................foo..........test................foo.........
> 16:46:07.424575 IP iam.9160 > iam.47158: Flags [.], ack 138, win 256,
> options [nop,nop,TS val 27474096 ecr 27474096], length 0
> E..4..@.@.<.........#..6].8..........(.....
> ..8...8.
> 16:46:07.428771 IP iam.9160 > iam.47158: Flags [P.], seq 30:173, ack 138,
> win 256, options [nop,nop,TS val 27474097 ecr 27474096], length 143
> @.@.<&........#..6].8................
> ............foo...............foo...............foo1.......bar2
> ........6h........foo2.......bar2
> ........I.....
> tcpdump of cassandra-cli:
> 16:30:55.945123 IP iam.47134 > iam.9160: Flags [P.], seq 370:479, ack 5310,
> win 387, options [nop,nop,TS val 27246226 ecr 27241207], length 109
> E.....@.@.9q..........#..n.X\
> .............
> ................get_range_slices..............test.........................................................d.........
> 16:30:55.945152 IP iam.9160 > iam.47134: Flags [.], ack 479, win 256,
> options [nop,nop,TS val 27246226 ecr 27246226], length 0
> E..4..@.@.".........#...\
> ...n.......(.....
> ........
> 16:30:55.949245 IP iam.9160 > iam.47134: Flags [P.], seq 5310:5461, ack
> 479, win 256, options [nop,nop,TS val 27246227 ecr 27246226], length 151
> E.....@.@."V........#...\
> ...n.............
> ....................get_range_slices...................foo..................foo...............foo1.......bar3
> ........&.........foo2.......bar3
> ........: .....
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3446) Problem SliceByNamesReadCommand
on super column family after flush operation
Posted by "Jonathan Ellis (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-3446:
--------------------------------------
Attachment: 0002-r-m-minTimestamp-method.patch
0001-fix-addColumn-and-collation-supercolumn-bugs.patch
> Problem SliceByNamesReadCommand on super column family after flush operation
> ----------------------------------------------------------------------------
>
> Key: CASSANDRA-3446
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3446
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: Linux iam 3.0.0-12-generic #20-Ubuntu SMP Fri Oct 7 14:56:25 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
> java version "1.6.0_23"
> OpenJDK Runtime Environment (IcedTea6 1.11pre) (6b23~pre10-0ubuntu5)
> OpenJDK 64-Bit Server VM (build 20.0-b11, mixed mode)
> hector-core; 1.0-1
> Reporter: Mike Smith
> Assignee: Jonathan Ellis
> Labels: supercolumns
> Fix For: 1.0.3
>
> Attachments: 0001-fix-addColumn-and-collation-supercolumn-bugs.patch, 0002-r-m-minTimestamp-method.patch, 3446-test.patch
>
>
> I'm having a problem with doing a multiget_slice on a super column family
> after its first flush. Updates to the column values work properly, but
> trying to retrieve the updated values using a multiget_slice operation fail
> to get the updated values. Instead they return the values from before the
> flush. The problem is not apparent with standard column families.
> I've seen this problem in Cassandra v1.0.0 and v1.0.1. The problem
> is not present in Cassandra v0.7.6.
> Steps to reproduce:
> 1. Create one or more super column entries
> 2. Verify the sub column values can be updated and that you can retrieve
> the new values
> 3. Use nodetool to flush the column family or restart cassandra
> 4. Update the sub column values
> 5. Verify they have been updated using cassandra-cli
> 6. Verify you *DO NOT* get the updated values when doing a
> multiget_slice; instead you get the old values from before the flush
> You can get the most recent value by doing a flush followed by a major
> compaction. However, future updates are not retrieved properly either.
> With debug turned on, it looks like the multiget_slice query uses the
> following command/consistency level:
> SliceByNamesReadCommand(table='test_cassandra', key=666f6f,
> columnParent='QueryPath(columnFamilyName='test', superColumnName='null',
> columnName='null')', columns=[foo,])/QUORUM.
> Cassandra-cli uses the following command/consistency level for a get_slice:
> SliceFromReadCommand(table='test_cassandra', key='666f6f',
> column_parent='QueryPath(columnFamilyName='test', superColumnName='null',
> columnName='null')', start='', finish='', reversed=false,
> count=1000000)/QUORUM
> Notice the test program gets 'bar2' for the column values and cassandra-cli
> gets 'bar3' for the column values:
> tcpdump from test program using hector-core:1.0-1
> 16:46:07.424562 IP iam.47158 > iam.9160: Flags [P.], seq 55:138, ack 30,
> win 257, options [nop,nop,TS val 27474096 ecr 27474095], length 83
> E....#@.@.PK.........6#.....].8......{.....
> ..8...8.........multiget_slice................foo..........test................foo.........
> 16:46:07.424575 IP iam.9160 > iam.47158: Flags [.], ack 138, win 256,
> options [nop,nop,TS val 27474096 ecr 27474096], length 0
> E..4..@.@.<.........#..6].8..........(.....
> ..8...8.
> 16:46:07.428771 IP iam.9160 > iam.47158: Flags [P.], seq 30:173, ack 138,
> win 256, options [nop,nop,TS val 27474097 ecr 27474096], length 143
> @.@.<&........#..6].8................
> ............foo...............foo...............foo1.......bar2
> ........6h........foo2.......bar2
> ........I.....
> tcpdump of cassandra-cli:
> 16:30:55.945123 IP iam.47134 > iam.9160: Flags [P.], seq 370:479, ack 5310,
> win 387, options [nop,nop,TS val 27246226 ecr 27241207], length 109
> E.....@.@.9q..........#..n.X\
> .............
> ................get_range_slices..............test.........................................................d.........
> 16:30:55.945152 IP iam.9160 > iam.47134: Flags [.], ack 479, win 256,
> options [nop,nop,TS val 27246226 ecr 27246226], length 0
> E..4..@.@.".........#...\
> ...n.......(.....
> ........
> 16:30:55.949245 IP iam.9160 > iam.47134: Flags [P.], seq 5310:5461, ack
> 479, win 256, options [nop,nop,TS val 27246227 ecr 27246226], length 151
> E.....@.@."V........#...\
> ...n.............
> ....................get_range_slices...................foo..................foo...............foo1.......bar3
> ........&.........foo2.......bar3
> ........: .....
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3446) Problem SliceByNamesReadCommand
on super column family after flush operation
Posted by "Jonas Borgström (Commented JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13147708#comment-13147708 ]
Jonas Borgström commented on CASSANDRA-3446:
--------------------------------------------
I'm not sure it's related at all but CASSANDRA-3466 is about the HintsColumnFamily mysteriously losing or gaining some subcolumns after a hints delivery and corresponding flush.
First assumption was that this was related to this ticket. But after some further testing with a patched 1.0.2 I was still able to reproduce the AssertionError.
So either these two tickets are not related at all or the proposed patches does not fix all cases.
If anyone's interested CASSANDRA-3466 contains an attachment with DEBUG-logs and data files from when I reproduced the issue.
> Problem SliceByNamesReadCommand on super column family after flush operation
> ----------------------------------------------------------------------------
>
> Key: CASSANDRA-3446
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3446
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: Linux iam 3.0.0-12-generic #20-Ubuntu SMP Fri Oct 7 14:56:25 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
> java version "1.6.0_23"
> OpenJDK Runtime Environment (IcedTea6 1.11pre) (6b23~pre10-0ubuntu5)
> OpenJDK 64-Bit Server VM (build 20.0-b11, mixed mode)
> hector-core; 1.0-1
> Reporter: Mike Smith
> Assignee: Jonathan Ellis
> Priority: Critical
> Labels: supercolumns
> Fix For: 1.0.3
>
> Attachments: 0001-fix-addColumn-and-collation-supercolumn-bugs.patch, 0002-r-m-minTimestamp-method.patch, 3446-test.patch
>
>
> I'm having a problem with doing a multiget_slice on a super column family
> after its first flush. Updates to the column values work properly, but
> trying to retrieve the updated values using a multiget_slice operation fail
> to get the updated values. Instead they return the values from before the
> flush. The problem is not apparent with standard column families.
> I've seen this problem in Cassandra v1.0.0 and v1.0.1. The problem
> is not present in Cassandra v0.7.6.
> Steps to reproduce:
> 1. Create one or more super column entries
> 2. Verify the sub column values can be updated and that you can retrieve
> the new values
> 3. Use nodetool to flush the column family or restart cassandra
> 4. Update the sub column values
> 5. Verify they have been updated using cassandra-cli
> 6. Verify you *DO NOT* get the updated values when doing a
> multiget_slice; instead you get the old values from before the flush
> You can get the most recent value by doing a flush followed by a major
> compaction. However, future updates are not retrieved properly either.
> With debug turned on, it looks like the multiget_slice query uses the
> following command/consistency level:
> SliceByNamesReadCommand(table='test_cassandra', key=666f6f,
> columnParent='QueryPath(columnFamilyName='test', superColumnName='null',
> columnName='null')', columns=[foo,])/QUORUM.
> Cassandra-cli uses the following command/consistency level for a get_slice:
> SliceFromReadCommand(table='test_cassandra', key='666f6f',
> column_parent='QueryPath(columnFamilyName='test', superColumnName='null',
> columnName='null')', start='', finish='', reversed=false,
> count=1000000)/QUORUM
> Notice the test program gets 'bar2' for the column values and cassandra-cli
> gets 'bar3' for the column values:
> tcpdump from test program using hector-core:1.0-1
> 16:46:07.424562 IP iam.47158 > iam.9160: Flags [P.], seq 55:138, ack 30,
> win 257, options [nop,nop,TS val 27474096 ecr 27474095], length 83
> E....#@.@.PK.........6#.....].8......{.....
> ..8...8.........multiget_slice................foo..........test................foo.........
> 16:46:07.424575 IP iam.9160 > iam.47158: Flags [.], ack 138, win 256,
> options [nop,nop,TS val 27474096 ecr 27474096], length 0
> E..4..@.@.<.........#..6].8..........(.....
> ..8...8.
> 16:46:07.428771 IP iam.9160 > iam.47158: Flags [P.], seq 30:173, ack 138,
> win 256, options [nop,nop,TS val 27474097 ecr 27474096], length 143
> @.@.<&........#..6].8................
> ............foo...............foo...............foo1.......bar2
> ........6h........foo2.......bar2
> ........I.....
> tcpdump of cassandra-cli:
> 16:30:55.945123 IP iam.47134 > iam.9160: Flags [P.], seq 370:479, ack 5310,
> win 387, options [nop,nop,TS val 27246226 ecr 27241207], length 109
> E.....@.@.9q..........#..n.X\
> .............
> ................get_range_slices..............test.........................................................d.........
> 16:30:55.945152 IP iam.9160 > iam.47134: Flags [.], ack 479, win 256,
> options [nop,nop,TS val 27246226 ecr 27246226], length 0
> E..4..@.@.".........#...\
> ...n.......(.....
> ........
> 16:30:55.949245 IP iam.9160 > iam.47134: Flags [P.], seq 5310:5461, ack
> 479, win 256, options [nop,nop,TS val 27246227 ecr 27246226], length 151
> E.....@.@."V........#...\
> ...n.............
> ....................get_range_slices...................foo..................foo...............foo1.......bar3
> ........&.........foo2.......bar3
> ........: .....
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3446) Problem SliceByNamesReadCommand
on super column family after flush operation
Posted by "Jonathan Ellis (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13142832#comment-13142832 ]
Jonathan Ellis commented on CASSANDRA-3446:
-------------------------------------------
Is this just a single-node cluster?
> Problem SliceByNamesReadCommand on super column family after flush operation
> ----------------------------------------------------------------------------
>
> Key: CASSANDRA-3446
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3446
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0, 1.0.1
> Environment: Linux iam 3.0.0-12-generic #20-Ubuntu SMP Fri Oct 7 14:56:25 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
> java version "1.6.0_23"
> OpenJDK Runtime Environment (IcedTea6 1.11pre) (6b23~pre10-0ubuntu5)
> OpenJDK 64-Bit Server VM (build 20.0-b11, mixed mode)
> hector-core; 1.0-1
> Reporter: Mike Smith
> Fix For: 0.7.6, 0.7.10
>
>
> I'm having a problem with doing a multiget_slice on a super column family
> after its first flush. Updates to the column values work properly, but
> trying to retrieve the updated values using a multiget_slice operation fail
> to get the updated values. Instead they return the values from before the
> flush. The problem is not apparent with standard column families.
> I've seen this problem in Cassandra v1.0.0 and v1.0.1. The problem
> is not present in Cassandra v0.7.6.
> Steps to reproduce:
> 1. Create one or more super column entries
> 2. Verify the sub column values can be updated and that you can retrieve
> the new values
> 3. Use nodetool to flush the column family or restart cassandra
> 4. Update the sub column values
> 5. Verify they have been updated using cassandra-cli
> 6. Verify you *DO NOT* get the updated values when doing a
> multiget_slice; instead you get the old values from before the flush
> You can get the most recent value by doing a flush followed by a major
> compaction. However, future updates are not retrieved properly either.
> With debug turned on, it looks like the multiget_slice query uses the
> following command/consistency level:
> SliceByNamesReadCommand(table='test_cassandra', key=666f6f,
> columnParent='QueryPath(columnFamilyName='test', superColumnName='null',
> columnName='null')', columns=[foo,])/QUORUM.
> Cassandra-cli uses the following command/consistency level for a get_slice:
> SliceFromReadCommand(table='test_cassandra', key='666f6f',
> column_parent='QueryPath(columnFamilyName='test', superColumnName='null',
> columnName='null')', start='', finish='', reversed=false,
> count=1000000)/QUORUM
> Notice the test program gets 'bar2' for the column values and cassandra-cli
> gets 'bar3' for the column values:
> tcpdump from test program using hector-core:1.0-1
> 16:46:07.424562 IP iam.47158 > iam.9160: Flags [P.], seq 55:138, ack 30,
> win 257, options [nop,nop,TS val 27474096 ecr 27474095], length 83
> E....#@.@.PK.........6#.....].8......{.....
> ..8...8.........multiget_slice................foo..........test................foo.........
> 16:46:07.424575 IP iam.9160 > iam.47158: Flags [.], ack 138, win 256,
> options [nop,nop,TS val 27474096 ecr 27474096], length 0
> E..4..@.@.<.........#..6].8..........(.....
> ..8...8.
> 16:46:07.428771 IP iam.9160 > iam.47158: Flags [P.], seq 30:173, ack 138,
> win 256, options [nop,nop,TS val 27474097 ecr 27474096], length 143
> @.@.<&........#..6].8................
> ............foo...............foo...............foo1.......bar2
> ........6h........foo2.......bar2
> ........I.....
> tcpdump of cassandra-cli:
> 16:30:55.945123 IP iam.47134 > iam.9160: Flags [P.], seq 370:479, ack 5310,
> win 387, options [nop,nop,TS val 27246226 ecr 27241207], length 109
> E.....@.@.9q..........#..n.X\
> .............
> ................get_range_slices..............test.........................................................d.........
> 16:30:55.945152 IP iam.9160 > iam.47134: Flags [.], ack 479, win 256,
> options [nop,nop,TS val 27246226 ecr 27246226], length 0
> E..4..@.@.".........#...\
> ...n.......(.....
> ........
> 16:30:55.949245 IP iam.9160 > iam.47134: Flags [P.], seq 5310:5461, ack
> 479, win 256, options [nop,nop,TS val 27246227 ecr 27246226], length 151
> E.....@.@."V........#...\
> ...n.............
> ....................get_range_slices...................foo..................foo...............foo1.......bar3
> ........&.........foo2.......bar3
> ........: .....
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3446) Problem SliceByNamesReadCommand
on super column family after flush operation
Posted by "Jonathan Ellis (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-3446:
--------------------------------------
Priority: Critical (was: Major)
> Problem SliceByNamesReadCommand on super column family after flush operation
> ----------------------------------------------------------------------------
>
> Key: CASSANDRA-3446
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3446
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0
> Environment: Linux iam 3.0.0-12-generic #20-Ubuntu SMP Fri Oct 7 14:56:25 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
> java version "1.6.0_23"
> OpenJDK Runtime Environment (IcedTea6 1.11pre) (6b23~pre10-0ubuntu5)
> OpenJDK 64-Bit Server VM (build 20.0-b11, mixed mode)
> hector-core; 1.0-1
> Reporter: Mike Smith
> Assignee: Jonathan Ellis
> Priority: Critical
> Labels: supercolumns
> Fix For: 1.0.3
>
> Attachments: 0001-fix-addColumn-and-collation-supercolumn-bugs.patch, 0002-r-m-minTimestamp-method.patch, 3446-test.patch
>
>
> I'm having a problem with doing a multiget_slice on a super column family
> after its first flush. Updates to the column values work properly, but
> trying to retrieve the updated values using a multiget_slice operation fail
> to get the updated values. Instead they return the values from before the
> flush. The problem is not apparent with standard column families.
> I've seen this problem in Cassandra v1.0.0 and v1.0.1. The problem
> is not present in Cassandra v0.7.6.
> Steps to reproduce:
> 1. Create one or more super column entries
> 2. Verify the sub column values can be updated and that you can retrieve
> the new values
> 3. Use nodetool to flush the column family or restart cassandra
> 4. Update the sub column values
> 5. Verify they have been updated using cassandra-cli
> 6. Verify you *DO NOT* get the updated values when doing a
> multiget_slice; instead you get the old values from before the flush
> You can get the most recent value by doing a flush followed by a major
> compaction. However, future updates are not retrieved properly either.
> With debug turned on, it looks like the multiget_slice query uses the
> following command/consistency level:
> SliceByNamesReadCommand(table='test_cassandra', key=666f6f,
> columnParent='QueryPath(columnFamilyName='test', superColumnName='null',
> columnName='null')', columns=[foo,])/QUORUM.
> Cassandra-cli uses the following command/consistency level for a get_slice:
> SliceFromReadCommand(table='test_cassandra', key='666f6f',
> column_parent='QueryPath(columnFamilyName='test', superColumnName='null',
> columnName='null')', start='', finish='', reversed=false,
> count=1000000)/QUORUM
> Notice the test program gets 'bar2' for the column values and cassandra-cli
> gets 'bar3' for the column values:
> tcpdump from test program using hector-core:1.0-1
> 16:46:07.424562 IP iam.47158 > iam.9160: Flags [P.], seq 55:138, ack 30,
> win 257, options [nop,nop,TS val 27474096 ecr 27474095], length 83
> E....#@.@.PK.........6#.....].8......{.....
> ..8...8.........multiget_slice................foo..........test................foo.........
> 16:46:07.424575 IP iam.9160 > iam.47158: Flags [.], ack 138, win 256,
> options [nop,nop,TS val 27474096 ecr 27474096], length 0
> E..4..@.@.<.........#..6].8..........(.....
> ..8...8.
> 16:46:07.428771 IP iam.9160 > iam.47158: Flags [P.], seq 30:173, ack 138,
> win 256, options [nop,nop,TS val 27474097 ecr 27474096], length 143
> @.@.<&........#..6].8................
> ............foo...............foo...............foo1.......bar2
> ........6h........foo2.......bar2
> ........I.....
> tcpdump of cassandra-cli:
> 16:30:55.945123 IP iam.47134 > iam.9160: Flags [P.], seq 370:479, ack 5310,
> win 387, options [nop,nop,TS val 27246226 ecr 27241207], length 109
> E.....@.@.9q..........#..n.X\
> .............
> ................get_range_slices..............test.........................................................d.........
> 16:30:55.945152 IP iam.9160 > iam.47134: Flags [.], ack 479, win 256,
> options [nop,nop,TS val 27246226 ecr 27246226], length 0
> E..4..@.@.".........#...\
> ...n.......(.....
> ........
> 16:30:55.949245 IP iam.9160 > iam.47134: Flags [P.], seq 5310:5461, ack
> 479, win 256, options [nop,nop,TS val 27246227 ecr 27246226], length 151
> E.....@.@."V........#...\
> ...n.............
> ....................get_range_slices...................foo..................foo...............foo1.......bar3
> ........&.........foo2.......bar3
> ........: .....
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3446) Problem SliceByNamesReadCommand
on super column family after flush operation
Posted by "Jonathan Ellis (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13145705#comment-13145705 ]
Jonathan Ellis commented on CASSANDRA-3446:
-------------------------------------------
There's at least two problems here. One is the one Nicola describes. The other is that the name-based path in CollationController stops as soon as it finds one subcolumn in a given supercolumn.
Working on a patch for both.
> Problem SliceByNamesReadCommand on super column family after flush operation
> ----------------------------------------------------------------------------
>
> Key: CASSANDRA-3446
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3446
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.0, 1.0.1
> Environment: Linux iam 3.0.0-12-generic #20-Ubuntu SMP Fri Oct 7 14:56:25 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
> java version "1.6.0_23"
> OpenJDK Runtime Environment (IcedTea6 1.11pre) (6b23~pre10-0ubuntu5)
> OpenJDK 64-Bit Server VM (build 20.0-b11, mixed mode)
> hector-core; 1.0-1
> Reporter: Mike Smith
> Fix For: 0.7.6, 0.7.10
>
> Attachments: 3446-test.patch
>
>
> I'm having a problem with doing a multiget_slice on a super column family
> after its first flush. Updates to the column values work properly, but
> trying to retrieve the updated values using a multiget_slice operation fail
> to get the updated values. Instead they return the values from before the
> flush. The problem is not apparent with standard column families.
> I've seen this problem in Cassandra v1.0.0 and v1.0.1. The problem
> is not present in Cassandra v0.7.6.
> Steps to reproduce:
> 1. Create one or more super column entries
> 2. Verify the sub column values can be updated and that you can retrieve
> the new values
> 3. Use nodetool to flush the column family or restart cassandra
> 4. Update the sub column values
> 5. Verify they have been updated using cassandra-cli
> 6. Verify you *DO NOT* get the updated values when doing a
> multiget_slice; instead you get the old values from before the flush
> You can get the most recent value by doing a flush followed by a major
> compaction. However, future updates are not retrieved properly either.
> With debug turned on, it looks like the multiget_slice query uses the
> following command/consistency level:
> SliceByNamesReadCommand(table='test_cassandra', key=666f6f,
> columnParent='QueryPath(columnFamilyName='test', superColumnName='null',
> columnName='null')', columns=[foo,])/QUORUM.
> Cassandra-cli uses the following command/consistency level for a get_slice:
> SliceFromReadCommand(table='test_cassandra', key='666f6f',
> column_parent='QueryPath(columnFamilyName='test', superColumnName='null',
> columnName='null')', start='', finish='', reversed=false,
> count=1000000)/QUORUM
> Notice the test program gets 'bar2' for the column values and cassandra-cli
> gets 'bar3' for the column values:
> tcpdump from test program using hector-core:1.0-1
> 16:46:07.424562 IP iam.47158 > iam.9160: Flags [P.], seq 55:138, ack 30,
> win 257, options [nop,nop,TS val 27474096 ecr 27474095], length 83
> E....#@.@.PK.........6#.....].8......{.....
> ..8...8.........multiget_slice................foo..........test................foo.........
> 16:46:07.424575 IP iam.9160 > iam.47158: Flags [.], ack 138, win 256,
> options [nop,nop,TS val 27474096 ecr 27474096], length 0
> E..4..@.@.<.........#..6].8..........(.....
> ..8...8.
> 16:46:07.428771 IP iam.9160 > iam.47158: Flags [P.], seq 30:173, ack 138,
> win 256, options [nop,nop,TS val 27474097 ecr 27474096], length 143
> @.@.<&........#..6].8................
> ............foo...............foo...............foo1.......bar2
> ........6h........foo2.......bar2
> ........I.....
> tcpdump of cassandra-cli:
> 16:30:55.945123 IP iam.47134 > iam.9160: Flags [P.], seq 370:479, ack 5310,
> win 387, options [nop,nop,TS val 27246226 ecr 27241207], length 109
> E.....@.@.9q..........#..n.X\
> .............
> ................get_range_slices..............test.........................................................d.........
> 16:30:55.945152 IP iam.9160 > iam.47134: Flags [.], ack 479, win 256,
> options [nop,nop,TS val 27246226 ecr 27246226], length 0
> E..4..@.@.".........#...\
> ...n.......(.....
> ........
> 16:30:55.949245 IP iam.9160 > iam.47134: Flags [P.], seq 5310:5461, ack
> 479, win 256, options [nop,nop,TS val 27246227 ecr 27246226], length 151
> E.....@.@."V........#...\
> ...n.............
> ....................get_range_slices...................foo..................foo...............foo1.......bar3
> ........&.........foo2.......bar3
> ........: .....
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira