You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Arya Goudarzi (JIRA)" <ji...@apache.org> on 2012/10/10 02:14:02 UTC
[jira] [Created] (CASSANDRA-4785) Secondary Index Sporadically
Doesn't Return Rows
Arya Goudarzi created CASSANDRA-4785:
----------------------------------------
Summary: Secondary Index Sporadically Doesn't Return Rows
Key: CASSANDRA-4785
URL: https://issues.apache.org/jira/browse/CASSANDRA-4785
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.1.5
Environment: Ubuntu 10.04
Java 6 Sun
Cassandra 1.1.5 upgraded from 1.1.2 -> 1.1.3 -> 1.1.5
Reporter: Arya Goudarzi
I have a ColumnFamily with caching = ALL. I have 2 secondary indexes on it. I have noticed if I query using the secondary index in the where clause, sometimes I get the results and sometimes I don't. Until 2 weeks ago, the caching option on this CF was set to NONE. So, I suspect something happened in secondary index caching scheme.
Here are things I tried:
1. I rebuild indexes for that CF on all nodes;
2. I set the caching to KEYS_ONLY and rebuild the index again;
3. I set the caching to NONE and rebuild the index again;
None of the above helped. I suppose the caching still exists as this behavior looks like cache mistmatch.
I did a bit research, and found CASSANDRA-4197 that could be related.
Please advice.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4785) Secondary Index Sporadically
Doesn't Return Rows
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472920#comment-13472920 ]
Brandon Williams commented on CASSANDRA-4785:
---------------------------------------------
If you do a fresh start with caching at NONE does the issue repro?
> Secondary Index Sporadically Doesn't Return Rows
> ------------------------------------------------
>
> Key: CASSANDRA-4785
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4785
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.1.5
> Environment: Ubuntu 10.04
> Java 6 Sun
> Cassandra 1.1.5 upgraded from 1.1.2 -> 1.1.3 -> 1.1.5
> Reporter: Arya Goudarzi
>
> I have a ColumnFamily with caching = ALL. I have 2 secondary indexes on it. I have noticed if I query using the secondary index in the where clause, sometimes I get the results and sometimes I don't. Until 2 weeks ago, the caching option on this CF was set to NONE. So, I suspect something happened in secondary index caching scheme.
> Here are things I tried:
> 1. I rebuild indexes for that CF on all nodes;
> 2. I set the caching to KEYS_ONLY and rebuild the index again;
> 3. I set the caching to NONE and rebuild the index again;
> None of the above helped. I suppose the caching still exists as this behavior looks like cache mistmatch.
> I did a bit research, and found CASSANDRA-4197 that could be related.
> Please advice.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4785) Secondary Index Sporadically
Doesn't Return Rows
Posted by "Arya Goudarzi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475466#comment-13475466 ]
Arya Goudarzi commented on CASSANDRA-4785:
------------------------------------------
If you mean clean slate installation, I have not done that, and I won't get any time soon to try it. But, let say I could not repro, then what should I do in my live cluster to take it out of this state?
> Secondary Index Sporadically Doesn't Return Rows
> ------------------------------------------------
>
> Key: CASSANDRA-4785
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4785
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.1.5
> Environment: Ubuntu 10.04
> Java 6 Sun
> Cassandra 1.1.5 upgraded from 1.1.2 -> 1.1.3 -> 1.1.5
> Reporter: Arya Goudarzi
>
> I have a ColumnFamily with caching = ALL. I have 2 secondary indexes on it. I have noticed if I query using the secondary index in the where clause, sometimes I get the results and sometimes I don't. Until 2 weeks ago, the caching option on this CF was set to NONE. So, I suspect something happened in secondary index caching scheme.
> Here are things I tried:
> 1. I rebuild indexes for that CF on all nodes;
> 2. I set the caching to KEYS_ONLY and rebuild the index again;
> 3. I set the caching to NONE and rebuild the index again;
> None of the above helped. I suppose the caching still exists as this behavior looks like cache mistmatch.
> I did a bit research, and found CASSANDRA-4197 that could be related.
> Please advice.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4785) Secondary Index Sporadically
Doesn't Return Rows
Posted by "Arya Goudarzi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13482922#comment-13482922 ]
Arya Goudarzi commented on CASSANDRA-4785:
------------------------------------------
wouldn't nodetool invalidatekeycache and invalidaterowcache do it?
> Secondary Index Sporadically Doesn't Return Rows
> ------------------------------------------------
>
> Key: CASSANDRA-4785
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4785
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.1.5
> Environment: Ubuntu 10.04
> Java 6 Sun
> Cassandra 1.1.5 upgraded from 1.1.2 -> 1.1.3 -> 1.1.5
> Reporter: Arya Goudarzi
>
> I have a ColumnFamily with caching = ALL. I have 2 secondary indexes on it. I have noticed if I query using the secondary index in the where clause, sometimes I get the results and sometimes I don't. Until 2 weeks ago, the caching option on this CF was set to NONE. So, I suspect something happened in secondary index caching scheme.
> Here are things I tried:
> 1. I rebuild indexes for that CF on all nodes;
> 2. I set the caching to KEYS_ONLY and rebuild the index again;
> 3. I set the caching to NONE and rebuild the index again;
> None of the above helped. I suppose the caching still exists as this behavior looks like cache mistmatch.
> I did a bit research, and found CASSANDRA-4197 that could be related.
> Please advice.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4785) Secondary Index Sporadically
Doesn't Return Rows
Posted by "Arya Goudarzi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13483844#comment-13483844 ]
Arya Goudarzi commented on CASSANDRA-4785:
------------------------------------------
OK, I did disable the cache and restarted the nodes. It worked fine. Right after I enabled key cache for the CF in question, I started getting mixed numbers when querying with secondary indexes.
> Secondary Index Sporadically Doesn't Return Rows
> ------------------------------------------------
>
> Key: CASSANDRA-4785
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4785
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.1.5, 1.1.6
> Environment: Ubuntu 10.04
> Java 6 Sun
> Cassandra 1.1.5 upgraded from 1.1.2 -> 1.1.3 -> 1.1.5
> Reporter: Arya Goudarzi
>
> I have a ColumnFamily with caching = ALL. I have 2 secondary indexes on it. I have noticed if I query using the secondary index in the where clause, sometimes I get the results and sometimes I don't. Until 2 weeks ago, the caching option on this CF was set to NONE. So, I suspect something happened in secondary index caching scheme.
> Here are things I tried:
> 1. I rebuild indexes for that CF on all nodes;
> 2. I set the caching to KEYS_ONLY and rebuild the index again;
> 3. I set the caching to NONE and rebuild the index again;
> None of the above helped. I suppose the caching still exists as this behavior looks like cache mistmatch.
> I did a bit research, and found CASSANDRA-4197 that could be related.
> Please advice.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4785) Secondary Index Sporadically
Doesn't Return Rows
Posted by "Arya Goudarzi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13482200#comment-13482200 ]
Arya Goudarzi commented on CASSANDRA-4785:
------------------------------------------
Hey Brandon,
I tried the clean cluster of 1.1.5 and 1.1.6 several times today and ran our unit tests and I cannot reproduce this issue. However, it is reproducible on my existing production cluster.
Please advise.
-Arya
> Secondary Index Sporadically Doesn't Return Rows
> ------------------------------------------------
>
> Key: CASSANDRA-4785
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4785
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.1.5
> Environment: Ubuntu 10.04
> Java 6 Sun
> Cassandra 1.1.5 upgraded from 1.1.2 -> 1.1.3 -> 1.1.5
> Reporter: Arya Goudarzi
>
> I have a ColumnFamily with caching = ALL. I have 2 secondary indexes on it. I have noticed if I query using the secondary index in the where clause, sometimes I get the results and sometimes I don't. Until 2 weeks ago, the caching option on this CF was set to NONE. So, I suspect something happened in secondary index caching scheme.
> Here are things I tried:
> 1. I rebuild indexes for that CF on all nodes;
> 2. I set the caching to KEYS_ONLY and rebuild the index again;
> 3. I set the caching to NONE and rebuild the index again;
> None of the above helped. I suppose the caching still exists as this behavior looks like cache mistmatch.
> I did a bit research, and found CASSANDRA-4197 that could be related.
> Please advice.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4785) Secondary Index Sporadically
Doesn't Return Rows
Posted by "Arya Goudarzi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472913#comment-13472913 ]
Arya Goudarzi commented on CASSANDRA-4785:
------------------------------------------
BTW, I tried my tests in CLI. So, there should not be my code issue.
> Secondary Index Sporadically Doesn't Return Rows
> ------------------------------------------------
>
> Key: CASSANDRA-4785
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4785
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.1.5
> Environment: Ubuntu 10.04
> Java 6 Sun
> Cassandra 1.1.5 upgraded from 1.1.2 -> 1.1.3 -> 1.1.5
> Reporter: Arya Goudarzi
>
> I have a ColumnFamily with caching = ALL. I have 2 secondary indexes on it. I have noticed if I query using the secondary index in the where clause, sometimes I get the results and sometimes I don't. Until 2 weeks ago, the caching option on this CF was set to NONE. So, I suspect something happened in secondary index caching scheme.
> Here are things I tried:
> 1. I rebuild indexes for that CF on all nodes;
> 2. I set the caching to KEYS_ONLY and rebuild the index again;
> 3. I set the caching to NONE and rebuild the index again;
> None of the above helped. I suppose the caching still exists as this behavior looks like cache mistmatch.
> I did a bit research, and found CASSANDRA-4197 that could be related.
> Please advice.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4785) Secondary Index Sporadically
Doesn't Return Rows
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13482254#comment-13482254 ]
Brandon Williams commented on CASSANDRA-4785:
---------------------------------------------
I would set caching to NONE, then doing a rolling restart of the nodes to ensure the cache is empty. If that does not help, I would then rebuild the indexes one more time with the cache empty.
> Secondary Index Sporadically Doesn't Return Rows
> ------------------------------------------------
>
> Key: CASSANDRA-4785
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4785
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.1.5
> Environment: Ubuntu 10.04
> Java 6 Sun
> Cassandra 1.1.5 upgraded from 1.1.2 -> 1.1.3 -> 1.1.5
> Reporter: Arya Goudarzi
>
> I have a ColumnFamily with caching = ALL. I have 2 secondary indexes on it. I have noticed if I query using the secondary index in the where clause, sometimes I get the results and sometimes I don't. Until 2 weeks ago, the caching option on this CF was set to NONE. So, I suspect something happened in secondary index caching scheme.
> Here are things I tried:
> 1. I rebuild indexes for that CF on all nodes;
> 2. I set the caching to KEYS_ONLY and rebuild the index again;
> 3. I set the caching to NONE and rebuild the index again;
> None of the above helped. I suppose the caching still exists as this behavior looks like cache mistmatch.
> I did a bit research, and found CASSANDRA-4197 that could be related.
> Please advice.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4785) Secondary Index Sporadically
Doesn't Return Rows
Posted by "Arya Goudarzi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arya Goudarzi updated CASSANDRA-4785:
-------------------------------------
Affects Version/s: 1.1.6
> Secondary Index Sporadically Doesn't Return Rows
> ------------------------------------------------
>
> Key: CASSANDRA-4785
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4785
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.1.5, 1.1.6
> Environment: Ubuntu 10.04
> Java 6 Sun
> Cassandra 1.1.5 upgraded from 1.1.2 -> 1.1.3 -> 1.1.5
> Reporter: Arya Goudarzi
>
> I have a ColumnFamily with caching = ALL. I have 2 secondary indexes on it. I have noticed if I query using the secondary index in the where clause, sometimes I get the results and sometimes I don't. Until 2 weeks ago, the caching option on this CF was set to NONE. So, I suspect something happened in secondary index caching scheme.
> Here are things I tried:
> 1. I rebuild indexes for that CF on all nodes;
> 2. I set the caching to KEYS_ONLY and rebuild the index again;
> 3. I set the caching to NONE and rebuild the index again;
> None of the above helped. I suppose the caching still exists as this behavior looks like cache mistmatch.
> I did a bit research, and found CASSANDRA-4197 that could be related.
> Please advice.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira