You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Jonathan Ellis (JIRA)" <ji...@apache.org> on 2010/08/20 20:44:18 UTC
[jira] Created: (CASSANDRA-1415) Allow creating indexes on existing
data
Allow creating indexes on existing data
---------------------------------------
Key: CASSANDRA-1415
URL: https://issues.apache.org/jira/browse/CASSANDRA-1415
Project: Cassandra
Issue Type: New Feature
Components: Core
Affects Versions: 0.7 beta 1
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Fix For: 0.7 beta 2
(This ticket is only for the index-CF creation part, not for the thrift API changes, which may come for free from CASSANDRA-1285)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1415) Allow creating indexes on
existing data
Posted by "Gary Dusbabek (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12912555#action_12912555 ]
Gary Dusbabek commented on CASSANDRA-1415:
------------------------------------------
Things have changed enough over the weekend so that 0004 fails to apply. Can you rebase?
> Allow creating indexes on existing data
> ---------------------------------------
>
> Key: CASSANDRA-1415
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1415
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Affects Versions: 0.7 beta 1
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.7 beta 2
>
> Attachments: 0001-clean-up-FileStatus-and-rename-to-StreamReply.-updates.txt, 0002-add-SESSION_FINISHED-reply.txt, 0003-Table.rebuildIndex.txt, 0004-extract-addIndex-method.txt, 0005-allow-addIndex-to-create-indexes-that-did-not-previous.txt, 0006-split-2ary-index-build-out-from-bloom-row-index-build-.txt, 0007-remove-IKeyIterator-and-move-ICompactionInfo-implement.txt, 0008-bug-fixes.txt
>
>
> (This ticket is only for the index-CF creation part, not for the thrift API changes, which may come for free from CASSANDRA-1285)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1415) Allow creating indexes on existing
data
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-1415:
--------------------------------------
Attachment: 0001-clean-up-FileStatus-and-rename-to-StreamReply.-updates.txt
0002-add-SESSION_FINISHED-reply.txt
0003-Table.rebuildIndex.txt
0004-extract-addIndex-method.txt
0005-allow-addIndex-to-create-indexes-that-did-not-previous.txt
0006-split-2ary-index-build-out-from-bloom-row-index-build-.txt
0007-remove-IKeyIterator-and-move-ICompactionInfo-implement.txt
0008-bug-fixes.txt
> Allow creating indexes on existing data
> ---------------------------------------
>
> Key: CASSANDRA-1415
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1415
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Affects Versions: 0.7 beta 1
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.7 beta 2
>
> Attachments: 0001-clean-up-FileStatus-and-rename-to-StreamReply.-updates.txt, 0002-add-SESSION_FINISHED-reply.txt, 0003-Table.rebuildIndex.txt, 0004-extract-addIndex-method.txt, 0005-allow-addIndex-to-create-indexes-that-did-not-previous.txt, 0006-split-2ary-index-build-out-from-bloom-row-index-build-.txt, 0007-remove-IKeyIterator-and-move-ICompactionInfo-implement.txt, 0008-bug-fixes.txt
>
>
> (This ticket is only for the index-CF creation part, not for the thrift API changes, which may come for free from CASSANDRA-1285)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1415) Allow creating indexes on
existing data
Posted by "Gary Dusbabek (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12902921#action_12902921 ]
Gary Dusbabek commented on CASSANDRA-1415:
------------------------------------------
01: it looks like the old version built the index by reading straight from the data file. The new way expects the CFS to be populated with sstables, which isn't always going to be the case (if you're recovering from a descriptor that points to a db that the CFS isn't going to load automatically).
> Allow creating indexes on existing data
> ---------------------------------------
>
> Key: CASSANDRA-1415
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1415
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Affects Versions: 0.7 beta 1
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.7 beta 2
>
> Attachments: 0001-Table.rebuildIndex.txt, 0002-extract-addIndex-method.txt, 0003-allow-addIndex-to-create-indexes-that-did-not-previous.txt
>
>
> (This ticket is only for the index-CF creation part, not for the thrift API changes, which may come for free from CASSANDRA-1285)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1415) Allow creating indexes on
existing data
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12910676#action_12910676 ]
Jonathan Ellis commented on CASSANDRA-1415:
-------------------------------------------
New patchset:
01 is refactoring to set up 02
02 adds SESSION_FINISHED reply so bootstrap target has time to do post-streaming 2ary index building before source declares it finished
03 - 05 are old 01 - 03
06 split 2ary index build out from bloom/row index build, and move into stream session post-process that we set up in 02. also, bloom/row index construction moved into SSTableWriter.Builder and is now run on CompactionManager
07 is cleanup of KeyIterator code
08 is bug fixes:
- index writes that happen during index-build process
- fix circular Table.open during index creation by passing Table object to CFS instead of String
- avoid writing empty index sstables
> Allow creating indexes on existing data
> ---------------------------------------
>
> Key: CASSANDRA-1415
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1415
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Affects Versions: 0.7 beta 1
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.7 beta 2
>
> Attachments: 0001-clean-up-FileStatus-and-rename-to-StreamReply.-updates.txt, 0002-add-SESSION_FINISHED-reply.txt, 0003-Table.rebuildIndex.txt, 0004-extract-addIndex-method.txt, 0005-allow-addIndex-to-create-indexes-that-did-not-previous.txt, 0006-split-2ary-index-build-out-from-bloom-row-index-build-.txt, 0007-remove-IKeyIterator-and-move-ICompactionInfo-implement.txt, 0008-bug-fixes.txt
>
>
> (This ticket is only for the index-CF creation part, not for the thrift API changes, which may come for free from CASSANDRA-1285)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1415) Allow creating indexes on existing
data
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-1415:
--------------------------------------
Attachment: (was: 0001-clean-up-FileStatus-and-rename-to-StreamReply.-updates.txt)
> Allow creating indexes on existing data
> ---------------------------------------
>
> Key: CASSANDRA-1415
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1415
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Affects Versions: 0.7 beta 1
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.7 beta 2
>
> Attachments: 0003-Table.rebuildIndex.txt, 0004-extract-addIndex-method.txt, 0005-allow-addIndex-to-create-indexes-that-did-not-previous.txt, 0006-split-2ary-index-build-out-from-bloom-row-index-build-.txt, 0007-remove-IKeyIterator-and-move-ICompactionInfo-implement.txt, 0008-bug-fixes.txt
>
>
> (This ticket is only for the index-CF creation part, not for the thrift API changes, which may come for free from CASSANDRA-1285)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1415) Allow creating indexes on existing
data
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-1415:
--------------------------------------
Attachment: 0001-Table.rebuildIndex.txt
0002-extract-addIndex-method.txt
0003-allow-addIndex-to-create-indexes-that-did-not-previous.txt
> Allow creating indexes on existing data
> ---------------------------------------
>
> Key: CASSANDRA-1415
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1415
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Affects Versions: 0.7 beta 1
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.7 beta 2
>
> Attachments: 0001-Table.rebuildIndex.txt, 0002-extract-addIndex-method.txt, 0003-allow-addIndex-to-create-indexes-that-did-not-previous.txt
>
>
> (This ticket is only for the index-CF creation part, not for the thrift API changes, which may come for free from CASSANDRA-1285)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1415) Allow creating indexes on
existing data
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12902485#action_12902485 ]
Jonathan Ellis commented on CASSANDRA-1415:
-------------------------------------------
01 changes the index creation approach used when building the index from the data file after streaming. the old approach scans through the new data file and applies the data in those row fragments to the index, but this is buggy: if the data file contains old data, or newer data arrives via a write after the file is streamed, we will corrupt our index. instead, 01 creates a Table.rebuildIndex method that scans the index columns from the _entire_ row (that is, merged w/ other sstables), with locking so that when we scan that row, we're guaranteed that it's the most recent version. The KeyIterator class is what we use to tell this scanner what keys to index. (We don't use getRangeSlice because the streamed rows aren't guaranteed to be contiguous.) The rebuild is done in the CompactionManager executor to avoid excessive i/o conflicts.
02 is a simple refactor, pulling addIndex out of CFS constructor.
03 extends addIndex to adding indexes that weren't previously defined. we add INDEX_CF to record when an index is created; if addIndex is called for one that doesn't exist, we initiate the build using a KeyIterator that merges all keys from all sstables. This makes sure that if we add an index, and the index build doesn't complete before the server is restarted, the build will restart too.
> Allow creating indexes on existing data
> ---------------------------------------
>
> Key: CASSANDRA-1415
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1415
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Affects Versions: 0.7 beta 1
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.7 beta 2
>
> Attachments: 0001-Table.rebuildIndex.txt, 0002-extract-addIndex-method.txt, 0003-allow-addIndex-to-create-indexes-that-did-not-previous.txt
>
>
> (This ticket is only for the index-CF creation part, not for the thrift API changes, which may come for free from CASSANDRA-1285)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1415) Allow creating indexes on
existing data
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12912751#action_12912751 ]
Jonathan Ellis commented on CASSANDRA-1415:
-------------------------------------------
rebased
> Allow creating indexes on existing data
> ---------------------------------------
>
> Key: CASSANDRA-1415
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1415
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Affects Versions: 0.7 beta 1
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.7 beta 2
>
> Attachments: 0001-clean-up-FileStatus-and-rename-to-StreamReply.-updates.txt, 0002-add-SESSION_FINISHED-reply.txt, 0003-Table.rebuildIndex.txt, 0004-extract-addIndex-method.txt, 0005-allow-addIndex-to-create-indexes-that-did-not-previous.txt, 0006-split-2ary-index-build-out-from-bloom-row-index-build-.txt, 0007-remove-IKeyIterator-and-move-ICompactionInfo-implement.txt, 0008-bug-fixes.txt
>
>
> (This ticket is only for the index-CF creation part, not for the thrift API changes, which may come for free from CASSANDRA-1285)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1415) Allow creating indexes on
existing data
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12910680#action_12910680 ]
Jonathan Ellis commented on CASSANDRA-1415:
-------------------------------------------
note: requires CASSANDRA-1512 to pass test suite.
> Allow creating indexes on existing data
> ---------------------------------------
>
> Key: CASSANDRA-1415
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1415
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Affects Versions: 0.7 beta 1
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.7 beta 2
>
> Attachments: 0001-clean-up-FileStatus-and-rename-to-StreamReply.-updates.txt, 0002-add-SESSION_FINISHED-reply.txt, 0003-Table.rebuildIndex.txt, 0004-extract-addIndex-method.txt, 0005-allow-addIndex-to-create-indexes-that-did-not-previous.txt, 0006-split-2ary-index-build-out-from-bloom-row-index-build-.txt, 0007-remove-IKeyIterator-and-move-ICompactionInfo-implement.txt, 0008-bug-fixes.txt
>
>
> (This ticket is only for the index-CF creation part, not for the thrift API changes, which may come for free from CASSANDRA-1285)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1415) Allow creating indexes on existing
data
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-1415:
--------------------------------------
Attachment: (was: 0002-add-SESSION_FINISHED-reply.txt)
> Allow creating indexes on existing data
> ---------------------------------------
>
> Key: CASSANDRA-1415
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1415
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Affects Versions: 0.7 beta 1
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.7 beta 2
>
> Attachments: 0003-Table.rebuildIndex.txt, 0004-extract-addIndex-method.txt, 0005-allow-addIndex-to-create-indexes-that-did-not-previous.txt, 0006-split-2ary-index-build-out-from-bloom-row-index-build-.txt, 0007-remove-IKeyIterator-and-move-ICompactionInfo-implement.txt, 0008-bug-fixes.txt
>
>
> (This ticket is only for the index-CF creation part, not for the thrift API changes, which may come for free from CASSANDRA-1285)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1415) Allow creating indexes on existing
data
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-1415:
--------------------------------------
Attachment: (was: 0002-extract-addIndex-method.txt)
> Allow creating indexes on existing data
> ---------------------------------------
>
> Key: CASSANDRA-1415
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1415
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Affects Versions: 0.7 beta 1
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.7 beta 2
>
> Attachments: 0001-clean-up-FileStatus-and-rename-to-StreamReply.-updates.txt, 0002-add-SESSION_FINISHED-reply.txt, 0003-Table.rebuildIndex.txt, 0004-extract-addIndex-method.txt, 0005-allow-addIndex-to-create-indexes-that-did-not-previous.txt, 0006-split-2ary-index-build-out-from-bloom-row-index-build-.txt, 0007-remove-IKeyIterator-and-move-ICompactionInfo-implement.txt, 0008-bug-fixes.txt
>
>
> (This ticket is only for the index-CF creation part, not for the thrift API changes, which may come for free from CASSANDRA-1285)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1415) Allow creating indexes on existing
data
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-1415:
--------------------------------------
Attachment: (was: 0003-allow-addIndex-to-create-indexes-that-did-not-previous.txt)
> Allow creating indexes on existing data
> ---------------------------------------
>
> Key: CASSANDRA-1415
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1415
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Affects Versions: 0.7 beta 1
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.7 beta 2
>
> Attachments: 0001-clean-up-FileStatus-and-rename-to-StreamReply.-updates.txt, 0002-add-SESSION_FINISHED-reply.txt, 0003-Table.rebuildIndex.txt, 0004-extract-addIndex-method.txt, 0005-allow-addIndex-to-create-indexes-that-did-not-previous.txt, 0006-split-2ary-index-build-out-from-bloom-row-index-build-.txt, 0007-remove-IKeyIterator-and-move-ICompactionInfo-implement.txt, 0008-bug-fixes.txt
>
>
> (This ticket is only for the index-CF creation part, not for the thrift API changes, which may come for free from CASSANDRA-1285)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1415) Allow creating indexes on existing
data
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-1415:
--------------------------------------
Attachment: (was: 0008-bug-fixes.txt)
> Allow creating indexes on existing data
> ---------------------------------------
>
> Key: CASSANDRA-1415
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1415
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Affects Versions: 0.7 beta 1
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.7 beta 2
>
>
> (This ticket is only for the index-CF creation part, not for the thrift API changes, which may come for free from CASSANDRA-1285)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1415) Allow creating indexes on existing
data
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-1415:
--------------------------------------
Attachment: (was: 0005-allow-addIndex-to-create-indexes-that-did-not-previous.txt)
> Allow creating indexes on existing data
> ---------------------------------------
>
> Key: CASSANDRA-1415
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1415
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Affects Versions: 0.7 beta 1
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.7 beta 2
>
>
> (This ticket is only for the index-CF creation part, not for the thrift API changes, which may come for free from CASSANDRA-1285)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1415) Allow creating indexes on existing
data
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-1415:
--------------------------------------
Attachment: (was: 0006-split-2ary-index-build-out-from-bloom-row-index-build-.txt)
> Allow creating indexes on existing data
> ---------------------------------------
>
> Key: CASSANDRA-1415
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1415
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Affects Versions: 0.7 beta 1
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.7 beta 2
>
>
> (This ticket is only for the index-CF creation part, not for the thrift API changes, which may come for free from CASSANDRA-1285)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1415) Allow creating indexes on existing
data
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-1415:
--------------------------------------
Attachment: (was: 0004-extract-addIndex-method.txt)
> Allow creating indexes on existing data
> ---------------------------------------
>
> Key: CASSANDRA-1415
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1415
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Affects Versions: 0.7 beta 1
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.7 beta 2
>
>
> (This ticket is only for the index-CF creation part, not for the thrift API changes, which may come for free from CASSANDRA-1285)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1415) Allow creating indexes on existing
data
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-1415:
--------------------------------------
Attachment: (was: 0001-Table.rebuildIndex.txt)
> Allow creating indexes on existing data
> ---------------------------------------
>
> Key: CASSANDRA-1415
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1415
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Affects Versions: 0.7 beta 1
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.7 beta 2
>
> Attachments: 0001-clean-up-FileStatus-and-rename-to-StreamReply.-updates.txt, 0002-add-SESSION_FINISHED-reply.txt, 0003-Table.rebuildIndex.txt, 0004-extract-addIndex-method.txt, 0005-allow-addIndex-to-create-indexes-that-did-not-previous.txt, 0006-split-2ary-index-build-out-from-bloom-row-index-build-.txt, 0007-remove-IKeyIterator-and-move-ICompactionInfo-implement.txt, 0008-bug-fixes.txt
>
>
> (This ticket is only for the index-CF creation part, not for the thrift API changes, which may come for free from CASSANDRA-1285)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1415) Allow creating indexes on existing
data
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-1415:
--------------------------------------
Attachment: 0001-clean-up-FileStatus-and-rename-to-StreamReply.-updates.txt
0002-add-SESSION_FINISHED-reply.txt
0003-Table.rebuildIndex.txt
0004-extract-addIndex-method.txt
0005-allow-addIndex-to-create-indexes-that-did-not-previous.txt
0006-split-2ary-index-build-out-from-bloom-row-index-build-.txt
0007-remove-IKeyIterator-and-move-ICompactionInfo-implement.txt
0008-bug-fixes.txt
> Allow creating indexes on existing data
> ---------------------------------------
>
> Key: CASSANDRA-1415
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1415
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Affects Versions: 0.7 beta 1
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.7 beta 2
>
> Attachments: 0001-clean-up-FileStatus-and-rename-to-StreamReply.-updates.txt, 0002-add-SESSION_FINISHED-reply.txt, 0003-Table.rebuildIndex.txt, 0004-extract-addIndex-method.txt, 0005-allow-addIndex-to-create-indexes-that-did-not-previous.txt, 0006-split-2ary-index-build-out-from-bloom-row-index-build-.txt, 0007-remove-IKeyIterator-and-move-ICompactionInfo-implement.txt, 0008-bug-fixes.txt
>
>
> (This ticket is only for the index-CF creation part, not for the thrift API changes, which may come for free from CASSANDRA-1285)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (CASSANDRA-1415) Allow creating
indexes on existing data
Posted by "Gary Dusbabek (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12902921#action_12902921 ]
Gary Dusbabek edited comment on CASSANDRA-1415 at 8/26/10 12:40 PM:
--------------------------------------------------------------------
01: it looks like the old version built the index by reading straight from the data file. The new way expects the CFS to be populated with sstables, which isn't always going to be the case (if you're recovering from a descriptor that points to a db that the CFS isn't going to load automatically).
EDIT: forgot to mention that SSTableWriterTest illustrates this.
was (Author: gdusbabek):
01: it looks like the old version built the index by reading straight from the data file. The new way expects the CFS to be populated with sstables, which isn't always going to be the case (if you're recovering from a descriptor that points to a db that the CFS isn't going to load automatically).
> Allow creating indexes on existing data
> ---------------------------------------
>
> Key: CASSANDRA-1415
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1415
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Affects Versions: 0.7 beta 1
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.7 beta 2
>
> Attachments: 0001-Table.rebuildIndex.txt, 0002-extract-addIndex-method.txt, 0003-allow-addIndex-to-create-indexes-that-did-not-previous.txt
>
>
> (This ticket is only for the index-CF creation part, not for the thrift API changes, which may come for free from CASSANDRA-1285)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1415) Allow creating indexes on
existing data
Posted by "Gary Dusbabek (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12913023#action_12913023 ]
Gary Dusbabek commented on CASSANDRA-1415:
------------------------------------------
02: Not relevant to this ticket, but since I spotted it: SRepVH.doVerb() could be simplified if SOS.validateCurrentFile() was made private (or eliminated) and called from startNext() and retry().
The codepaths for index generation on pre-existing sstables and not-yet-existing sstables is different enough that it would be a good idea to modify StreamingTransferTest to verify that 2ary indexes are getting rebuilt properly.
Everything else, +1.
> Allow creating indexes on existing data
> ---------------------------------------
>
> Key: CASSANDRA-1415
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1415
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Affects Versions: 0.7 beta 1
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.7 beta 2
>
> Attachments: 0001-clean-up-FileStatus-and-rename-to-StreamReply.-updates.txt, 0002-add-SESSION_FINISHED-reply.txt, 0003-Table.rebuildIndex.txt, 0004-extract-addIndex-method.txt, 0005-allow-addIndex-to-create-indexes-that-did-not-previous.txt, 0006-split-2ary-index-build-out-from-bloom-row-index-build-.txt, 0007-remove-IKeyIterator-and-move-ICompactionInfo-implement.txt, 0008-bug-fixes.txt
>
>
> (This ticket is only for the index-CF creation part, not for the thrift API changes, which may come for free from CASSANDRA-1285)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1415) Allow creating indexes on existing
data
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-1415:
--------------------------------------
Attachment: (was: 0007-remove-IKeyIterator-and-move-ICompactionInfo-implement.txt)
> Allow creating indexes on existing data
> ---------------------------------------
>
> Key: CASSANDRA-1415
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1415
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Affects Versions: 0.7 beta 1
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.7 beta 2
>
>
> (This ticket is only for the index-CF creation part, not for the thrift API changes, which may come for free from CASSANDRA-1285)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1415) Allow creating indexes on existing
data
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-1415:
--------------------------------------
Attachment: (was: 0003-Table.rebuildIndex.txt)
> Allow creating indexes on existing data
> ---------------------------------------
>
> Key: CASSANDRA-1415
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1415
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Affects Versions: 0.7 beta 1
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.7 beta 2
>
>
> (This ticket is only for the index-CF creation part, not for the thrift API changes, which may come for free from CASSANDRA-1285)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1415) Allow creating indexes on
existing data
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12913348#action_12913348 ]
Jonathan Ellis commented on CASSANDRA-1415:
-------------------------------------------
bq. SRepVH.doVerb() could be simplified if SOS.validateCurrentFile() was made private (or eliminated) and called from startNext() and retry()
I partly agree, but then you'd have to pass file as an argument to startNext and retry, and I want to emphasize that that's not necessary b/c the session has the necessary state.
bq. it would be a good idea to modify StreamingTransferTest to verify that 2ary indexes are getting rebuilt properly
will do.
> Allow creating indexes on existing data
> ---------------------------------------
>
> Key: CASSANDRA-1415
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1415
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Affects Versions: 0.7 beta 1
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Fix For: 0.7 beta 2
>
> Attachments: 0001-clean-up-FileStatus-and-rename-to-StreamReply.-updates.txt, 0002-add-SESSION_FINISHED-reply.txt, 0003-Table.rebuildIndex.txt, 0004-extract-addIndex-method.txt, 0005-allow-addIndex-to-create-indexes-that-did-not-previous.txt, 0006-split-2ary-index-build-out-from-bloom-row-index-build-.txt, 0007-remove-IKeyIterator-and-move-ICompactionInfo-implement.txt, 0008-bug-fixes.txt
>
>
> (This ticket is only for the index-CF creation part, not for the thrift API changes, which may come for free from CASSANDRA-1285)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.