[jira] [Comment Edited] (CASSANDRA-8460) Make it possible to move non-compacting sstables to slow/big storage in DTCS

["Jeff Jirsa (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14583809#comment-14583809 ] 

Jeff Jirsa edited comment on CASSANDRA-8460 at 6/12/15 6:07 PM:
----------------------------------------------------------------

{quote}yes, I've been thinking maybe adding priorities or tags to the data directories, but that is probably not needed now. Adding a flag to each data_directory that states whether it is for archival storage or not is probably enough for now.{quote}

Asking for clarification to make sure I don't go too far into pony land:

So my initial approach was to define a second config item, separate from {{data_file_directories}} entirely, so that no other code needed to be aware of it except for classes explicitly wanting to use `archive` tier storage ( {{dd.getAllDataFileLocations()}} would not return the archive tier, but rather add a {{dd.getArchiveDataFileLocations()}} specifically for the slow class of storage).  

It sounds from your description you're envisioning changing the list of data_file_locations to a map {noformat} [tag1:location1,tag1:location2,tag3:location3] {noformat} or {noformat} tag1:[location1,location2],tag3:[location3] {noformat} In this case, we'd also need to maintain backwards compatibility, which seems fairly straight forward to do (check to see if the provided {{data_files_directory}} is an old-format list rather than map and apply some default tag?)

The first approach is clean and isolated, unlikely to introduce surprises, but potentially limits us from being able to do more interesting work with tagged data file directories later (ie: only store data for KS W in data directories tagged X, and KS Y in data directories tagged Z). Can you clarify which best fits your expectations? 



was (Author: jjirsa):
{quote}yes, I've been thinking maybe adding priorities or tags to the data directories, but that is probably not needed now. Adding a flag to each data_directory that states whether it is for archival storage or not is probably enough for now.{quote}

Asking for clarification to make sure I don't go too far into pony land:

So my initial approach was to define a second config item, separate from {{data_file_directories}} entirely, so that no other code needed to be aware of it except for classes explicitly wanting to use `archive` tier storage ( {{dd.getAllDataFileLocations()}} would not return the archive tier, but rather add a {{dd.getArchiveDataFileLocations()}} specifically for the slow class of storage).  

It sounds from your description you're envisioning changing the list of data_file_locations to a list of maps {noformat} [tag1:location1,tag1:location2,tag3:location3] {noformat} or {noformat} tag1:[location1,location2],tag3:[location3] {noformat} In this case, we'd also need to maintain backwards compatibility, which seems fairly straight forward to do (check to see if the provided {{data_files_directory}} is an old-format list rather than map and apply some default tag?)

The first approach is clean and isolated, unlikely to introduce surprises, but potentially limits us from being able to do more interesting work with tagged data file directories later (ie: only store data for KS W in data directories tagged X, and KS Y in data directories tagged Z). Can you clarify which best fits your expectations? 


> Make it possible to move non-compacting sstables to slow/big storage in DTCS
> ----------------------------------------------------------------------------
>
>                 Key: CASSANDRA-8460
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-8460
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Marcus Eriksson
>              Labels: dtcs
>
> It would be nice if we could configure DTCS to have a set of extra data directories where we move the sstables once they are older than max_sstable_age_days. 
> This would enable users to have a quick, small SSD for hot, new data, and big spinning disks for data that is rarely read and never compacted.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)