You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Vladimir Ozerov <vo...@gridgain.com> on 2018/11/20 07:45:16 UTC
Rename "cache group" to "tablespace"?
Igniters,
We had several discussion about cache groups [1]. Our current consensus is
that this is not that good design decision - bad scan performance, no API.
A kind of shortcut to solve memory consumption problems. For this reason we
try to avoid exposing "cache group" to API when possible. Also we
understand that current file-per-partition approach is not ideal either -
when there are too many partitions we have to fsync a lot of files. And as
an idea for "bright future" we consider tablespaces - one or several files
where all database objects are stored in dedicated "segments", which are
allocated in smaller units called "extents".
I was thinking on how to "legalize" cache groups on API on the one hand
(product needs them currently), and to let real tablespaces to appear in
future. Idea is simple - treat cache group as special kind of tablespace.
We may say that current storage organization is one type of tablespace, and
proposed "file-segment-extent" storage organization is another type of
tablespace.
With this in mind we can configure tablespaces in IgniteConfiguration, then
assign each cache a tablespace. As advanced tasks we may allow dynamic
table space create/alter/drop, I'll show SQL examples below.
// Interface
interface Tablespace {
String name();
}
// Current non-shared tablespace (aka "physical cache")
class FilePerPartitionTablespace implements Tablespace {
// ...
}
// Shared tablespace (aka "cache group") - note that now we have a legal
place for cache group configuration:
class FilePerPartitionSharedTablespace implements Tablespace {
CacheMode cacheMode;
CacheAtomicityMode cacheAtomicityMode;
AffinityFunction affinityFunction;
// + Other parameters from
ClusterCachesInfo.validateCacheGroupConfiguration
// + Some parameters from "DataRegionConfiguration" (e.g. paths)
}
// Future "real" tablespace
class SegmentedTablespace implements Tablespace {
// Whatever is needed.
}
// Cache configuration
class CacheConfiguration {
...
String tablespace;
...
}
// Managing tablespaces from SQL:
CREATE TABLESPACE my_tbs ENGINE=FILE_PER_PARTITION_SHARED
CACHE_MODE=PARTITIONED BACKUPS=1
ALTER TABLESPACE my_tbs ENCRYPTED=1
CREATE TABLE my_table (...) TABLESPACE my_tbs
What do you think?
Vladimir.
[1]
http://apache-ignite-developers.2346864.n4.nabble.com/Remove-cache-groups-in-AI-3-0-td29184.html
Re: Rename "cache group" to "tablespace"?
Posted by Vladimir Ozerov <vo...@gridgain.com>.
Hi Ivan,
I do not have answers to some of your questions, as haven't designed
tablespaces for Ignite so far. This is common term used by many vendors
(Oracle, MySQL, MongoDB to name a few). Essentially, "tablespace" is how
data of certain objects are organized into files. Multiple tablespaces for
separate objects provides performance and management benefits. For example,
different sets of objects might be written to different disks on local
machine to improve throughput. Or writes to a single object (e.g. cache)
may be parallelized in the same way.
But in this topic I would prefer to avoid digging much into details. My
goal for now is to understand whether we can create a single concept which
will be applicable to both existing and future storage formats.
Vladimir.
On Fri, Nov 23, 2018 at 10:41 AM Павлухин Иван <vo...@gmail.com> wrote:
> Hi Vladimir,
>
> How do you see a default tablespace configuration in the "bright"
> future? Is it going to be a single file for an Ignite instance? In
> what cases will be there benefits to explicitly configure additional
> tablespaces?
> By the way was the name "tablespace" hand-crafted or it was it
> borrowed somewhere?
> Also if it is TABLEspace should it be aligned with renaming
> IgniteCache to IgniteTable discussed in [1]?
>
> [1]
> http://apache-ignite-developers.2346864.n4.nabble.com/Applicability-of-term-cache-to-Apache-Ignite-td36541.html
> вт, 20 нояб. 2018 г. в 10:57, Dmitriy Setrakyan <ds...@apache.org>:
> >
> > I really like the idea.
> >
> > On Mon, Nov 19, 2018, 23:45 Vladimir Ozerov <vozerov@gridgain.com wrote:
> >
> > > Igniters,
> > >
> > > We had several discussion about cache groups [1]. Our current
> consensus is
> > > that this is not that good design decision - bad scan performance, no
> API.
> > > A kind of shortcut to solve memory consumption problems. For this
> reason we
> > > try to avoid exposing "cache group" to API when possible. Also we
> > > understand that current file-per-partition approach is not ideal
> either -
> > > when there are too many partitions we have to fsync a lot of files.
> And as
> > > an idea for "bright future" we consider tablespaces - one or several
> files
> > > where all database objects are stored in dedicated "segments", which
> are
> > > allocated in smaller units called "extents".
> > >
> > > I was thinking on how to "legalize" cache groups on API on the one hand
> > > (product needs them currently), and to let real tablespaces to appear
> in
> > > future. Idea is simple - treat cache group as special kind of
> tablespace.
> > > We may say that current storage organization is one type of
> tablespace, and
> > > proposed "file-segment-extent" storage organization is another type of
> > > tablespace.
> > >
> > > With this in mind we can configure tablespaces in IgniteConfiguration,
> then
> > > assign each cache a tablespace. As advanced tasks we may allow dynamic
> > > table space create/alter/drop, I'll show SQL examples below.
> > >
> > > // Interface
> > > interface Tablespace {
> > > String name();
> > > }
> > >
> > > // Current non-shared tablespace (aka "physical cache")
> > > class FilePerPartitionTablespace implements Tablespace {
> > > // ...
> > > }
> > >
> > > // Shared tablespace (aka "cache group") - note that now we have a
> legal
> > > place for cache group configuration:
> > > class FilePerPartitionSharedTablespace implements Tablespace {
> > > CacheMode cacheMode;
> > > CacheAtomicityMode cacheAtomicityMode;
> > > AffinityFunction affinityFunction;
> > > // + Other parameters from
> > > ClusterCachesInfo.validateCacheGroupConfiguration
> > > // + Some parameters from "DataRegionConfiguration" (e.g. paths)
> > > }
> > >
> > > // Future "real" tablespace
> > > class SegmentedTablespace implements Tablespace {
> > > // Whatever is needed.
> > > }
> > >
> > > // Cache configuration
> > > class CacheConfiguration {
> > > ...
> > > String tablespace;
> > > ...
> > > }
> > >
> > > // Managing tablespaces from SQL:
> > > CREATE TABLESPACE my_tbs ENGINE=FILE_PER_PARTITION_SHARED
> > > CACHE_MODE=PARTITIONED BACKUPS=1
> > > ALTER TABLESPACE my_tbs ENCRYPTED=1
> > > CREATE TABLE my_table (...) TABLESPACE my_tbs
> > >
> > > What do you think?
> > >
> > > Vladimir.
> > >
> > > [1]
> > >
> > >
> http://apache-ignite-developers.2346864.n4.nabble.com/Remove-cache-groups-in-AI-3-0-td29184.html
> > >
>
>
>
> --
> Best regards,
> Ivan Pavlukhin
>
Re: Rename "cache group" to "tablespace"?
Posted by Павлухин Иван <vo...@gmail.com>.
Hi Vladimir,
How do you see a default tablespace configuration in the "bright"
future? Is it going to be a single file for an Ignite instance? In
what cases will be there benefits to explicitly configure additional
tablespaces?
By the way was the name "tablespace" hand-crafted or it was it
borrowed somewhere?
Also if it is TABLEspace should it be aligned with renaming
IgniteCache to IgniteTable discussed in [1]?
[1] http://apache-ignite-developers.2346864.n4.nabble.com/Applicability-of-term-cache-to-Apache-Ignite-td36541.html
вт, 20 нояб. 2018 г. в 10:57, Dmitriy Setrakyan <ds...@apache.org>:
>
> I really like the idea.
>
> On Mon, Nov 19, 2018, 23:45 Vladimir Ozerov <vozerov@gridgain.com wrote:
>
> > Igniters,
> >
> > We had several discussion about cache groups [1]. Our current consensus is
> > that this is not that good design decision - bad scan performance, no API.
> > A kind of shortcut to solve memory consumption problems. For this reason we
> > try to avoid exposing "cache group" to API when possible. Also we
> > understand that current file-per-partition approach is not ideal either -
> > when there are too many partitions we have to fsync a lot of files. And as
> > an idea for "bright future" we consider tablespaces - one or several files
> > where all database objects are stored in dedicated "segments", which are
> > allocated in smaller units called "extents".
> >
> > I was thinking on how to "legalize" cache groups on API on the one hand
> > (product needs them currently), and to let real tablespaces to appear in
> > future. Idea is simple - treat cache group as special kind of tablespace.
> > We may say that current storage organization is one type of tablespace, and
> > proposed "file-segment-extent" storage organization is another type of
> > tablespace.
> >
> > With this in mind we can configure tablespaces in IgniteConfiguration, then
> > assign each cache a tablespace. As advanced tasks we may allow dynamic
> > table space create/alter/drop, I'll show SQL examples below.
> >
> > // Interface
> > interface Tablespace {
> > String name();
> > }
> >
> > // Current non-shared tablespace (aka "physical cache")
> > class FilePerPartitionTablespace implements Tablespace {
> > // ...
> > }
> >
> > // Shared tablespace (aka "cache group") - note that now we have a legal
> > place for cache group configuration:
> > class FilePerPartitionSharedTablespace implements Tablespace {
> > CacheMode cacheMode;
> > CacheAtomicityMode cacheAtomicityMode;
> > AffinityFunction affinityFunction;
> > // + Other parameters from
> > ClusterCachesInfo.validateCacheGroupConfiguration
> > // + Some parameters from "DataRegionConfiguration" (e.g. paths)
> > }
> >
> > // Future "real" tablespace
> > class SegmentedTablespace implements Tablespace {
> > // Whatever is needed.
> > }
> >
> > // Cache configuration
> > class CacheConfiguration {
> > ...
> > String tablespace;
> > ...
> > }
> >
> > // Managing tablespaces from SQL:
> > CREATE TABLESPACE my_tbs ENGINE=FILE_PER_PARTITION_SHARED
> > CACHE_MODE=PARTITIONED BACKUPS=1
> > ALTER TABLESPACE my_tbs ENCRYPTED=1
> > CREATE TABLE my_table (...) TABLESPACE my_tbs
> >
> > What do you think?
> >
> > Vladimir.
> >
> > [1]
> >
> > http://apache-ignite-developers.2346864.n4.nabble.com/Remove-cache-groups-in-AI-3-0-td29184.html
> >
--
Best regards,
Ivan Pavlukhin
Re: Rename "cache group" to "tablespace"?
Posted by Dmitriy Setrakyan <ds...@apache.org>.
I really like the idea.
On Mon, Nov 19, 2018, 23:45 Vladimir Ozerov <vozerov@gridgain.com wrote:
> Igniters,
>
> We had several discussion about cache groups [1]. Our current consensus is
> that this is not that good design decision - bad scan performance, no API.
> A kind of shortcut to solve memory consumption problems. For this reason we
> try to avoid exposing "cache group" to API when possible. Also we
> understand that current file-per-partition approach is not ideal either -
> when there are too many partitions we have to fsync a lot of files. And as
> an idea for "bright future" we consider tablespaces - one or several files
> where all database objects are stored in dedicated "segments", which are
> allocated in smaller units called "extents".
>
> I was thinking on how to "legalize" cache groups on API on the one hand
> (product needs them currently), and to let real tablespaces to appear in
> future. Idea is simple - treat cache group as special kind of tablespace.
> We may say that current storage organization is one type of tablespace, and
> proposed "file-segment-extent" storage organization is another type of
> tablespace.
>
> With this in mind we can configure tablespaces in IgniteConfiguration, then
> assign each cache a tablespace. As advanced tasks we may allow dynamic
> table space create/alter/drop, I'll show SQL examples below.
>
> // Interface
> interface Tablespace {
> String name();
> }
>
> // Current non-shared tablespace (aka "physical cache")
> class FilePerPartitionTablespace implements Tablespace {
> // ...
> }
>
> // Shared tablespace (aka "cache group") - note that now we have a legal
> place for cache group configuration:
> class FilePerPartitionSharedTablespace implements Tablespace {
> CacheMode cacheMode;
> CacheAtomicityMode cacheAtomicityMode;
> AffinityFunction affinityFunction;
> // + Other parameters from
> ClusterCachesInfo.validateCacheGroupConfiguration
> // + Some parameters from "DataRegionConfiguration" (e.g. paths)
> }
>
> // Future "real" tablespace
> class SegmentedTablespace implements Tablespace {
> // Whatever is needed.
> }
>
> // Cache configuration
> class CacheConfiguration {
> ...
> String tablespace;
> ...
> }
>
> // Managing tablespaces from SQL:
> CREATE TABLESPACE my_tbs ENGINE=FILE_PER_PARTITION_SHARED
> CACHE_MODE=PARTITIONED BACKUPS=1
> ALTER TABLESPACE my_tbs ENCRYPTED=1
> CREATE TABLE my_table (...) TABLESPACE my_tbs
>
> What do you think?
>
> Vladimir.
>
> [1]
>
> http://apache-ignite-developers.2346864.n4.nabble.com/Remove-cache-groups-in-AI-3-0-td29184.html
>