EntityStore entity_state length limit(MySQL varchar(10000))? binary format EntityStore?

[zhuangmz08 <zh...@qq.com>]

Hi, I like the idea to persist domain data as entities in EntityStore so much! I have some questions as follows, any help will be appreciated.
1. I find if I use MySQL, entity_state will be limited to varchar(10000)? If I use Postgres, entity_state will be [text] whose limitation might be 1GB?
Another confusion is that there is only ONE SQL table for ALL entity type? Or ONE SQL table for ONE entity type?
2. I'd like to use Zest for designing domain model on timeseries-based data which will be very big scale. SQL might be too slow. How could I using EntityStore for KairosDB(https://github.com/kairosdb/kairosdb) or  Influxdb(https://github.com/influxdata/influxdb)?
3. EntityStore is persisted in JSON text format? Can I using binary format which might occupy less disk space? 
Thanks a lot!

Re: EntityStore entity_state length limit(MySQL varchar(10000))? binary format EntityStore?

[Niclas Hedhman <ni...@hedhman.org>]

1. I haven't worked on the SQL entity store, but I think it is a single
table for all types. The reason for this is that any Entity can have any
number of EntityTypes, overlapping or not. This means that potentially each
Entity instance could be a slightly different type from any other entity
instance. By using the document serialization format, we can handle this
quite well, including casts between incompatible types, and yet keep the
state intact when saved.
As for the column size, feel free to submit patch against MySQL, if you
know what is a better way.


2. Direct Timeseries support in Zest is planned (mostly because I want to
use it as well). See https://issues.apache.org/jira/browse/ZEST-96, but no
work has been done on it yet.
At the moment, it depends on what you want to do. Those DBs are not really
made for storing arbitrary entities (think documents), so it can be done if
you really want to. Implementing a EntityStore based on the
org.apache.zest.spi.entitystore.helpers.JSONMapEntityStoreMixin and
knowledge of the DB in question, shouldn't take more than half a day. I did
a Coherence implementation in less time than it took Randy Stafford from
Oracle to explain what Coherence was at a conference.


3. Yes. The prevalence of JSON as storage is primarily because
org.apache.zest.spi.entitystore.helpers.JSONMapEntityStoreMixin is so easy
to use. But for instance Neo4j EntityStore is completely native to Neo4j's
data model, storing Zest Property values as Node properties, and Zest
Associations as Neo4j Relationships. The work might be a lot more
exhaustive, but indeed doable.


Cheers
Niclas

On Fri, Apr 15, 2016 at 10:14 AM, zhuangmz08 <zh...@qq.com> wrote:

> Hi, I like the idea to persist domain data as entities in EntityStore so
> much! I have some questions as follows, any help will be appreciated.
> 1. I find if I use MySQL, entity_state will be limited to varchar(10000)?
> If I use Postgres, entity_state will be [text] whose limitation might be
> 1GB?
> Another confusion is that there is only ONE SQL table for ALL entity type?
> Or ONE SQL table for ONE entity type?
> 2. I'd like to use Zest for designing domain model on timeseries-based
> data which will be very big scale. SQL might be too slow. How could I using
> EntityStore for KairosDB(https://github.com/kairosdb/kairosdb) or
> Influxdb(https://github.com/influxdata/influxdb)?
> 3. EntityStore is persisted in JSON text format? Can I using binary format
> which might occupy less disk space?
> Thanks a lot!




-- 
Niclas Hedhman, Software Developer
http://zest.apache.org - New Energy for Java