Data Types

[zeng jienan <ze...@hotmail.com>]

Hi,

Will druid support other data types in the future? Such as boolean, byte, short, int.

Re: Data Types

[zeng jienan <ze...@hotmail.com>]

I currently want to try to use druid to store detailed data, using computing engine such as presto, spark directly pull data from the druid for sql query.  With druid columnar storage and inverted indexing,  I expect it will achieve good query performance, while supporting real-time data write

Of course, there may be some problems。Here are some of the challenges I have come up with.

1. Each data requires a time attribute
	For dimension tables, the __time column gives a default constant value, and the fact table generally has a time column.
2. Index Overhead
	For a string column that does not use an index, it is forbidden to build an inverted index, reducing the index creation overhead.
3. data type 
	I think there are several advantages to the refinement of the data type.
		1. It will be more friendly for users who are transferred from other databases.
		2. Proper data types can reduce storage space and of course accelerate calculations indirectly
	Of course, I also think that the current data type is enough.


> 在 2019年3月29日，下午11:37，Gian Merlino <gi...@apache.org> 写道：
> 
> I don't see why it shouldn't support other data types, but like Charles
> said, it should ideally be use case driven, since adding new data types
> bumps up the complexity level of the core code permanently. I don't see a
> reason to add every single SQL data type just because it exists, but it
> does make sense to me to add data types that add meaningful new
> capabilities or allow meaningful new optimizations. One would be "DECIMAL",
> would definitely be useful to have that one.
> 
> There is also the possibility of adding types that only exist in the SQL
> layer, for example adding "TIMESTAMP" or "DATE" types that under the hood
> are really just long (int64) columns. We do that right now for the __time
> column but that is a special-case, so you cannot currently add secondary
> TIMESTAMP columns. But it might be cool to allow it.
> 
> On Fri, Mar 29, 2019 at 8:21 AM Charles Allen
> <ch...@snap.com.invalid> wrote:
> 
>> For my team we start from the other direction. What are people DOING with
>> the data. For example, if they are doing counts and sums with basic
>> predicates, then in what ways does the existing feature set not meet those
>> needs?
>> 
>> If they are doing other things, what is the end result they are trying to
>> achieve?
>> 
>> Can you provide more context on the end use cases?
>> 
>> On Fri, Mar 29, 2019, 12:24 AM zeng jienan <ze...@hotmail.com> wrote:
>> 
>>> Hi,
>>> 
>>> Will druid support other data types in the future? Such as boolean, byte,
>>> short, int.
>>> 
>> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@druid.apache.org
For additional commands, e-mail: dev-help@druid.apache.org

Re: Data Types

[Gian Merlino <gi...@apache.org>]

I don't see why it shouldn't support other data types, but like Charles
said, it should ideally be use case driven, since adding new data types
bumps up the complexity level of the core code permanently. I don't see a
reason to add every single SQL data type just because it exists, but it
does make sense to me to add data types that add meaningful new
capabilities or allow meaningful new optimizations. One would be "DECIMAL",
would definitely be useful to have that one.

There is also the possibility of adding types that only exist in the SQL
layer, for example adding "TIMESTAMP" or "DATE" types that under the hood
are really just long (int64) columns. We do that right now for the __time
column but that is a special-case, so you cannot currently add secondary
TIMESTAMP columns. But it might be cool to allow it.

On Fri, Mar 29, 2019 at 8:21 AM Charles Allen
<ch...@snap.com.invalid> wrote:

> For my team we start from the other direction. What are people DOING with
> the data. For example, if they are doing counts and sums with basic
> predicates, then in what ways does the existing feature set not meet those
> needs?
>
> If they are doing other things, what is the end result they are trying to
> achieve?
>
> Can you provide more context on the end use cases?
>
> On Fri, Mar 29, 2019, 12:24 AM zeng jienan <ze...@hotmail.com> wrote:
>
> > Hi,
> >
> > Will druid support other data types in the future? Such as boolean, byte,
> > short, int.
> >
>

Re: Data Types

[Charles Allen <ch...@snap.com.INVALID>]

For my team we start from the other direction. What are people DOING with
the data. For example, if they are doing counts and sums with basic
predicates, then in what ways does the existing feature set not meet those
needs?

If they are doing other things, what is the end result they are trying to
achieve?

Can you provide more context on the end use cases?

On Fri, Mar 29, 2019, 12:24 AM zeng jienan <ze...@hotmail.com> wrote:

> Hi,
>
> Will druid support other data types in the future? Such as boolean, byte,
> short, int.
>