You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2022/10/12 08:10:37 UTC
[GitHub] [pinot] jatin5251 opened a new issue, #9575: Add Support for DATE datatype in Pinot
jatin5251 opened a new issue, #9575:
URL: https://github.com/apache/pinot/issues/9575
We have use case where we need to store date datatype in pinot.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Add Support for DATE datatype in Pinot [pinot]
Posted by "jatin5251 (via GitHub)" <gi...@apache.org>.
jatin5251 commented on issue #9575:
URL: https://github.com/apache/pinot/issues/9575#issuecomment-1870499582
@Jackie-Jiang @walterddr @xiangfu0 @kishoreg Guys, can we support this in near future?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] jatin5251 commented on issue #9575: Add Support for DATE datatype in Pinot
Posted by GitBox <gi...@apache.org>.
jatin5251 commented on issue #9575:
URL: https://github.com/apache/pinot/issues/9575#issuecomment-1288186971
@walterddr we are looking to have DATE type in pinot, we can insert date type from trino and read from python connector (https://pypi.org/project/pinotdb/ ). i am ok with Datetime as well but i need to make modification in connector code to make it compatible with date type only
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] jatin5251 commented on issue #9575: Add Support for DATE datatype in Pinot
Posted by GitBox <gi...@apache.org>.
jatin5251 commented on issue #9575:
URL: https://github.com/apache/pinot/issues/9575#issuecomment-1275844555
We are using tableau for visualization and due to its licensing cost we are in process to deprecate it with superset
But tableau has extract feature in which we save output of sql query in tableau server as in-memory.
To add this functionality in superset we build one service that will take sql query from superset, run it on trino then trino-pinot connector will insert data into pinot(insert support added by @elonazoulay in trino )
In our service we first take the query from user, get its schema from trino and convert the schema in a way that will be compatible to pinot like bigint to long
As hive tables has support for date datatype but not in pinot then we need to ask users to cast date type to varchar in query.
As we have all primitive data type support in pinot except Date. We want to add this support in pinot so that users will have good experience.
please let me know if you need more information
Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] kishoreg commented on issue #9575: Add Support for DATE datatype in Pinot
Posted by GitBox <gi...@apache.org>.
kishoreg commented on issue #9575:
URL: https://github.com/apache/pinot/issues/9575#issuecomment-1279869694
+1 to add date as a first class data type. Will be useful as we integrate with tools other than presto/Trino
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on issue #9575: Add Support for DATE datatype in Pinot
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on issue #9575:
URL: https://github.com/apache/pinot/issues/9575#issuecomment-1281656419
Related to #9546
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] walterddr commented on issue #9575: Add Support for DATE datatype in Pinot
Posted by GitBox <gi...@apache.org>.
walterddr commented on issue #9575:
URL: https://github.com/apache/pinot/issues/9575#issuecomment-1282498435
is this a duplicate of https://github.com/apache/pinot/issues/8045 or do we expect more / different from a DateTime type?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] jatin5251 commented on issue #9575: Add Support for DATE datatype in Pinot
Posted by GitBox <gi...@apache.org>.
jatin5251 commented on issue #9575:
URL: https://github.com/apache/pinot/issues/9575#issuecomment-1280042964
@xiangfu0 we are inserting data through trino and reading directly from superset. i think this property will not help much in this case but thanks to your suggestion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] jatin5251 commented on issue #9575: Add Support for DATE datatype in Pinot
Posted by GitBox <gi...@apache.org>.
jatin5251 commented on issue #9575:
URL: https://github.com/apache/pinot/issues/9575#issuecomment-1278258294
we are only looking for plain text at the moment. i think we can store as long (millis since epoch) that will help in future once we introduce DATE_TIME in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] xiangfu0 commented on issue #9575: Add Support for DATE datatype in Pinot
Posted by GitBox <gi...@apache.org>.
xiangfu0 commented on issue #9575:
URL: https://github.com/apache/pinot/issues/9575#issuecomment-1279857045
Actually, in PrestoDB, there is a config to infer `DATE` type based on Pinot Schema: `pinot.infer-date-type-in-schema`.
https://prestodb.io/docs/current/connector/pinot.html#map-pinot-schema-to-presto-schema
I feel maybe we can implement this in Trino as a temporary workaround before this type is supported in Pinot.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] jatin5251 commented on issue #9575: Add Support for DATE datatype in Pinot
Posted by GitBox <gi...@apache.org>.
jatin5251 commented on issue #9575:
URL: https://github.com/apache/pinot/issues/9575#issuecomment-1288187700
@Jackie-Jiang
> Related to #9546
we are looking for date type support only in which we don't have any time part. let me know your thoughts
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on issue #9575: Add Support for DATE datatype in Pinot
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on issue #9575:
URL: https://github.com/apache/pinot/issues/9575#issuecomment-1278251563
For the `DATE` type, do you need time zone info, or the plain text such as `2022-10-13`? To support it, internally we can store the value as either long (millis since epoch) or int (days since epoch).
We need to consider whether to support the comparison across different time types, e.g. comparing `DATE` with `TIMESTAMP`. We also want to introduce `DATE_TIME` in the future, which support custom date time pattern and time zone.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] xiangfu0 commented on issue #9575: Add Support for DATE datatype in Pinot
Posted by GitBox <gi...@apache.org>.
xiangfu0 commented on issue #9575:
URL: https://github.com/apache/pinot/issues/9575#issuecomment-1275793036
Can you elaborate more on the use case and curious if TIMESTAMP can be an alternative for it?
cc: @Jackie-Jiang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Add Support for DATE datatype in Pinot [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on issue #9575:
URL: https://github.com/apache/pinot/issues/9575#issuecomment-1897117240
@jatin5251 It is not on our top priority list, but contribution is very welcome.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org