You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@carbondata.apache.org by haomarch <ma...@126.com> on 2020/07/14 09:27:39 UTC
[Disscuss] The precise of timestamp is limited to millisecond in
carbondata, which is incompatiable with DB&Spark&Hive
The SimpleDateFormat is used to convert string and timestamp. But
SimpleDateFormat doesn't support nanoseconds and microsecond, which is is
incompatiable with DB&Spark&Hive.
To support nanoseconds and microsecond, we shall change SimpleDateFormat to
DateTimeFormatter.
Shall we involve this work？
--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Re: [Disscuss] The precise of timestamp is limited to millisecond
in carbondata, which is incompatiable with DB&Spark&Hive
Posted by David CaiQiang <da...@gmail.com>.
agree with Ravi
-----
Best Regards
David Cai
--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Re: [Disscuss] The precise of timestamp is limited to millisecond in
carbondata, which is incompatiable with DB&Spark&Hive
Posted by Kumar Vishal <ku...@gmail.com>.
Hi,
I agree with Ravi, first we need to come up with solution how to store till
nano/mico second in carbon. Then changing the formatter will be useful for
us.
Regards
Kumar Vishal
On Wed, 15 Jul 2020 at 6:18 PM, Ravindra Pesala <ra...@gmail.com>
wrote:
> Hi,
>
> I think it is bigger than just changing to DateTimeFormatter. As of now,
> carbon uses only 64 bit to store timestamp so it can accommodate till
> milliseconds. In order to support till nanoseconds, we need to use 96
> bits. If you check spark-parquet it uses 96 bits to store timestamp. It
> would be good if we also go in that direction to support till nanosecond
> level. Thank you.
>
> Regards,
> Ravindra.
>
> On Wed, 15 Jul 2020 at 14:50, Zhangshunyu <zh...@126.com> wrote:
>
> > +1
> >
> >
> >
> > -----
> > My English name is Sunday
> > --
> > Sent from:
> > http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
> >
>
>
> --
> Thanks & Regards,
> Ravi
>
Re: [Disscuss] The precise of timestamp is limited to millisecond in
carbondata, which is incompatiable with DB&Spark&Hive
Posted by Ravindra Pesala <ra...@gmail.com>.
Hi,
I think it is bigger than just changing to DateTimeFormatter. As of now,
carbon uses only 64 bit to store timestamp so it can accommodate till
milliseconds. In order to support till nanoseconds, we need to use 96
bits. If you check spark-parquet it uses 96 bits to store timestamp. It
would be good if we also go in that direction to support till nanosecond
level. Thank you.
Regards,
Ravindra.
On Wed, 15 Jul 2020 at 14:50, Zhangshunyu <zh...@126.com> wrote:
> +1
>
>
>
> -----
> My English name is Sunday
> --
> Sent from:
> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
>
--
Thanks & Regards,
Ravi
Re: [Disscuss] The precise of timestamp is limited to millisecond
in carbondata, which is incompatiable with DB&Spark&Hive
Posted by Zhangshunyu <zh...@126.com>.
+1
-----
My English name is Sunday
--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Re: [Disscuss] The precise of timestamp is limited to millisecond in
carbondata, which is incompatiable with DB&Spark&Hive
Posted by Ajantha Bhat <aj...@gmail.com>.
+ 1,
as SimpleDateFormat doesn't support nanoseconds and microsecond.
Thanks,
Ajantha
On Tue, Jul 14, 2020 at 5:03 PM xubo245 <60...@qq.com> wrote:
> +1, please consider compatable for history data
>
>
>
> --
> Sent from:
> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
>
Re: [Disscuss] The precise of timestamp is limited to millisecond
in carbondata, which is incompatiable with DB&Spark&Hive
Posted by xubo245 <60...@qq.com>.
+1, please consider compatable for history data
--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/