Question on TimestampNTZ handling direction in Comet (issue #3180)

[Vignesh Siva <vi...@gmail.com>]

Hi all,

I’m working on issue #3180 related to incorrect timezone conversion for
hour/minute/second on TimestampNTZ inputs.

While preparing a fix, I wanted to clarify the intended direction:

Is the goal to fully support Spark-compatible TimestampNTZ semantics
directly in the Rust implementation (i.e., bypassing timezone conversion),
or is TimestampNTZ expected to remain unsupported at the Scala serde layer
until the broader Comet timezone handling work is completed?

I’d like to ensure the fix aligns with the long-term design rather than
introducing behavior that may be reconsidered later.

Thanks for the guidance,
Vignesh

Re: Question on TimestampNTZ handling direction in Comet (issue #3180)

[Vignesh Siva <vi...@gmail.com>]

Hi Parth,

Thanks for the clarification; that makes sense.

I agree it’s better to hold off on making targeted changes TimestampNTZ in
the native layer if a broader review and redesign of timestamp handling is
planned. I didn’t want to introduce a fix that might conflict with upcoming
design decisions.

I’ll pause work on this specific improvement for now and keep an eye on the
larger timestamp/TimestampNTZ overhaul discussions. Happy to revisit once
the direction is clearer.

Thanks again for the guidance.

Best,
Vignesh

On Mon, 2 Feb 2026 at 22:47, Parth Chandra <pa...@apache.org> wrote:

> While the goal is for TimestampNTZ in the native code to be compatible with
> Spark, it might be a good idea to hold off on this specific improvement
> until we review and overhaul the timestamp handling in general (and
> TimestampNTZ in particular).
>
> On Sat, Jan 24, 2026 at 8:16 AM Vignesh Siva <vi...@gmail.com>
> wrote:
>
> > Hi all,
> >
> > I’m working on issue #3180 related to incorrect timezone conversion for
> > hour/minute/second on TimestampNTZ inputs.
> >
> > While preparing a fix, I wanted to clarify the intended direction:
> >
> > Is the goal to fully support Spark-compatible TimestampNTZ semantics
> > directly in the Rust implementation (i.e., bypassing timezone
> conversion),
> > or is TimestampNTZ expected to remain unsupported at the Scala serde
> layer
> > until the broader Comet timezone handling work is completed?
> >
> > I’d like to ensure the fix aligns with the long-term design rather than
> > introducing behavior that may be reconsidered later.
> >
> > Thanks for the guidance,
> > Vignesh
> >
>

Re: Question on TimestampNTZ handling direction in Comet (issue #3180)

[Parth Chandra <pa...@apache.org>]

While the goal is for TimestampNTZ in the native code to be compatible with
Spark, it might be a good idea to hold off on this specific improvement
until we review and overhaul the timestamp handling in general (and
TimestampNTZ in particular).

On Sat, Jan 24, 2026 at 8:16 AM Vignesh Siva <vi...@gmail.com>
wrote:

> Hi all,
>
> I’m working on issue #3180 related to incorrect timezone conversion for
> hour/minute/second on TimestampNTZ inputs.
>
> While preparing a fix, I wanted to clarify the intended direction:
>
> Is the goal to fully support Spark-compatible TimestampNTZ semantics
> directly in the Rust implementation (i.e., bypassing timezone conversion),
> or is TimestampNTZ expected to remain unsupported at the Scala serde layer
> until the broader Comet timezone handling work is completed?
>
> I’d like to ensure the fix aligns with the long-term design rather than
> introducing behavior that may be reconsidered later.
>
> Thanks for the guidance,
> Vignesh
>