You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "erezrokah (via GitHub)" <gi...@apache.org> on 2025/09/16 12:02:03 UTC

[PR] feat: Add support for specifying `Timestamp` `valueStr` output format [arrow-go]

erezrokah opened a new pull request, #510:
URL: https://github.com/apache/arrow-go/pull/510

   ### Rationale for this change
   
   See discussion in https://github.com/apache/arrow-go/pull/450
   
   ### What changes are included in this PR?
   
   Allow setting the formatting of timestamps when using `ValueStr` (or JSON marshalling)
   
   ### Are these changes tested?
   
   Yes - re-added the original test that was modified in https://github.com/apache/arrow-go/pull/450
   
   ### Are there any user-facing changes?
   
   Yes - users will now be able to revert back to the old format before https://github.com/apache/arrow-go/pull/450, to avoid unintentional breakage
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat: Add support for specifying `Timestamp` `ValueStr` output layout [arrow-go]

Posted by "erezrokah (via GitHub)" <gi...@apache.org>.
erezrokah commented on PR #510:
URL: https://github.com/apache/arrow-go/pull/510#issuecomment-3400762939

   Hi @zeroshade any chance of doing a release with this feature soon?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat: Add support for specifying `Timestamp` `ValueStr` output layout [arrow-go]

Posted by "erezrokah (via GitHub)" <gi...@apache.org>.
erezrokah commented on PR #510:
URL: https://github.com/apache/arrow-go/pull/510#issuecomment-3311280685

   Thank you @zeroshade and @efd6 for the swift review process, appreciate it


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat: Add support for specifying `Timestamp` `valueStr` output format [arrow-go]

Posted by "efd6 (via GitHub)" <gi...@apache.org>.
efd6 commented on code in PR #510:
URL: https://github.com/apache/arrow-go/pull/510#discussion_r2353638867


##########
arrow/array/timestamp.go:
##########
@@ -34,11 +34,16 @@ import (
 type Timestamp struct {
 	array
 	values []arrow.Timestamp
+	format string
 }
 
 // NewTimestampData creates a new Timestamp from Data.
 func NewTimestampData(data arrow.ArrayData) *Timestamp {
-	a := &Timestamp{}
+	return NewTimestampDataWithFormat(data, time.RFC3339Nano)
+}
+
+func NewTimestampDataWithFormat(data arrow.ArrayData, format string) *Timestamp {

Review Comment:
   ```suggestion
   func NewTimestampDataWithLayout(data arrow.ArrayData, layout string) *Timestamp {
   ```
   reflecting the terminology used in stdlib/time (`s/format/layout/g` throughout)
   
   Also probably want godoc for this.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat: Add support for specifying `Timestamp` `ValueStr` output layout [arrow-go]

Posted by "erezrokah (via GitHub)" <gi...@apache.org>.
erezrokah commented on code in PR #510:
URL: https://github.com/apache/arrow-go/pull/510#discussion_r2358776126


##########
arrow/array/timestamp.go:
##########
@@ -34,11 +34,18 @@ import (
 type Timestamp struct {
 	array
 	values []arrow.Timestamp
+	layout string
 }
 
 // NewTimestampData creates a new Timestamp from Data.
 func NewTimestampData(data arrow.ArrayData) *Timestamp {
-	a := &Timestamp{}
+	return NewTimestampDataWithLayout(data, time.RFC3339Nano)
+}
+
+// NewTimestampDataWithLayout creates a new Timestamp from Data with a custom string value layout.
+// This is useful for cases where consumers expect a non standard layout
+func NewTimestampDataWithLayout(data arrow.ArrayData, layout string) *Timestamp {

Review Comment:
   How about `NewTimestampDataWithValueStrLayout`? Happy to change it if this is not what you have in mind



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat: Add support for specifying `Timestamp` `ValueStr` output layout [arrow-go]

Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #510:
URL: https://github.com/apache/arrow-go/pull/510#discussion_r2356428560


##########
arrow/array/timestamp.go:
##########
@@ -34,11 +34,18 @@ import (
 type Timestamp struct {
 	array
 	values []arrow.Timestamp
+	layout string
 }
 
 // NewTimestampData creates a new Timestamp from Data.
 func NewTimestampData(data arrow.ArrayData) *Timestamp {
-	a := &Timestamp{}
+	return NewTimestampDataWithLayout(data, time.RFC3339Nano)
+}
+
+// NewTimestampDataWithLayout creates a new Timestamp from Data with a custom string value layout.
+// This is useful for cases where consumers expect a non standard layout
+func NewTimestampDataWithLayout(data arrow.ArrayData, layout string) *Timestamp {

Review Comment:
   I'd rather slightly different naming to indicate we're talking about the layout *when converting to a string*
   
   Also, we should indicate in the comment that the layout refers to the layout from `[time.Time.Format]`



##########
arrow/array/timestamp.go:
##########
@@ -130,10 +141,15 @@ type TimestampBuilder struct {
 	dtype   *arrow.TimestampType
 	data    *memory.Buffer
 	rawData []arrow.Timestamp
+	layout  string
 }
 
 func NewTimestampBuilder(mem memory.Allocator, dtype *arrow.TimestampType) *TimestampBuilder {
-	tb := &TimestampBuilder{builder: builder{mem: mem}, dtype: dtype}
+	return NewTimestampBuilderWithLayout(mem, dtype, time.RFC3339Nano)
+}
+
+func NewTimestampBuilderWithLayout(mem memory.Allocator, dtype *arrow.TimestampType, layout string) *TimestampBuilder {

Review Comment:
   same as above, not a big fan of this naming and we should indicate in a comment that the layout is referring to the `[time.Time.Format]` layout



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat: Add support for specifying `Timestamp` `ValueStr` output layout [arrow-go]

Posted by "erezrokah (via GitHub)" <gi...@apache.org>.
erezrokah commented on PR #510:
URL: https://github.com/apache/arrow-go/pull/510#issuecomment-3308988172

   > Looks good to me, just one nitpick: can we add a JSON test which verifies the JSON marshalling output respects the layout
   
   Added a test in https://github.com/apache/arrow-go/pull/510/commits/36fd6898441bb3a60a59770686facea5013b4500, and discovered I had a redundant method `func (b *TimestampBuilder) NewTimestampArrayWithFormat(format string) (a *Timestamp)` which I removed


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat: Add support for specifying `Timestamp` `ValueStr` output layout [arrow-go]

Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade merged PR #510:
URL: https://github.com/apache/arrow-go/pull/510


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat: Add support for specifying `Timestamp` `valueStr` output format [arrow-go]

Posted by "erezrokah (via GitHub)" <gi...@apache.org>.
erezrokah commented on code in PR #510:
URL: https://github.com/apache/arrow-go/pull/510#discussion_r2352236949


##########
arrow/avro/testdata/testdata.go:
##########
@@ -51,7 +51,6 @@ type TimestampMicros int64
 
 func (t TimestampMicros) MarshalJSON() ([]byte, error) {
 	ts := time.Unix(0, int64(t)*int64(time.Microsecond)).UTC().Format(time.RFC3339Nano)
-	// arrow record marshaller trims trailing zero digits from timestamp so we do the same

Review Comment:
   This comment seems no longer relevant



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org