You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "davidhcoe (via GitHub)" <gi...@apache.org> on 2025/06/13 20:04:32 UTC
[PR] fix(go/adbc/driver/snowflake): The Timestamp values in the JSON path [arrow-adbc]
davidhcoe opened a new pull request, #2965:
URL: https://github.com/apache/arrow-adbc/pull/2965
After the update to adjust the Timestamp precision, the JSON path (which is hit when using SHOW commands) was still using nanoseconds even though the schema was specifying the precision as microseconds, resulting in downstream C# callers getting the error
```
System.ArgumentOutOfRangeException: Ticks must be between DateTime.MinValue.Ticks and DateTime.MaxValue.Ticks.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(go/adbc/driver/snowflake): Adjust the precision of the Timestamp values in the JSON-only path [arrow-adbc]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on code in PR #2965:
URL: https://github.com/apache/arrow-adbc/pull/2965#discussion_r2147453833
##########
go/adbc/driver/snowflake/driver_test.go:
##########
@@ -1724,6 +1724,118 @@ func (suite *SnowflakeTests) TestBooleanType() {
}
}
+func (suite *SnowflakeTests) TestTimestampPrecisionJson() {
+ opts := suite.Quirks.DatabaseOptions()
+ opts[driver.OptionMaxTimestampPrecision] = "microseconds"
+
+ db, err := suite.driver.NewDatabase(opts)
+ suite.NoError(err)
+ defer validation.CheckedClose(suite.T(), db)
+ cnxn, err := db.Open(suite.ctx)
+ suite.NoError(err)
+ defer validation.CheckedClose(suite.T(), cnxn)
+ stmt, _ := cnxn.NewStatement()
+
+ id := uuid.New()
+ tempTable := "pk_" + strings.ReplaceAll(id.String(), "-", "")
+
+ query := fmt.Sprintf(`CREATE OR REPLACE TABLE %s.%s.%s (
+ id INT PRIMARY KEY,
+ name STRING);`, suite.Quirks.catalogName, suite.Quirks.schemaName, tempTable)
+
+ suite.Require().NoError(stmt.SetSqlQuery(query))
+ _, err = stmt.ExecuteUpdate(suite.ctx)
+ suite.Require().NoError(err)
+
+ query = fmt.Sprintf("SHOW PRIMARY KEYS IN TABLE %s.%s.%s", suite.Quirks.catalogName, suite.Quirks.schemaName, tempTable)
+ stmt, _ = cnxn.NewStatement()
+ suite.Require().NoError(stmt.SetSqlQuery(query))
+ rdr, _, err := stmt.ExecuteQuery(suite.ctx)
+ defer rdr.Release()
+ suite.Require().NoError(err)
+
+ suite.True(rdr.Next())
+ rec := rdr.Record()
+
+ result := validateRecord(
+ rec,
+ suite.Quirks.catalogName,
+ suite.Quirks.schemaName,
+ tempTable,
+ )
+
+ suite.True(result, "Expected primary key record not found")
+
+ query = fmt.Sprintf("DROP TABLE %s.%s.%s", suite.Quirks.catalogName, suite.Quirks.schemaName, tempTable)
+ stmt, _ = cnxn.NewStatement()
+ suite.Require().NoError(stmt.SetSqlQuery(query))
+ _, err = stmt.ExecuteUpdate(suite.ctx)
+ defer rdr.Release()
+ suite.Require().NoError(err)
+}
+
+func validateRecord(
+ rec arrow.Record,
+ catalogName string,
+ schemaName string,
+ tableName string,
+) bool {
+ // Get column indexes
+ getColIdx := func(name string) int {
+ for i := 0; i < int(rec.NumCols()); i++ {
+ if rec.ColumnName(i) == name {
+ return i
+ }
+ }
+ return -1
+ }
+
+ // Expected column names
+ dbIdx := getColIdx("database_name")
+ schemaIdx := getColIdx("schema_name")
+ tableIdx := getColIdx("table_name")
+ colIdx := getColIdx("column_name")
+ seqIdx := getColIdx("key_sequence")
+ createdIdx := getColIdx("created_on")
+
+ if dbIdx == -1 || schemaIdx == -1 || tableIdx == -1 || colIdx == -1 || seqIdx == -1 || createdIdx == -1 {
+ fmt.Println("Missing expected columns")
+ return false
+ }
+
+ numRows := int(rec.NumRows())
+ for i := 0; i < numRows; i++ {
+ dbName := rec.Column(dbIdx).(*array.String).Value(i)
+ schema := rec.Column(schemaIdx).(*array.String).Value(i)
+ tbl := rec.Column(tableIdx).(*array.String).Value(i)
+ column := rec.Column(colIdx).(*array.String).Value(i)
+ keySeq := rec.Column(seqIdx).(*array.Int64).Value(i)
+
+ // Created_on should be a timestamp array
+ createdCol := rec.Column(createdIdx).(*array.Timestamp)
+ created := time.Unix(0, int64(createdCol.Value(i))*int64(time.Microsecond)).UTC()
+
+ // Perform checks
+ if strings.EqualFold(dbName, catalogName) &&
+ strings.EqualFold(schema, schemaName) &&
+ strings.EqualFold(tbl, tableName) &&
+ strings.EqualFold(column, "id") &&
+ keySeq == 1 {
+
+ now := time.Now().UTC()
+ diff := now.Sub(created)
+ if diff < 0 {
+ diff = -diff
+ }
+ if diff <= 2*time.Minute {
+ return true
+ }
+ }
+ }
+
+ return false
Review Comment:
Is it intended for this loop to stop once it finds one matching row?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(go/adbc/driver/snowflake): Adjust the precision of the Timestamp values in the JSON-only path [arrow-adbc]
Posted by "CurtHagenlocher (via GitHub)" <gi...@apache.org>.
CurtHagenlocher commented on PR #2965:
URL: https://github.com/apache/arrow-adbc/pull/2965#issuecomment-2972987608
I'm not sure how it can be said that the problem doesn't exist in Go; if the fix is in the driver then the problem is in the driver. If the schema reported by the Arrow array stream is different than the schema reported by the record batch, then that's the bug and it can be tested for in Go.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(go/adbc/driver/snowflake): Adjust the precision of the Timestamp values in the JSON-only path [arrow-adbc]
Posted by "CurtHagenlocher (via GitHub)" <gi...@apache.org>.
CurtHagenlocher commented on code in PR #2965:
URL: https://github.com/apache/arrow-adbc/pull/2965#discussion_r2146209223
##########
csharp/test/Apache.Arrow.Adbc.Tests/ColumnNetTypeArrowTypeValue.cs:
##########
@@ -32,12 +41,38 @@ public ColumnNetTypeArrowTypeValue(string name, Type expectedNetType, Type expec
this.ExpectedValue = expectedValue;
}
+ /// <summary>
+ /// Instantiates <see cref="ColumnNetTypeArrowTypeValue"/>.
+ /// </summary>
+ /// <param name="name">The column name</param>
+ /// <param name="expectedNetType">The expected .NET type</param>
+ /// <param name="expectedArrowArrayType">The expected Arrow type</param>
+ /// <param name="expectedValue">The expected value</param>
+ /// <param name="isValid">A function that can be run to compare values</param>
+ /// <exception cref="ArgumentNullException"></exception>
+ public ColumnNetTypeArrowTypeValue(string name, Type expectedNetType, Type expectedArrowArrayType, bool expectedValue, Func<object?, bool> isValid)
+ {
+ this.Name = name;
+ this.ExpectedNetType = expectedNetType;
+ this.ExpectedArrowArrayType = expectedArrowArrayType;
+ this.ExpectedValue = expectedValue;
+
+ _isValid = isValid ?? throw new ArgumentNullException(nameof(isValid));
+ }
+
public string Name { get; set; }
public Type ExpectedNetType { get; set; }
public Type ExpectedArrowArrayType { get; set; }
public object? ExpectedValue { get; set; }
+
+ public bool IsCalcuatedResult => _isValid != null;
Review Comment:
Typo
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(go/adbc/driver/snowflake): Adjust the precision of the Timestamp values in the JSON-only path [arrow-adbc]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on code in PR #2965:
URL: https://github.com/apache/arrow-adbc/pull/2965#discussion_r2148856333
##########
go/adbc/driver/snowflake/driver_test.go:
##########
@@ -1724,6 +1724,100 @@ func (suite *SnowflakeTests) TestBooleanType() {
}
}
+func (suite *SnowflakeTests) TestTimestampPrecisionJson() {
+ opts := suite.Quirks.DatabaseOptions()
+ opts[driver.OptionMaxTimestampPrecision] = "microseconds"
+
+ db, err := suite.driver.NewDatabase(opts)
+ suite.NoError(err)
+ defer validation.CheckedClose(suite.T(), db)
+ cnxn, err := db.Open(suite.ctx)
+ suite.NoError(err)
+ defer validation.CheckedClose(suite.T(), cnxn)
+ stmt, _ := cnxn.NewStatement()
+
+ id := uuid.New()
+ tempTable := "pk_" + strings.ReplaceAll(id.String(), "-", "")
+
+ query := fmt.Sprintf(`CREATE OR REPLACE TABLE %s.%s.%s (
+ id INT PRIMARY KEY,
+ name STRING);`, suite.Quirks.catalogName, suite.Quirks.schemaName, tempTable)
+
+ suite.Require().NoError(stmt.SetSqlQuery(query))
+ _, err = stmt.ExecuteUpdate(suite.ctx)
+ suite.Require().NoError(err)
+
+ query = fmt.Sprintf("SHOW PRIMARY KEYS IN TABLE %s.%s.%s", suite.Quirks.catalogName, suite.Quirks.schemaName, tempTable)
+ stmt, _ = cnxn.NewStatement()
+ suite.Require().NoError(stmt.SetSqlQuery(query))
+ rdr, _, err := stmt.ExecuteQuery(suite.ctx)
+ defer rdr.Release()
+ suite.Require().NoError(err)
+
+ suite.True(rdr.Next())
+ rec := rdr.Record()
+
+ suite.True(rec.NumRows() == 1)
+
+ // Get column indexes
+ getColIdx := func(name string) int {
+ for i := 0; i < int(rec.NumCols()); i++ {
+ if rec.ColumnName(i) == name {
+ return i
+ }
+ }
+ return -1
+ }
+
+ // Expected column names
+ dbIdx := getColIdx("database_name")
+ schemaIdx := getColIdx("schema_name")
+ tableIdx := getColIdx("table_name")
+ colIdx := getColIdx("column_name")
+ seqIdx := getColIdx("key_sequence")
+ createdIdx := getColIdx("created_on")
+
+ if dbIdx == -1 || schemaIdx == -1 || tableIdx == -1 || colIdx == -1 || seqIdx == -1 || createdIdx == -1 {
Review Comment:
suite.Equal instead of panic?
Or just panic inside of getColIndex with the column name.
##########
go/adbc/driver/snowflake/driver_test.go:
##########
@@ -1724,6 +1724,100 @@ func (suite *SnowflakeTests) TestBooleanType() {
}
}
+func (suite *SnowflakeTests) TestTimestampPrecisionJson() {
+ opts := suite.Quirks.DatabaseOptions()
+ opts[driver.OptionMaxTimestampPrecision] = "microseconds"
+
+ db, err := suite.driver.NewDatabase(opts)
+ suite.NoError(err)
+ defer validation.CheckedClose(suite.T(), db)
+ cnxn, err := db.Open(suite.ctx)
+ suite.NoError(err)
+ defer validation.CheckedClose(suite.T(), cnxn)
+ stmt, _ := cnxn.NewStatement()
+
+ id := uuid.New()
+ tempTable := "pk_" + strings.ReplaceAll(id.String(), "-", "")
+
+ query := fmt.Sprintf(`CREATE OR REPLACE TABLE %s.%s.%s (
+ id INT PRIMARY KEY,
+ name STRING);`, suite.Quirks.catalogName, suite.Quirks.schemaName, tempTable)
+
+ suite.Require().NoError(stmt.SetSqlQuery(query))
+ _, err = stmt.ExecuteUpdate(suite.ctx)
+ suite.Require().NoError(err)
+
+ query = fmt.Sprintf("SHOW PRIMARY KEYS IN TABLE %s.%s.%s", suite.Quirks.catalogName, suite.Quirks.schemaName, tempTable)
+ stmt, _ = cnxn.NewStatement()
+ suite.Require().NoError(stmt.SetSqlQuery(query))
+ rdr, _, err := stmt.ExecuteQuery(suite.ctx)
+ defer rdr.Release()
+ suite.Require().NoError(err)
+
+ suite.True(rdr.Next())
+ rec := rdr.Record()
+
+ suite.True(rec.NumRows() == 1)
+
+ // Get column indexes
+ getColIdx := func(name string) int {
+ for i := 0; i < int(rec.NumCols()); i++ {
+ if rec.ColumnName(i) == name {
+ return i
+ }
+ }
+ return -1
+ }
+
+ // Expected column names
+ dbIdx := getColIdx("database_name")
+ schemaIdx := getColIdx("schema_name")
+ tableIdx := getColIdx("table_name")
+ colIdx := getColIdx("column_name")
+ seqIdx := getColIdx("key_sequence")
+ createdIdx := getColIdx("created_on")
+
+ if dbIdx == -1 || schemaIdx == -1 || tableIdx == -1 || colIdx == -1 || seqIdx == -1 || createdIdx == -1 {
+ panic("Missing expected columns")
+ }
+
+ i := 0
+ dbName := rec.Column(dbIdx).(*array.String).Value(i)
+ schema := rec.Column(schemaIdx).(*array.String).Value(i)
+ tbl := rec.Column(tableIdx).(*array.String).Value(i)
+ column := rec.Column(colIdx).(*array.String).Value(i)
+ keySeq := rec.Column(seqIdx).(*array.Int64).Value(i)
+
+ // Created_on should be a timestamp array
+ createdCol := rec.Column(createdIdx).(*array.Timestamp)
+ created := time.Unix(0, int64(createdCol.Value(i))*int64(time.Microsecond)).UTC()
+
+ // Perform checks
+ if strings.EqualFold(dbName, suite.Quirks.catalogName) &&
+ strings.EqualFold(schema, suite.Quirks.schemaName) &&
+ strings.EqualFold(tbl, tempTable) &&
+ strings.EqualFold(column, "id") &&
+ keySeq == 1 {
+
+ now := time.Now().UTC()
+ diff := now.Sub(created)
+ if diff < 0 {
+ diff = -diff
+ }
+ // since this was just created, make sure the times are within a short difference
+ suite.Assert().True(diff <= 2*time.Minute)
+ } else {
+ panic("Invalid values")
+ }
+
+ query = fmt.Sprintf("DROP TABLE %s.%s.%s", suite.Quirks.catalogName, suite.Quirks.schemaName, tempTable)
+ stmt, _ = cnxn.NewStatement()
Review Comment:
This isn't being cleaned up, though I suppose it doesn't seem to matter
##########
go/adbc/driver/snowflake/driver_test.go:
##########
@@ -1724,6 +1724,100 @@ func (suite *SnowflakeTests) TestBooleanType() {
}
}
+func (suite *SnowflakeTests) TestTimestampPrecisionJson() {
+ opts := suite.Quirks.DatabaseOptions()
+ opts[driver.OptionMaxTimestampPrecision] = "microseconds"
+
+ db, err := suite.driver.NewDatabase(opts)
+ suite.NoError(err)
+ defer validation.CheckedClose(suite.T(), db)
+ cnxn, err := db.Open(suite.ctx)
+ suite.NoError(err)
+ defer validation.CheckedClose(suite.T(), cnxn)
+ stmt, _ := cnxn.NewStatement()
+
+ id := uuid.New()
+ tempTable := "pk_" + strings.ReplaceAll(id.String(), "-", "")
+
+ query := fmt.Sprintf(`CREATE OR REPLACE TABLE %s.%s.%s (
+ id INT PRIMARY KEY,
+ name STRING);`, suite.Quirks.catalogName, suite.Quirks.schemaName, tempTable)
+
+ suite.Require().NoError(stmt.SetSqlQuery(query))
+ _, err = stmt.ExecuteUpdate(suite.ctx)
+ suite.Require().NoError(err)
+
+ query = fmt.Sprintf("SHOW PRIMARY KEYS IN TABLE %s.%s.%s", suite.Quirks.catalogName, suite.Quirks.schemaName, tempTable)
+ stmt, _ = cnxn.NewStatement()
+ suite.Require().NoError(stmt.SetSqlQuery(query))
+ rdr, _, err := stmt.ExecuteQuery(suite.ctx)
+ defer rdr.Release()
+ suite.Require().NoError(err)
+
+ suite.True(rdr.Next())
+ rec := rdr.Record()
+
+ suite.True(rec.NumRows() == 1)
+
+ // Get column indexes
+ getColIdx := func(name string) int {
+ for i := 0; i < int(rec.NumCols()); i++ {
+ if rec.ColumnName(i) == name {
+ return i
+ }
+ }
+ return -1
+ }
+
+ // Expected column names
+ dbIdx := getColIdx("database_name")
+ schemaIdx := getColIdx("schema_name")
+ tableIdx := getColIdx("table_name")
+ colIdx := getColIdx("column_name")
+ seqIdx := getColIdx("key_sequence")
+ createdIdx := getColIdx("created_on")
+
+ if dbIdx == -1 || schemaIdx == -1 || tableIdx == -1 || colIdx == -1 || seqIdx == -1 || createdIdx == -1 {
+ panic("Missing expected columns")
+ }
+
+ i := 0
+ dbName := rec.Column(dbIdx).(*array.String).Value(i)
+ schema := rec.Column(schemaIdx).(*array.String).Value(i)
+ tbl := rec.Column(tableIdx).(*array.String).Value(i)
+ column := rec.Column(colIdx).(*array.String).Value(i)
+ keySeq := rec.Column(seqIdx).(*array.Int64).Value(i)
+
+ // Created_on should be a timestamp array
+ createdCol := rec.Column(createdIdx).(*array.Timestamp)
+ created := time.Unix(0, int64(createdCol.Value(i))*int64(time.Microsecond)).UTC()
+
+ // Perform checks
+ if strings.EqualFold(dbName, suite.Quirks.catalogName) &&
+ strings.EqualFold(schema, suite.Quirks.schemaName) &&
+ strings.EqualFold(tbl, tempTable) &&
+ strings.EqualFold(column, "id") &&
+ keySeq == 1 {
+
+ now := time.Now().UTC()
+ diff := now.Sub(created)
+ if diff < 0 {
+ diff = -diff
+ }
+ // since this was just created, make sure the times are within a short difference
+ suite.Assert().True(diff <= 2*time.Minute)
+ } else {
+ panic("Invalid values")
+ }
+
+ query = fmt.Sprintf("DROP TABLE %s.%s.%s", suite.Quirks.catalogName, suite.Quirks.schemaName, tempTable)
+ stmt, _ = cnxn.NewStatement()
+ suite.Require().NoError(stmt.SetSqlQuery(query))
+ _, err = stmt.ExecuteUpdate(suite.ctx)
+ defer rdr.Release()
Review Comment:
Isn't this a double release? And why is it all the way down here?
##########
go/adbc/driver/snowflake/driver_test.go:
##########
@@ -1724,6 +1724,100 @@ func (suite *SnowflakeTests) TestBooleanType() {
}
}
+func (suite *SnowflakeTests) TestTimestampPrecisionJson() {
+ opts := suite.Quirks.DatabaseOptions()
+ opts[driver.OptionMaxTimestampPrecision] = "microseconds"
+
+ db, err := suite.driver.NewDatabase(opts)
+ suite.NoError(err)
+ defer validation.CheckedClose(suite.T(), db)
+ cnxn, err := db.Open(suite.ctx)
+ suite.NoError(err)
+ defer validation.CheckedClose(suite.T(), cnxn)
+ stmt, _ := cnxn.NewStatement()
+
+ id := uuid.New()
+ tempTable := "pk_" + strings.ReplaceAll(id.String(), "-", "")
+
+ query := fmt.Sprintf(`CREATE OR REPLACE TABLE %s.%s.%s (
+ id INT PRIMARY KEY,
+ name STRING);`, suite.Quirks.catalogName, suite.Quirks.schemaName, tempTable)
+
+ suite.Require().NoError(stmt.SetSqlQuery(query))
+ _, err = stmt.ExecuteUpdate(suite.ctx)
+ suite.Require().NoError(err)
+
+ query = fmt.Sprintf("SHOW PRIMARY KEYS IN TABLE %s.%s.%s", suite.Quirks.catalogName, suite.Quirks.schemaName, tempTable)
+ stmt, _ = cnxn.NewStatement()
+ suite.Require().NoError(stmt.SetSqlQuery(query))
+ rdr, _, err := stmt.ExecuteQuery(suite.ctx)
+ defer rdr.Release()
+ suite.Require().NoError(err)
+
+ suite.True(rdr.Next())
+ rec := rdr.Record()
+
+ suite.True(rec.NumRows() == 1)
Review Comment:
suite.Equal?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(go/adbc/driver/snowflake): Adjust the precision of the Timestamp values in the JSON-only path [arrow-adbc]
Posted by "davidhcoe (via GitHub)" <gi...@apache.org>.
davidhcoe commented on PR #2965:
URL: https://github.com/apache/arrow-adbc/pull/2965#issuecomment-2971884204
> I think this should have a test in Go. That probably requires creating a table that has a primary key? There should be existing tests for returning a schema whose setup can be reused for that, I think.
I started in Go and deleted it because it was specific to a single instance. I don’t think it would be too hard to do in Go what exists in C#.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(go/adbc/driver/snowflake): Adjust the precision of the Timestamp values in the JSON-only path [arrow-adbc]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on PR #2965:
URL: https://github.com/apache/arrow-adbc/pull/2965#issuecomment-2973647216
If there's only one row in the result, can we assert that fact? The loop is rather misleading in this case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(go/adbc/driver/snowflake): Adjust the precision of the Timestamp values in the JSON-only path [arrow-adbc]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm merged PR #2965:
URL: https://github.com/apache/arrow-adbc/pull/2965
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(go/adbc/driver/snowflake): Adjust the precision of the Timestamp values in the JSON-only path [arrow-adbc]
Posted by "davidhcoe (via GitHub)" <gi...@apache.org>.
davidhcoe commented on PR #2965:
URL: https://github.com/apache/arrow-adbc/pull/2965#issuecomment-2972924743
> > I think this should have a test in Go. That probably requires creating a table that has a primary key? There should be existing tests for returning a schema whose setup can be reused for that, I think.
>
> I started in Go and deleted it because it was specific to a single instance. I don’t think it would be too hard to do in Go what exists in C#.
Actually, the problem doesn't occur in Go. The problem was happening because the schema was reported by the Go driver as microseconds and the value was in nanoseconds, so when the IArrowArray.ValueAt method was called, it was failing on the internal ValueConverter at:
```
ArrowTypeId.Timestamp:
return (array, index) => ((TimestampArray)array).GetTimestamp(index);
```
which goes to the internal TimstampArray here:
```
switch (type.Unit)
{
case TimeUnit.Nanosecond:
ticks = value / 100;
break;
case TimeUnit.Microsecond:
ticks = value * 10; // <------ was incorrect because unit was microsecond and value was nanosecond
break;
```
All of that to say, I don't know what a separate Go test will really do for that scenario.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(go/adbc/driver/snowflake): Adjust the precision of the Timestamp values in the JSON-only path [arrow-adbc]
Posted by "davidhcoe (via GitHub)" <gi...@apache.org>.
davidhcoe commented on PR #2965:
URL: https://github.com/apache/arrow-adbc/pull/2965#issuecomment-2973645258
Yes, but there is only one row in the result.
________________________________
From: David Li ***@***.***>
Sent: Sunday, June 15, 2025 2:20:04 AM
To: apache/arrow-adbc ***@***.***>
Cc: David Coe ***@***.***>; Author ***@***.***>
Subject: Re: [apache/arrow-adbc] fix(go/adbc/driver/snowflake): Adjust the precision of the Timestamp values in the JSON-only path (PR #2965)
@lidavidm commented on this pull request.
________________________________
In go/adbc/driver/snowflake/driver_test.go<https://github.com/apache/arrow-adbc/pull/2965#discussion_r2147453833>:
> + numRows := int(rec.NumRows())
+ for i := 0; i < numRows; i++ {
+ dbName := rec.Column(dbIdx).(*array.String).Value(i)
+ schema := rec.Column(schemaIdx).(*array.String).Value(i)
+ tbl := rec.Column(tableIdx).(*array.String).Value(i)
+ column := rec.Column(colIdx).(*array.String).Value(i)
+ keySeq := rec.Column(seqIdx).(*array.Int64).Value(i)
+
+ // Created_on should be a timestamp array
+ createdCol := rec.Column(createdIdx).(*array.Timestamp)
+ created := time.Unix(0, int64(createdCol.Value(i))*int64(time.Microsecond)).UTC()
+
+ // Perform checks
+ if strings.EqualFold(dbName, catalogName) &&
+ strings.EqualFold(schema, schemaName) &&
+ strings.EqualFold(tbl, tableName) &&
+ strings.EqualFold(column, "id") &&
+ keySeq == 1 {
+
+ now := time.Now().UTC()
+ diff := now.Sub(created)
+ if diff < 0 {
+ diff = -diff
+ }
+ if diff <= 2*time.Minute {
+ return true
+ }
+ }
+ }
+
+ return false
Is it intended for this loop to stop once it finds one matching row?
—
Reply to this email directly, view it on GitHub<https://github.com/apache/arrow-adbc/pull/2965#pullrequestreview-2928997494>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ADFTVNOQSUARO37AOZN6NKT3DUGBJAVCNFSM6AAAAAB7I66K46VHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDSMRYHE4TONBZGQ>.
You are receiving this because you authored the thread.Message ID: ***@***.***>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org