You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@iotdb.apache.org by "betterlmy (via GitHub)" <gi...@apache.org> on 2026/03/09 14:06:24 UTC
[PR] fix: fix EOF error when decoding columns with empty string or zero po… [iotdb-client-go]
betterlmy opened a new pull request, #155:
URL: https://github.com/apache/iotdb-client-go/pull/155
# Bug 复现与修复报告
## 一、问题描述
在使用 IoTDB Go Client 查询包含空字符串（`""`）的 TEXT 列时，客户端报 `EOF` 错误，导致查询失败。同一条 SQL 在 IoTDB 命令行中执行正常。
---
## 二、复现步骤
**环境：** IoTDB 2.0.5
**第一步：写入一条 TEXT 列值为空字符串的数据**
```sql
INSERT INTO root.dev.property.`09354fae7b31c824d5888bcd184af887`.Ew7dSmO5m0PdZt7dU9sytGEyPgUlHH
(timestamp, mouldNo_value)
VALUES (now(), '');
```
**第二步：用 Go Client 查询该列**
```go
sql := "SELECT mouldNo_value FROM root.dev.property.09354fae7b31c824d5888bcd184af887.Ew7dSmO5m0PdZt7dU9sytGEyPgUlHH ORDER BY time DESC LIMIT 2"
dataset, err := session.ExecuteQueryStatement(sql, nil)
// err == nil
for {
hasNext, err := dataset.Next()
// err = EOF <-- 此处报错
}
```
**复现条件：**
- 查询列为单列 TEXT 类型
- 结果集中最后一行的该列值为空字符串 `""`（非 null）
**报错信息：**
```
EOF
```
---
## 三、根本原因分析
问题出在 `client/column_decoder.go` 的 `BinaryArrayColumnDecoder.ReadColumn`。
IoTDB 服务端对 TEXT 类型的每个值序列化格式为：
```
+---------------+-------+
| value length | value |
+---------------+-------+
| int32 | bytes |
+---------------+-------+
```
当值为空字符串时，服务端发送 `length = 0`，后跟 0 字节内容。
旧代码：
```go
var length int32
binary.Read(reader, binary.BigEndian, &length)
value := make([]byte, length) // length=0，创建空 slice
_, err = reader.Read(value) // 对空 slice 调用 Read，reader 恰好处于 EOF 边界时返回 io.EOF
```
Go 标准库 `bytes.Reader.Read` 在 reader 已无剩余字节时，即使传入空 slice（读取 0 字节），也会返回 `io.EOF`。当空字符串恰好是 TsBlock 字节流中最后一个值时，`reader.Read([]byte{})` 触发 EOF，导致解码失败。
**同时发现的第二个 bug：** 当服务端返回 `positionCount = 0` 的列时，所有类型的 ColumnDecoder 都会调用 `deserializeNullIndicators`，后者对空 reader 执行 `ReadByte()`，同样返回 `io.EOF`。
---
## 四、修复方案
**Bug 1（空字符串 EOF）：** 在 `BinaryArrayColumnDecoder` 中，当 `length == 0` 时直接创建空 Binary，跳过 `reader.Read`：
```go
if length == 0 {
values[i] = NewBinary([]byte{})
} else {
value := make([]byte, length)
_, err = reader.Read(value)
if err != nil {
return nil, err
}
values[i] = NewBinary(value)
}
```
**Bug 2（positionCount=0 EOF）：** 在四种 ColumnDecoder 的 `ReadColumn` 入口处，对 `positionCount == 0` 提前返回空 Column：
```go
if positionCount == 0 {
return NewBinaryColumn(0, 0, nil, []*Binary{})
}
```
---
## 五、验证结果
**修复前：**
```
迭代结果集失败（第 1 行）: EOF
--- FAIL: TestQueryControlSignal
```
**修复后：**
```
--- 第 1 行 --- mouldNo_value (TEXT) = ""
--- 第 2 行 --- mouldNo_value (TEXT) = ""
共返回 2 行
--- PASS: TestQueryControlSignal
```
---
## 六、新增 Feature：SessionDataSet.GetCurrentRowTime()
`SessionDataSet` 新增 `GetCurrentRowTime()` 方法，返回当前行的原始时间戳（int64，毫秒级 Unix 时间戳）。
**背景：** 原有的 `GetTimestamp(columnName)` 方法返回 `time.Time`，需要指定列名，且经过时区转换。对于需要直接操作原始时间戳数值的场景（如比较、存储、序列化），需要再做一次转换，不够便捷。
**用法：**
```go
for {
hasNext, _ := dataset.Next()
if !hasNext {
break
}
ts := dataset.GetCurrentRowTime() // 直接获取行时间戳，单位毫秒
fmt.Println(ts)
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: fix EOF error when decoding columns with empty string or zero po… [iotdb-client-go]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #155:
URL: https://github.com/apache/iotdb-client-go/pull/155#discussion_r2922961652
##########
client/session.go:
##########
@@ -569,10 +569,12 @@ func (s *Session) ExecuteQueryStatement(sql string, timeoutMs *int64) (*SessionD
request.SessionId = s.sessionId
request.StatementId = s.requestStatementId
resp, err = s.client.ExecuteQueryStatementV2(context.Background(), &request)
- if statusErr := VerifySuccess(resp.Status); statusErr == nil {
- return NewSessionDataSet(sql, resp.Columns, resp.DataTypeList, resp.ColumnNameIndexMap, *resp.QueryId, s.requestStatementId, s.client, s.sessionId, resp.QueryResult_, resp.IgnoreTimeStamp != nil && *resp.IgnoreTimeStamp, timeoutMs, *resp.MoreData, s.config.FetchSize, s.config.TimeZone, s.timeFactor, resp.GetColumnIndex2TsBlockColumnIndexList())
- } else {
- return nil, statusErr
+ if err == nil && resp != nil {
+ if statusErr := VerifySuccess(resp.Status); statusErr == nil {
+ return NewSessionDataSet(sql, resp.Columns, resp.DataTypeList, resp.ColumnNameIndexMap, *resp.QueryId, s.requestStatementId, s.client, s.sessionId, resp.QueryResult_, resp.IgnoreTimeStamp != nil && *resp.IgnoreTimeStamp, timeoutMs, *resp.MoreData, s.config.FetchSize, s.config.TimeZone, s.timeFactor, resp.GetColumnIndex2TsBlockColumnIndexList())
+ } else {
+ return nil, statusErr
+ }
}
}
return nil, err
Review Comment:
In the reconnect branch, if the second ExecuteQueryStatementV2 call returns (err == nil, resp == nil), this function falls through to `return nil, err` and ends up returning (nil, nil). Please handle the `resp == nil` case explicitly (return a non-nil error) so callers never receive a nil *SessionDataSet with a nil error.
##########
client/sessiondataset.go:
##########
@@ -125,3 +125,7 @@ func (s *SessionDataSet) GetColumnNames() []string {
func (s *SessionDataSet) GetColumnTypes() []string {
return s.ioTDBRpcDataSet.columnTypeList
}
+
Review Comment:
`GetCurrentRowTime()` forwards the raw `IoTDBRpcDataSet.time` value. That value is in the server/session timestamp precision (ms/us/ns depending on `timeFactor`), not necessarily milliseconds. To avoid API confusion, consider documenting the unit explicitly or converting to a fixed unit (e.g., always milliseconds) and/or reflecting “raw/precision-dependent” in the method name.
```suggestion
// GetCurrentRowTime returns the raw timestamp of the current row.
// The value is in the server/session timestamp precision (milliseconds,
// microseconds, or nanoseconds) depending on the configured timeFactor.
// Callers that require a fixed unit should convert this value explicitly.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: fix EOF error when decoding columns with empty string or zero po… [iotdb-client-go]
Posted by "HTHou (via GitHub)" <gi...@apache.org>.
HTHou commented on PR #155:
URL: https://github.com/apache/iotdb-client-go/pull/155#issuecomment-4052271194
我们从服务端代码分析了一下，理论上不会出现这种场景，不过加一点防御编程也没啥问题的
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: fix EOF error when decoding columns with empty string or zero po… [iotdb-client-go]
Posted by "betterlmy (via GitHub)" <gi...@apache.org>.
betterlmy commented on PR #155:
URL: https://github.com/apache/iotdb-client-go/pull/155#issuecomment-4051793765
bug1是实际生产中遇到的，bug2是Claude帮忙分析bug1时顺路发现的问题，暂时没可以复现的方法，稍等我今晚下班后研究一下
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: fix EOF error when decoding columns with empty string or zero po… [iotdb-client-go]
Posted by "shuwenwei (via GitHub)" <gi...@apache.org>.
shuwenwei commented on PR #155:
URL: https://github.com/apache/iotdb-client-go/pull/155#issuecomment-4051751735
能提供一下 Bug2 的复现方法吗
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: fix EOF error when decoding columns with empty string or zero po… [iotdb-client-go]
Posted by "betterlmy (via GitHub)" <gi...@apache.org>.
betterlmy commented on PR #155:
URL: https://github.com/apache/iotdb-client-go/pull/155#issuecomment-4028787437
@HTHou 帮忙看一下这个pr💗
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: fix EOF error when decoding columns with empty string or zero po… [iotdb-client-go]
Posted by "HTHou (via GitHub)" <gi...@apache.org>.
HTHou commented on PR #155:
URL: https://github.com/apache/iotdb-client-go/pull/155#issuecomment-4051784795
@betterlmy 我们暂时没发现 bug 2 中提到的导致 positionCount == 0 的场景，可以分享下吗
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: fix EOF error when decoding columns with empty string or zero po… [iotdb-client-go]
Posted by "jt2594838 (via GitHub)" <gi...@apache.org>.
jt2594838 merged PR #155:
URL: https://github.com/apache/iotdb-client-go/pull/155
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: fix EOF error when decoding columns with empty string or zero po… [iotdb-client-go]
Posted by "betterlmy (via GitHub)" <gi...@apache.org>.
betterlmy commented on code in PR #155:
URL: https://github.com/apache/iotdb-client-go/pull/155#discussion_r2923369655
##########
client/session.go:
##########
@@ -569,10 +569,12 @@ func (s *Session) ExecuteQueryStatement(sql string, timeoutMs *int64) (*SessionD
request.SessionId = s.sessionId
request.StatementId = s.requestStatementId
resp, err = s.client.ExecuteQueryStatementV2(context.Background(), &request)
- if statusErr := VerifySuccess(resp.Status); statusErr == nil {
- return NewSessionDataSet(sql, resp.Columns, resp.DataTypeList, resp.ColumnNameIndexMap, *resp.QueryId, s.requestStatementId, s.client, s.sessionId, resp.QueryResult_, resp.IgnoreTimeStamp != nil && *resp.IgnoreTimeStamp, timeoutMs, *resp.MoreData, s.config.FetchSize, s.config.TimeZone, s.timeFactor, resp.GetColumnIndex2TsBlockColumnIndexList())
- } else {
- return nil, statusErr
+ if err == nil && resp != nil {
+ if statusErr := VerifySuccess(resp.Status); statusErr == nil {
+ return NewSessionDataSet(sql, resp.Columns, resp.DataTypeList, resp.ColumnNameIndexMap, *resp.QueryId, s.requestStatementId, s.client, s.sessionId, resp.QueryResult_, resp.IgnoreTimeStamp != nil && *resp.IgnoreTimeStamp, timeoutMs, *resp.MoreData, s.config.FetchSize, s.config.TimeZone, s.timeFactor, resp.GetColumnIndex2TsBlockColumnIndexList())
+ } else {
+ return nil, statusErr
+ }
}
}
return nil, err
Review Comment:
Fixed in 869634829f759dbeb10ca4ca6300231b50e43995.
Added an explicit resp == nil check in all four reconnect branches (ExecuteQueryStatement, ExecuteAggregationQuery, ExecuteAggregationQueryWithLegalNodes, ExecuteFastLastDataQueryForOnePrefixPath). When the second call returns err == nil but resp == nil, the function now returns a non-nil error instead of falling through to return nil, nil.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: fix EOF error when decoding columns with empty string or zero po… [iotdb-client-go]
Posted by "betterlmy (via GitHub)" <gi...@apache.org>.
betterlmy commented on code in PR #155:
URL: https://github.com/apache/iotdb-client-go/pull/155#discussion_r2923370903
##########
client/sessiondataset.go:
##########
@@ -125,3 +125,7 @@ func (s *SessionDataSet) GetColumnNames() []string {
func (s *SessionDataSet) GetColumnTypes() []string {
return s.ioTDBRpcDataSet.columnTypeList
}
+
Review Comment:
Fixed in 51449e79b7ef4acbf8c3345c6753a75b06c72fe4.
SessionDataSet.GetCurrentRowTime() now returns time.Time instead of the raw int64 timestamp. The underlying IoTDBRpcDataSet.GetCurrentRowTime() is kept as int64 for internal use; the public-facing method converts it via convertToTimestamp(rawTime, timeFactor), which normalizes across all server timestamp precisions (ms/us/ns).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org