You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "JimWen (via GitHub)" <gi...@apache.org> on 2025/10/16 06:48:14 UTC
[I] [Bug] json fuzzy parse not work on x86 [doris]
JimWen opened a new issue, #57042:
URL: https://github.com/apache/doris/issues/57042
### Search before asking
- [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues.
### Version
2.1.6
### What's Wrong?
# x86 平台
无论是否开启 fuzzy_parse=true，数组数据 json 对象的 key 的顺序和第一行不一致，也能正常入库，耗时一致
fuzzy_parse配置无效
# arm平台(kunpeng920)
开启 fuzzy_parse=true，数组数据 json 对象的 key 的顺序和第一行不一致，无法正常入库
fuzzy_parse配置生效
x86 Platform
Regardless of whether fuzzy_parse=true is enabled, JSON array data whose object key order differs from the first row can still be imported successfully, and the load performance remains the same.
The fuzzy_parse configuration appears to have no effect.
ARM Platform (Kunpeng 920)
When fuzzy_parse=true is enabled, JSON array data whose object key order differs from the first row cannot be imported successfully.
The fuzzy_parse configuration takes effect on this platform.
### What You Expected?
希望所有平台fuzzy 都生效
hope with fuzzy_parse = true work on all platform
### How to Reproduce?
# create table
create db testdb;
CREATE TABLE testdb.test_streamload(
user_id BIGINT NOT NULL COMMENT "user id",
name VARCHAR(20) COMMENT "name",
age INT COMMENT "age"
)
DUPLICATE KEY(user_id)
DISTRIBUTED BY HASH(user_id) BUCKETS 10
PROPERTIES (
"replication_num" = "1",
"compression"="zstd"
);
# make data
vim testdb.json
[
{"user_id":4,"name":"Alexander","age":60},
{"name":"Ava","age":17,"user_id":5},
{"name":"William","age":69,"user_id":6},
]
# load test
fuzzy vs no fuzzy
curl --location-trusted -u root:test123. \
-H "Expect:100-continue" \
-H "format:json" -H "strip_outer_array:true" \
-T testdb.json \
-XPUT http://10.106.40.81:8030/api/testdb/test_streamload/_stream_load
curl --location-trusted -u root:test123. \
-H "Expect:100-continue" -H "fuzzy_parse:true" \
-H "format:json" -H "strip_outer_array:true" \
-T testdb.json \
-XPUT http://10.106.40.81:8030/api/testdb/test_streamload/_stream_load
### Anything Else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [I] [Bug] json fuzzy parse not work on x86 [doris]
Posted by "uchenily (via GitHub)" <gi...@apache.org>.
uchenily commented on issue #57042:
URL: https://github.com/apache/doris/issues/57042#issuecomment-3449752836
In master branch, the JSON reader has been refactored by simdjson, and it seems that the `fuzzy_parse` parameter no longer has any effect now.
see #54134
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [I] [Bug] json fuzzy parse not work on x86 [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #57042:
URL: https://github.com/apache/doris/issues/57042#issuecomment-4320896305
We're closing this issue because it hasn't been updated in a while.
If you'd like to revive this issue, please reopen it and feel free a maintainer to remove the Stale tag!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [I] [Bug] json fuzzy parse not work on x86 [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #57042: [Bug] json fuzzy parse not work on x86
URL: https://github.com/apache/doris/issues/57042
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org