You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "amorynan (via GitHub)" <gi...@apache.org> on 2024/09/25 10:03:19 UTC
[PR] [fix](new_json_reader)fix new_json_reader core [doris]
amorynan opened a new pull request, #41290:
URL: https://github.com/apache/doris/pull/41290
## Proposed changes
if we put not json object or json array in json file with select tvf , here will meet core like :
```
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory]
[WARNING!] /sys/kernel/mm/transparent_hugepage/enabled: [always] madvise never, Doris not recommend turning on THP, which may cause the BE process to use more memory and cannot be freed in time. Turn off THP: `echo madvise | sudo tee /sys/kernel/mm/transparent_hugepage/enabled`
start BE in local mode
doris_be: /mnt/disk1/wangqiannan/amory/doris/thirdparty/installed/include/rapidjson/document.h:1178：SizeType rapidjson::GenericValue<rapidjson::UTF8<>>::MemberCount() const [Encoding = rapidjson::UTF8<>, Allocator = rapidjson::MemoryPoolAllocator<>]: 假设 ‘IsObject()’ 失
败。
*** Query id: 0-0 ***
*** is nereids: 0 ***
*** tablet id: 0 ***
*** Aborted at 1727253403 (unix time) try "date -d @1727253403" if you are using GNU date ***
*** Current BE git commitID: daf0fa72c5 ***
*** SIGABRT unknown detail explain (@0x461000ac22f) received by PID 705071 (TID 711574 OR 0x7f35fb40a700) from PID 705071; stack trace: ***
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /mnt/disk1/wangqiannan/amory/doris/be/src/common/signal_handler.h:421
1# 0x00007F3A3902EB50 in /lib64/libc.so.6
2# gsignal in /lib64/libc.so.6
3# __GI_abort in /lib64/libc.so.6
4# _nl_load_domain.cold.0 in /lib64/libc.so.6
5# 0x00007F3A39027426 in /lib64/libc.so.6
6# rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >::MemberCount() const at /mnt/disk1/wangqiannan/amory/doris/thirdparty/installed/include/rapidjson/document.h:1178
7# doris::vectorized::NewJsonReader::get_parsed_schema(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >*, std::vector<doris::TypeDescriptor, std::allocator<doris::TypeDescriptor> >*) at /mnt/disk1/wangqiannan/amory/doris/be/src/vec/exec/format/json/new_json_reader.cpp:347
8# doris::PInternalService::fetch_table_schema(google::protobuf::RpcController*, doris::PFetchTableSchemaRequest const*, doris::PFetchTableSchemaResult*, google::protobuf::Closure*)::$_0::operator()() const at /mnt/disk1/wangqiannan/amory/doris/be/src/service/internal_service.cpp:832
9# void std::__invoke_impl<void, doris::PInternalService::fetch_table_schema(google::protobuf::RpcController*, doris::PFetchTableSchemaRequest const*, doris::PFetchTableSchemaResult*, google::protobuf::Closure*)::$_0&>(std::__invoke_other, doris::PInternalService::fetch_table_schema(google::protobuf::RpcController*, doris::PFetchTableSchemaRequest const*, doris::PFetchTableSchemaResult*, google::protobuf::Closure*)::$_0&) at /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61
10# std::enable_if<is_invocable_r_v<void, doris::PInternalService::fetch_table_schema(google::protobuf::RpcController*, doris::PFetchTableSchemaRequest const*, doris::PFetchTableSchemaResult*, google::protobuf::Closure*)::$_0&>, void>::type std::__invoke_r<void, doris::PInternalService::fetch_table_schema(google::protobuf::RpcController*, doris::PFetchTableSchemaRequest const*, doris::PFetchTableSchemaResult*, google::protobuf::Closure*)::$_0&>(doris::PInternalService::fetch_table_schema(google::protobuf::RpcController*, doris::PFetchTableSchemaRequest const*, doris::PFetchTableSchemaResult*, google::protobuf::Closure*)::$_0&) at /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:117
11# std::_Function_handler<void (), doris::PInternalService::fetch_table_schema(google::protobuf::RpcController*, doris::PFetchTableSchemaRequest const*, doris::PFetchTableSchemaResult*, google::protobuf::Closure*)::$_0>::_M_invoke(std::_Any_data const&) at /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
12# std::function<void ()>::operator()() const at /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560
13# doris::WorkThreadPool<false>::work_thread(int) at /mnt/disk1/wangqiannan/amory/doris/be/src/util/work_thread_pool.hpp:158
14# void std::__invoke_impl<void, void (doris::WorkThreadPool<false>::* const&)(int), doris::WorkThreadPool<false>*&, int&>(std::__invoke_memfun_deref, void (doris::WorkThreadPool<false>::* const&)(int), doris::WorkThreadPool<false>*&, int&) at /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:74
15# std::__invoke_result<void (doris::WorkThreadPool<false>::* const&)(int), doris::WorkThreadPool<false>*&, int&>::type std::__invoke<void (doris::WorkThreadPool<false>::* const&)(int), doris::WorkThreadPool<false>*&, int&>(void (doris::WorkThreadPool<false>::* const&)(int), doris::WorkThreadPool<false>*&, int&) at /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:96
16# decltype (std::__invoke((*this)._M_pmf, std::forward<doris::WorkThreadPool<false>*&>({parm#1}), std::forward<int&>({parm#1}))) std::_Mem_fn_base<void (doris::WorkThreadPool<false>::*)(int), true>::operator()<doris::WorkThreadPool<false>*&, int&>(doris::WorkThreadPool<false>*&, int&) const at /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/functional:131
17# void std::__invoke_impl<void, std::_Mem_fn<void (doris::WorkThreadPool<false>::*)(int)>&, doris::WorkThreadPool<false>*&, int&>(std::__invoke_other, std::_Mem_fn<void (doris::WorkThreadPool<false>::*)(int)>&, doris::WorkThreadPool<false>*&, int&) at /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61
18# std::enable_if<is_invocable_r_v<void, std::_Mem_fn<void (doris::WorkThreadPool<false>::*)(int)>&, doris::WorkThreadPool<false>*&, int&>, void>::type std::__invoke_r<void, std::_Mem_fn<void (doris::WorkThreadPool<false>::*)(int)>&, doris::WorkThreadPool<false>*&, int&>(std::_Mem_fn<void (doris::WorkThreadPool<false>::*)(int)>&, doris::WorkThreadPool<false>*&, int&) at /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:117
19# void std::_Bind_result<void, std::_Mem_fn<void (doris::WorkThreadPool<false>::*)(int)> (doris::WorkThreadPool<false>*, int)>::__call<void, , 0ul, 1ul>(std::tuple<>&&, std::_Index_tuple<0ul, 1ul>) at /mnt/disk1/wangqiannan/tool/ldb_toolchain_16/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/functional:570
```
Issue Number: close #xxx
<!--Describe your changes.-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](new_json_reader)fix new_json_reader core [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #41290:
URL: https://github.com/apache/doris/pull/41290#issuecomment-2453395487
TeamCity be ut coverage result:
Function Coverage: 37.82% (9821/25967)
Line Coverage: 28.98% (81597/281567)
Region Coverage: 28.26% (42147/149155)
Branch Coverage: 24.84% (21386/86108)
Coverage Report: http://coverage.selectdb-in.cc/coverage/9a95c48857dcc2b12443b116d7d047b6dbb6187c_9a95c48857dcc2b12443b116d7d047b6dbb6187c/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](new_json_reader)fix new_json_reader core [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #41290:
URL: https://github.com/apache/doris/pull/41290#issuecomment-2373634148
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](new_json_reader)fix new_json_reader core [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #41290:
URL: https://github.com/apache/doris/pull/41290#issuecomment-2451615762
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](new_json_reader)fix new_json_reader core [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #41290:
URL: https://github.com/apache/doris/pull/41290#issuecomment-2453330109
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](new_json_reader)fix new_json_reader core [doris]
Posted by "xiaokang (via GitHub)" <gi...@apache.org>.
xiaokang commented on code in PR #41290:
URL: https://github.com/apache/doris/pull/41290#discussion_r1825315273
##########
be/src/vec/exec/format/json/new_json_reader.cpp:
##########
@@ -330,6 +330,10 @@ Status NewJsonReader::get_parsed_schema(std::vector<std::string>* col_names,
objectValue = _json_doc;
}
+ if (!objectValue->IsObject()) {
+ return Status::DataQualityError("JSON data is not an object.");
Review Comment:
What do you mean by `streamload will return false`. I'm concerned that this change might affect other load.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](new_json_reader)fix new_json_reader core [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #41290:
URL: https://github.com/apache/doris/pull/41290#issuecomment-2375595372
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](new_json_reader)fix new_json_reader core [doris]
Posted by "amorynan (via GitHub)" <gi...@apache.org>.
amorynan commented on PR #41290:
URL: https://github.com/apache/doris/pull/41290#issuecomment-2451570747
run cloud_p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](new_json_reader)fix new_json_reader core [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #41290:
URL: https://github.com/apache/doris/pull/41290#issuecomment-2453385892
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](new_json_reader)fix new_json_reader core [doris]
Posted by "amorynan (via GitHub)" <gi...@apache.org>.
amorynan commented on code in PR #41290:
URL: https://github.com/apache/doris/pull/41290#discussion_r1826952018
##########
be/src/vec/exec/format/json/new_json_reader.cpp:
##########
@@ -330,6 +330,10 @@ Status NewJsonReader::get_parsed_schema(std::vector<std::string>* col_names,
objectValue = _json_doc;
}
+ if (!objectValue->IsObject()) {
+ return Status::DataQualityError("JSON data is not an object.");
Review Comment:
this function just for scanner to read json file when in tvf , other load will not call this function
just with VFileScanner::_get_block_wrapped --> NewJsonReader::get_next_block --> NewJsonReader::_read_json_column --> NewJsonReader::_vhandle_simple_json --> NewJsonReader::_set_column_value
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](new_json_reader)fix new_json_reader core [doris]
Posted by "amorynan (via GitHub)" <gi...@apache.org>.
amorynan commented on PR #41290:
URL: https://github.com/apache/doris/pull/41290#issuecomment-2453384554
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](new_json_reader)fix new_json_reader core [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #41290:
URL: https://github.com/apache/doris/pull/41290#issuecomment-2408470203
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](new_json_reader)fix new_json_reader core [doris]
Posted by "amorynan (via GitHub)" <gi...@apache.org>.
amorynan commented on PR #41290:
URL: https://github.com/apache/doris/pull/41290#issuecomment-2376783497
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](new_json_reader)fix new_json_reader core [doris]
Posted by "amorynan (via GitHub)" <gi...@apache.org>.
amorynan commented on PR #41290:
URL: https://github.com/apache/doris/pull/41290#issuecomment-2408468969
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](new_json_reader)fix new_json_reader core [doris]
Posted by "amorynan (via GitHub)" <gi...@apache.org>.
amorynan commented on code in PR #41290:
URL: https://github.com/apache/doris/pull/41290#discussion_r1826953806
##########
be/src/vec/exec/format/json/new_json_reader.cpp:
##########
@@ -330,6 +330,10 @@ Status NewJsonReader::get_parsed_schema(std::vector<std::string>* col_names,
objectValue = _json_doc;
}
+ if (!objectValue->IsObject()) {
+ return Status::DataQualityError("JSON data is not an object.");
Review Comment:
and I think I could print origin type instead of n bytes...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](new_json_reader)fix new_json_reader core [doris]
Posted by "xiaokang (via GitHub)" <gi...@apache.org>.
xiaokang commented on code in PR #41290:
URL: https://github.com/apache/doris/pull/41290#discussion_r1777858592
##########
be/src/vec/exec/format/json/new_json_reader.cpp:
##########
@@ -330,6 +330,10 @@ Status NewJsonReader::get_parsed_schema(std::vector<std::string>* col_names,
objectValue = _json_doc;
}
+ if (!objectValue->IsObject()) {
+ return Status::DataQualityError("JSON data is not an object.");
Review Comment:
add first n bytes of data in error message
##########
be/src/vec/exec/format/json/new_json_reader.cpp:
##########
@@ -330,6 +330,10 @@ Status NewJsonReader::get_parsed_schema(std::vector<std::string>* col_names,
objectValue = _json_doc;
}
+ if (!objectValue->IsObject()) {
+ return Status::DataQualityError("JSON data is not an object.");
Review Comment:
Does this issue only occur in the TVF?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](new_json_reader)fix new_json_reader core [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #41290:
URL: https://github.com/apache/doris/pull/41290#issuecomment-2451482348
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](new_json_reader)fix new_json_reader core [doris]
Posted by "xiaokang (via GitHub)" <gi...@apache.org>.
xiaokang merged PR #41290:
URL: https://github.com/apache/doris/pull/41290
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](new_json_reader)fix new_json_reader core [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #41290:
URL: https://github.com/apache/doris/pull/41290#issuecomment-2453615281
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](new_json_reader)fix new_json_reader core [doris]
Posted by "amorynan (via GitHub)" <gi...@apache.org>.
amorynan commented on PR #41290:
URL: https://github.com/apache/doris/pull/41290#issuecomment-2451477193
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](new_json_reader)fix new_json_reader core [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #41290:
URL: https://github.com/apache/doris/pull/41290#issuecomment-2451509355
TeamCity be ut coverage result:
Function Coverage: 37.82% (9821/25971)
Line Coverage: 29.00% (81633/281540)
Region Coverage: 28.27% (42157/149124)
Branch Coverage: 24.85% (21394/86076)
Coverage Report: http://coverage.selectdb-in.cc/coverage/01a601be04b00903bb6a860536b6364344e89186_01a601be04b00903bb6a860536b6364344e89186/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](new_json_reader)fix new_json_reader core [doris]
Posted by "amorynan (via GitHub)" <gi...@apache.org>.
amorynan commented on code in PR #41290:
URL: https://github.com/apache/doris/pull/41290#discussion_r1799455443
##########
be/src/vec/exec/format/json/new_json_reader.cpp:
##########
@@ -330,6 +330,10 @@ Status NewJsonReader::get_parsed_schema(std::vector<std::string>* col_names,
objectValue = _json_doc;
}
+ if (!objectValue->IsObject()) {
+ return Status::DataQualityError("JSON data is not an object.");
Review Comment:
Yes , only in select tvf . streamload will return false
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](new_json_reader)fix new_json_reader core [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #41290:
URL: https://github.com/apache/doris/pull/41290#issuecomment-2408476695
TeamCity be ut coverage result:
Function Coverage: 37.43% (9679/25862)
Line Coverage: 28.68% (80277/279925)
Region Coverage: 28.11% (41513/147685)
Branch Coverage: 24.70% (21109/85464)
Coverage Report: http://coverage.selectdb-in.cc/coverage/7ab2f15f3bfe2334ee722b056c1950f4bf72456e_7ab2f15f3bfe2334ee722b056c1950f4bf72456e/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](new_json_reader)fix new_json_reader core [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #41290:
URL: https://github.com/apache/doris/pull/41290#issuecomment-2376844960
TeamCity be ut coverage result:
Function Coverage: 37.31% (9635/25823)
Line Coverage: 28.72% (79756/277656)
Region Coverage: 28.14% (41228/146507)
Branch Coverage: 24.76% (20991/84766)
Coverage Report: http://coverage.selectdb-in.cc/coverage/b6fd419f271b2b9a7d38712b124e41d30f20c804_b6fd419f271b2b9a7d38712b124e41d30f20c804/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](new_json_reader)fix new_json_reader core [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #41290:
URL: https://github.com/apache/doris/pull/41290#issuecomment-2375595333
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](new_json_reader)fix new_json_reader core [doris]
Posted by "xiaokang (via GitHub)" <gi...@apache.org>.
xiaokang commented on code in PR #41290:
URL: https://github.com/apache/doris/pull/41290#discussion_r1825315273
##########
be/src/vec/exec/format/json/new_json_reader.cpp:
##########
@@ -330,6 +330,10 @@ Status NewJsonReader::get_parsed_schema(std::vector<std::string>* col_names,
objectValue = _json_doc;
}
+ if (!objectValue->IsObject()) {
+ return Status::DataQualityError("JSON data is not an object.");
Review Comment:
What do you mean by `streamload will return false`? I'm concerned that this change might affect other load.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org