You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "taiyang-li (via GitHub)" <gi...@apache.org> on 2024/11/06 01:25:52 UTC
[I] [CH] failed ut: "read data from orc file format" [incubator-gluten]
taiyang-li opened a new issue, #7823:
URL: https://github.com/apache/incubator-gluten/issues/7823
### Backend
CH (ClickHouse)
### Bug description
``` sql
09:04:33.819 ERROR org.apache.spark.task.TaskResources: Task 1 failed by error:
org.apache.gluten.exception.GlutenException: org.apache.gluten.exception.GlutenException: Input value -1094795586 of a column "date_field" exceeds the range of type Date32: (in file/uri /data1/liyang/cppproject/gluten/backends-clickhouse/target/scala-2.12/test-classes/orc-data/all_data_types_with_non_primitive_type.snappy.orc): While executing SubstraitFileSource
0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000030bfadb4
1. ./build_gcc/./src/Common/Exception.cpp:109: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001a9d6c09
2. ./src/Common/Exception.h:110: DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000009e064c5
3. ./src/Common/Exception.h:128: DB::Exception::Exception<int&, String const&>(int, FormatStringHelperImpl<std::type_identity<int&>::type, std::type_identity<String const&>::type>, int&, String const&) @ 0x0000000029b9b3fa
4. ./build_gcc/./src/Processors/Formats/Impl/NativeORCBlockInputFormat.cpp:1554: DB::ORCColumnToCHColumn::readColumnFromORCColumn(orc::ColumnVectorBatch const*, orc::Type const*, String const&, bool, std::shared_ptr<DB::IDataType const>) const @ 0x0000000029b84eb7
5. ./build_gcc/./src/Processors/Formats/Impl/NativeORCBlockInputFormat.cpp:1601: DB::ORCColumnToCHColumn::readColumnFromORCColumn(orc::ColumnVectorBatch const*, orc::Type const*, String const&, bool, std::shared_ptr<DB::IDataType const>) const @ 0x0000000029b793aa
6. ./build_gcc/./src/Processors/Formats/Impl/NativeORCBlockInputFormat.cpp:1892: DB::ORCColumnToCHColumn::orcColumnsToCHChunk(DB::Chunk&, std::unordered_map<String, std::pair<orc::ColumnVectorBatch const*, orc::Type const*>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, std::pair<orc::ColumnVectorBatch const*, orc::Type const*>>>>&, unsigned long, DB::BlockMissingValues*) @ 0x0000000029b75a04
7. ./build_gcc/./src/Processors/Formats/Impl/NativeORCBlockInputFormat.cpp:1136: DB::ORCColumnToCHColumn::orcTableToCHChunk(DB::Chunk&, orc::Type const*, orc::ColumnVectorBatch const*, unsigned long, DB::BlockMissingValues*) @ 0x0000000029b7134a
8. ./build_gcc/./src/Processors/Formats/Impl/NativeORCBlockInputFormat.cpp:1034: DB::NativeORCBlockInputFormat::read() @ 0x0000000029b70986
9. ./build_gcc/./src/Processors/Formats/IInputFormat.cpp:19: DB::IInputFormat::generate() @ 0x0000000029a96486
10. ./build_gcc/./utils/extern-local-engine/Storages/SubstraitSource/SubstraitFileSource.cpp:375: local_engine::NormalFileReader::pull(DB::Chunk&) @ 0x000000001b9d23d7
11. ./build_gcc/./utils/extern-local-engine/Storages/SubstraitSource/SubstraitFileSource.cpp:113: local_engine::SubstraitFileSource::generate() @ 0x000000001b9cc72d
12. ./build_gcc/./src/Processors/ISource.cpp:139: DB::ISource::tryGenerate() @ 0x0000000029a3fa38
13. ./build_gcc/./src/Processors/ISource.cpp:108: DB::ISource::work() @ 0x0000000029a3eec1
14. ./build_gcc/./src/Processors/Executors/ExecutionThreadContext.cpp:47: DB::ExecutionThreadContext::executeTask() @ 0x0000000029a7c98e
15. ./build_gcc/./src/Processors/Executors/PipelineExecutor.cpp:289: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000029a601b1
16. ./build_gcc/./src/Processors/Executors/PipelineExecutor.cpp:163: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x0000000029a5f012
17. ./build_gcc/./src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x0000000029a8c116
18. ./build_gcc/./src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x0000000029a8c676
19. ./build_gcc/./utils/extern-local-engine/Parser/LocalExecutor.cpp:68: local_engine::LocalExecutor::hasNext() @ 0x000000001b2b4233
20. ./build_gcc/./utils/extern-local-engine/local_engine_jni.cpp:308: Java_org_apache_gluten_vectorized_BatchIterator_nativeHasNext @ 0x0000000009dd7493
```
### Spark version
None
### Spark configurations
_No response_
### System information
_No response_
### Relevant logs
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] [CH] failed ut: "read data from orc file format" and "test table bucketed by all typed columns" [incubator-gluten]
Posted by "zzcclp (via GitHub)" <gi...@apache.org>.
zzcclp closed issue #7823: [CH] failed ut: "read data from orc file format" and "test table bucketed by all typed columns"
URL: https://github.com/apache/incubator-gluten/issues/7823
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] [CH] failed ut: "read data from orc file format" [incubator-gluten]
Posted by "taiyang-li (via GitHub)" <gi...@apache.org>.
taiyang-li commented on issue #7823:
URL: https://github.com/apache/incubator-gluten/issues/7823#issuecomment-2458589536
Currently ignored by https://github.com/apache/incubator-gluten/pull/7821
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] [CH] failed ut: "read data from orc file format" [incubator-gluten]
Posted by "taiyang-li (via GitHub)" <gi...@apache.org>.
taiyang-li commented on issue #7823:
URL: https://github.com/apache/incubator-gluten/issues/7823#issuecomment-2458536734
Reason: After https://github.com/ClickHouse/ClickHouse/pull/69473, `DataBuffer` constructor will not set each element to zero, which means that after `ColumnReader::next`, only `batch.data[i]` in those rows with `batch.notNulls[i] != 0` will be initialized. For rows with `batch.notNulls[i] == 0`, `batch.data[i]` is not initialized.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] [CH] failed ut: "read data from orc file format" [incubator-gluten]
Posted by "taiyang-li (via GitHub)" <gi...@apache.org>.
taiyang-li commented on issue #7823:
URL: https://github.com/apache/incubator-gluten/issues/7823#issuecomment-2458573126
``` c++
template <class T>
DataBuffer<T>::DataBuffer(MemoryPool& pool, uint64_t newSize)
- : memoryPool(pool), buf(nullptr), currentSize(0), currentCapacity(0) {
- resize(newSize);
+ : memoryPool_(pool), buf_(nullptr), currentSize_(0), currentCapacity_(0) {
+ reserve(newSize);
+ currentSize_ = newSize;
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org