You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2022/05/05 07:54:09 UTC

[GitHub] [incubator-doris] Gabriel39 opened a new issue, #9374: [Bug] `replace_if_not_null` not found in vectorized compaction

Gabriel39 opened a new issue, #9374:
URL: https://github.com/apache/incubator-doris/issues/9374

   ### Search before asking
   
   - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues.
   
   
   ### Version
   
   master branch
   
   ### What's Wrong?
   
   Some thing wrong for `REPLACE NOT NULL` aggregation type column in vectorized compaction now.
   
   `
    1# 0x00007FE7DDEC6400 in /lib64/libc.so.6
    2# doris::vectorized::BlockReader::_init_agg_state(doris::TabletReader::ReaderParams const&) at /root/incubator-doris/be/src/vec/olap/block_reader.cpp:107
    3# doris::vectorized::BlockReader::init(doris::TabletReader::ReaderParams const&) at /root/incubator-doris/be/src/vec/olap/block_reader.cpp:162
    4# doris::Merger::vmerge_rowsets(std::shared_ptr<doris::Tablet>, doris::ReaderType, std::vector<std::shared_ptr<doris::RowsetReader>, std::allocator<std::shared_ptr<doris::RowsetReader> > > const&, doris::RowsetWriter*, doris::Merger::Statistics*) at /root/incubator-doris/be/src/olap/merger.cpp:106
    5# doris::Compaction::do_compaction_impl(long) at /root/incubator-doris/be/src/olap/compaction.cpp:95
    6# doris::Compaction::do_compaction(long) at /root/incubator-doris/be/src/olap/compaction.cpp:57
    7# doris::CumulativeCompaction::execute_compact_impl() at /root/incubator-doris/be/src/olap/cumulative_compaction.cpp:75
    8# doris::Compaction::execute_compact() at /root/incubator-doris/be/src/olap/compaction.cpp:46
    9# doris::Tablet::execute_compaction(doris::CompactionType) at /root/incubator-doris/be/src/olap/tablet.cpp:1407
   10# std::_Function_handler<void (), doris::StorageEngine::_submit_compaction_task(std::shared_ptr<doris::Tablet>, doris::CompactionType)::{lambda()#1}>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/include/c++/11/bits/std_function.h:291
   11# doris::ThreadPool::dispatch_thread() at /root/incubator-doris/be/src/util/threadpool.cpp:548
   12# doris::Thread::supervise_thread(void*) at /root/incubator-doris/be/src/util/thread.cpp:409
   13# start_thread in /lib64/libpthread.so.0
   14# clone in /lib64/libc.so.6
   
   `
   
   ### What You Expected?
   
   fix it
   
   ### How to Reproduce?
   
   _No response_
   
   ### Anything Else?
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] xinyiZzz commented on issue #9374: [Bug] `replace_if_not_null` not found in vectorized compaction

Posted by GitBox <gi...@apache.org>.
xinyiZzz commented on issue #9374:
URL: https://github.com/apache/incubator-doris/issues/9374#issuecomment-1118270960

   I also had this problem the other day, but it disappeared after rebase master...


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] yiguolei closed issue #9374: [Bug] `replace_if_not_null` not found in vectorized compaction

Posted by GitBox <gi...@apache.org>.
yiguolei closed issue #9374: [Bug] `replace_if_not_null` not found in vectorized compaction
URL: https://github.com/apache/incubator-doris/issues/9374


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] xinyiZzz commented on issue #9374: [Bug] `replace_if_not_null` not found in vectorized compaction

Posted by GitBox <gi...@apache.org>.
xinyiZzz commented on issue #9374:
URL: https://github.com/apache/incubator-doris/issues/9374#issuecomment-1118299344

   The reason why the bug disappeared after I rebase master was because config::enable_vectorized_compaction was added, and vmerge_rowsets was not used...
   
   > a nullable column with `REPLACE NOT NULL` agg type
   My reproduce case is similar.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] Toms1999 commented on issue #9374: [Bug] `replace_if_not_null` not found in vectorized compaction

Posted by GitBox <gi...@apache.org>.
Toms1999 commented on issue #9374:
URL: https://github.com/apache/incubator-doris/issues/9374#issuecomment-1118282646

   OK
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] Gabriel39 commented on issue #9374: [Bug] `replace_if_not_null` not found in vectorized compaction

Posted by GitBox <gi...@apache.org>.
Gabriel39 commented on issue #9374:
URL: https://github.com/apache/incubator-doris/issues/9374#issuecomment-1118281604

   > I also had this problem the other day, but it disappeared after rebase master...
   
   @xinyiZzz 
   I reproduce this bug using up-to-date master branch. In fact, if you have a nullable column with `REPLACE NOT NULL` agg type and turn `enable_vectorized_compaction` on and then you can also reproduce it again.
   
   In my case, I have a column ``last_update_date` DATETIME REPLACE_IF_NOT_NULL DEFAULT "1970-01-01 00:00:00"`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] BiteTheDDDDt commented on issue #9374: [Bug] `replace_if_not_null` not found in vectorized compaction

Posted by GitBox <gi...@apache.org>.
BiteTheDDDDt commented on issue #9374:
URL: https://github.com/apache/incubator-doris/issues/9374#issuecomment-1118284554

   Do you have a sql that can reproduce this error?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [incubator-doris] Gabriel39 commented on issue #9374: [Bug] `replace_if_not_null` not found in vectorized compaction

Posted by GitBox <gi...@apache.org>.
Gabriel39 commented on issue #9374:
URL: https://github.com/apache/incubator-doris/issues/9374#issuecomment-1118291912

   > Do you have a sql that can reproduce this error?
   @BiteTheDDDDt 
   I create table using statement:
   `
   CREATE TABLE IF NOT EXISTS compaction_tbl
   (
       `user_id` LARGEINT NOT NULL COMMENT "用户id",
       `date` DATE NOT NULL COMMENT "数据灌入日期时间",
       `city` VARCHAR(20) COMMENT "用户所在城市",
       `age` SMALLINT COMMENT "用户年龄",
       `sex` TINYINT COMMENT "用户性别",
       `last_visit_date` DATETIME REPLACE DEFAULT "1970-01-01 00:00:00" COMMENT "用户最后一次访问时间",
       `last_update_date` DATETIME REPLACE_IF_NOT_NULL DEFAULT "1970-01-01 00:00:00" COMMENT "用户最后一次更新时间",
       `last_visit_date_not_null` DATETIME REPLACE NOT NULL DEFAULT "1970-01-01 00:00:00" COMMENT "用户最后一次访问时间",
       `cost` BIGINT SUM DEFAULT "0" COMMENT "用户总消费",
       `max_dwell_time` INT MAX DEFAULT "0" COMMENT "用户最大停留时间",
       `min_dwell_time` INT MIN DEFAULT "99999" COMMENT "用户最小停留时间",
       `hll_col` HLL HLL_UNION NOT NULL COMMENT "HLL列",
       `bitmap_col` Bitmap BITMAP_UNION NOT NULL COMMENT "bitmap列"
   )
   AGGREGATE KEY(`user_id`, `date`, `city`, `age`, `sex`)
   DISTRIBUTED BY HASH(user_id) PROPERTIES("replication_num" = "1");
   
   `
   
   Insert some test data and after a while, background compaction task will core.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org