You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2023/01/16 10:28:16 UTC
[GitHub] [doris] BePPPower opened a new pull request, #15980: [Enhencement](Push Handle) use VParquetScanner in PushHandle
BePPPower opened a new pull request, #15980:
URL: https://github.com/apache/doris/pull/15980
# Proposed changes
Issue Number: close #xxx
## Problem summary
Describe your changes.
## Checklist(Required)
1. Does it affect the original behavior:
- [ ] Yes
- [ ] No
- [ ] I don't know
2. Has unit tests been added:
- [ ] Yes
- [ ] No
- [ ] No Need
3. Has document been added or modified:
- [ ] Yes
- [ ] No
- [ ] No Need
4. Does it need to update dependencies:
- [ ] Yes
- [ ] No
5. Are there any changes that cannot be rolled back:
- [ ] Yes (If Yes, please explain WHY)
- [ ] No
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #15980: [Enhencement](Push Handle) use VParquetScanner in PushHandle
Posted by GitBox <gi...@apache.org>.
hello-stephen commented on PR #15980:
URL: https://github.com/apache/doris/pull/15980#issuecomment-1384855472
TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 36.08 seconds
load time: 500 seconds
storage size: 17121017887 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230117053322_clickbench_pr_81805.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #15980: [Enhencement](Push Handle) use VParquetScanner in PushHandle
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #15980:
URL: https://github.com/apache/doris/pull/15980#issuecomment-1384772627
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] BePPPower commented on a diff in pull request #15980: [Enhencement](Push Handle) use VParquetScanner in PushHandle
Posted by GitBox <gi...@apache.org>.
BePPPower commented on code in PR #15980:
URL: https://github.com/apache/doris/pull/15980#discussion_r1071693102
##########
be/src/olap/push_handler.cpp:
##########
@@ -842,9 +833,9 @@ Status PushBrokerReader::init(const Schema* schema, const TBrokerScanRange& t_sc
BaseScanner* scanner = nullptr;
switch (t_scan_range.ranges[0].format_type) {
case TFileFormatType::FORMAT_PARQUET:
- scanner = new ParquetScanner(_runtime_state.get(), _runtime_profile, t_scan_range.params,
Review Comment:
ok
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on a diff in pull request #15980: [Enhencement](Push Handle) use VParquetScanner in PushHandle
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on code in PR #15980:
URL: https://github.com/apache/doris/pull/15980#discussion_r1071077977
##########
be/src/olap/push_handler.h:
##########
@@ -178,31 +178,27 @@ class LzoBinaryReader : public IBinaryReader {
class PushBrokerReader {
public:
- PushBrokerReader() : _ready(false), _eof(false), _fill_tuple(false) {}
+ PushBrokerReader() : _ready(false), _eof(false) {}
~PushBrokerReader() = default;
Status init(const Schema* schema, const TBrokerScanRange& t_scan_range,
const TDescriptorTable& t_desc_tbl);
- Status next(ContiguousRow* row);
+ Status next(vectorized::Block* block);
void print_profile();
Status close() {
_ready = false;
return Status::OK();
}
bool eof() const { return _eof; }
- bool is_fill_tuple() const { return _fill_tuple; }
MemPool* mem_pool() { return _mem_pool.get(); }
private:
Status fill_field_row(RowCursorCell* dst, const char* src, bool src_null, MemPool* mem_pool,
FieldType type);
bool _ready;
bool _eof;
- bool _fill_tuple;
TupleDescriptor* _tuple_desc;
Review Comment:
warning: private field '_tuple_desc' is not used [clang-diagnostic-unused-private-field]
```cpp
TupleDescriptor* _tuple_desc;
^
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Gabriel39 commented on a diff in pull request #15980: [Enhencement](Push Handle) use VParquetScanner in PushHandle
Posted by GitBox <gi...@apache.org>.
Gabriel39 commented on code in PR #15980:
URL: https://github.com/apache/doris/pull/15980#discussion_r1071084674
##########
be/src/olap/push_handler.cpp:
##########
@@ -842,9 +833,9 @@ Status PushBrokerReader::init(const Schema* schema, const TBrokerScanRange& t_sc
BaseScanner* scanner = nullptr;
switch (t_scan_range.ranges[0].format_type) {
case TFileFormatType::FORMAT_PARQUET:
- scanner = new ParquetScanner(_runtime_state.get(), _runtime_profile, t_scan_range.params,
Review Comment:
You can remove ParquetScanner from Doris after this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #15980: [Enhencement](Push Handle) use VParquetScanner in PushHandle
Posted by GitBox <gi...@apache.org>.
yiguolei merged PR #15980:
URL: https://github.com/apache/doris/pull/15980
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org