You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "DarvenDuan (via GitHub)" <gi...@apache.org> on 2023/11/30 10:18:11 UTC
[PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
DarvenDuan opened a new pull request, #27828:
URL: https://github.com/apache/doris/pull/27828
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
add a field `scanTables` in edit log to record table name and index name for a query
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1835842492
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1849383698
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1835291500
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1833705255
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 43.33 seconds
stream load tsv: 561 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.8 seconds inserted 10000000 Rows, about 347K ops/s
storage size: 17163949257 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1846871968
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1849394986
run arm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1837162682
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1837321347
run pipelinex
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1837178437
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 43.77 seconds
stream load tsv: 580 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 29.1 seconds inserted 10000000 Rows, about 343K ops/s
storage size: 17164006140 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1837445516
run arm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on code in PR #27828:
URL: https://github.com/apache/doris/pull/27828#discussion_r1410500057
##########
fe/fe-core/src/main/java/org/apache/doris/plugin/AuditEvent.java:
##########
@@ -60,6 +60,9 @@ public enum EventType {
public String catalog = "";
@AuditField(value = "Db")
public String db = "";
+ // scan table is formatted as {db}.{table} and {db}.{table}.{index} if table is OlapTable
Review Comment:
Thanks for your reply, I will implement this latter
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1851361957
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1835577363
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1837324038
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1837321289
run arm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1837137858
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1847045452
run compile
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1849459830
run compile
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1846684829
run compile
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1837164017
> If a user is using doris 2.0 and using audit log plugin to load the log into doris table. If he upgrade doris 2.0 to doris 2.1 and there is a new column tables, what should he do??? I think there is a upgrade process and maybe need a doc.
I modified `audit-plugin.md` and added some notes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1837377006
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 43.7 seconds
stream load tsv: 573 seconds loaded 74807831229 Bytes, about 124 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 27.8 seconds inserted 10000000 Rows, about 359K ops/s
storage size: 17167410254 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #27828: [Enhancement](auditLog) support recording table name and index name in audit log
URL: https://github.com/apache/doris/pull/27828
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-2156915013
We're closing this PR because it hasn't been updated in a while.
This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and feel free a maintainer to remove the Stale tag!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1849395188
run pipelinex p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1849483491
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1851456523
run beut
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1833496803
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1837321590
run pipelinex_p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1837163339
> You should implement audit logger plugin in same PR, we will release 2.1 now, if this PR is merged and you not implement another PR, then 2.1 could not be released.
I had add a new field in audit log plugin.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1837320521
run pipline
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1844650122
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1846939908
run feut
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1844243418
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1844988725
run compile
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1851223082
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1833889939
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit 0ddf2d16526fc5a9073f438ba88b1276ad246672, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4892 4667 4646 4646
q2 347 176 161 161
q3 1520 1315 1258 1258
q4 1141 927 955 927
q5 3231 3252 3214 3214
q6 252 131 132 131
q7 983 513 557 513
q8 2292 2253 2240 2240
q9 6948 6954 6916 6916
q10 3312 3383 3385 3383
q11 334 212 210 210
q12 351 220 211 211
q13 4644 3887 3887 3887
q14 252 221 218 218
q15 586 538 527 527
q16 441 400 388 388
q17 1050 736 632 632
q18 7998 7913 7338 7338
q19 1597 1541 1603 1541
q20 1235 309 291 291
q21 3432 2929 2957 2929
q22 361 301 312 301
Total cold run time: 47199 ms
Total hot run time: 41862 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4615 4581 4585 4581
q2 319 212 229 212
q3 3744 3754 3746 3746
q4 2527 2519 2510 2510
q5 6180 6180 6190 6180
q6 248 123 127 123
q7 2583 1961 1994 1961
q8 3746 3755 3685 3685
q9 9464 9407 9451 9407
q10 4093 4141 4159 4141
q11 634 515 514 514
q12 808 643 648 643
q13 4416 3629 3639 3629
q14 283 240 250 240
q15 585 531 522 522
q16 517 486 485 485
q17 2114 2097 2084 2084
q18 9628 8992 9239 8992
q19 1827 1767 1788 1767
q20 2322 1982 1975 1975
q21 7403 7160 6822 6822
q22 677 543 557 543
Total cold run time: 68733 ms
Total hot run time: 64762 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #27828:
URL: https://github.com/apache/doris/pull/27828#discussion_r1410486509
##########
fe/fe-core/src/main/java/org/apache/doris/plugin/AuditEvent.java:
##########
@@ -60,6 +60,9 @@ public enum EventType {
public String catalog = "";
@AuditField(value = "Db")
public String db = "";
+ // scan table is formatted as {db}.{table} and {db}.{table}.{index} if table is OlapTable
Review Comment:
maybe you also need modify auditlog plugin， because it will load the log into a doris table. I think you add a new field to auditlog
and the table schema need change.
##########
fe/fe-core/src/main/java/org/apache/doris/plugin/AuditEvent.java:
##########
@@ -60,6 +60,9 @@ public enum EventType {
public String catalog = "";
@AuditField(value = "Db")
public String db = "";
+ // scan table is formatted as {db}.{table} and {db}.{table}.{index} if table is OlapTable
Review Comment:
maybe you also need modify auditlog plugin， because it will load the log into a doris table. I think you add a new field to auditlog
and the table schema need change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1837027883
You should implement audit logger plugin in same PR, we will release 2.1 now, if this PR is merged and you not implement another PR, then 2.1 could not be released.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1844795792
run pipelinex_p0 10
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1844952316
run pipelinex_p0 10
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1846554651
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit 631a7e0b813c8185af3d4735e6765eca07cb6255, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4707 4498 4442 4442
q2 372 141 158 141
q3 1455 1272 1205 1205
q4 1109 913 917 913
q5 3117 3141 3174 3141
q6 245 130 130 130
q7 1003 482 477 477
q8 2221 2231 2221 2221
q9 6660 6663 6662 6662
q10 3197 3257 3283 3257
q11 322 209 203 203
q12 344 209 204 204
q13 4615 3808 3774 3774
q14 237 214 223 214
q15 566 525 523 523
q16 439 383 381 381
q17 1005 619 560 560
q18 7520 7054 7278 7054
q19 1544 1335 1431 1335
q20 533 278 289 278
q21 3067 2608 2630 2608
q22 353 287 287 287
Total cold run time: 44631 ms
Total hot run time: 40010 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4392 4395 4387 4387
q2 269 163 175 163
q3 3532 3525 3517 3517
q4 2385 2370 2367 2367
q5 5736 5725 5725 5725
q6 242 122 122 122
q7 2365 1893 1897 1893
q8 3525 3519 3527 3519
q9 9044 9034 8959 8959
q10 3901 3996 4000 3996
q11 495 396 372 372
q12 779 589 608 589
q13 4314 3590 3562 3562
q14 289 255 257 255
q15 578 519 523 519
q16 493 463 472 463
q17 1892 1853 1846 1846
q18 8722 8194 8274 8194
q19 1709 1741 1722 1722
q20 2268 1964 1917 1917
q21 6532 6208 6194 6194
q22 501 433 412 412
Total cold run time: 63963 ms
Total hot run time: 60693 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1846564200
run p0 10
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1844078070
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1849231029
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1849234442
run arm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1849661778
run p0 10
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1849925911
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1837320176
run arm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1851446728
run feut
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1844793546
run pipelinex
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1846498735
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1846750519
run p0 10
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1846562705
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.8 seconds
stream load tsv: 582 seconds loaded 74807831229 Bytes, about 122 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 29.1 seconds inserted 10000000 Rows, about 343K ops/s
storage size: 17213408484 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](auditLog) support recording table name and index name in audit log [doris]
Posted by "DarvenDuan (via GitHub)" <gi...@apache.org>.
DarvenDuan commented on PR #27828:
URL: https://github.com/apache/doris/pull/27828#issuecomment-1846565202
run pipelinex_p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org