You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2020/03/12 11:20:59 UTC
[GitHub] [incubator-doris] Seaven edited a comment on issue #2463: Add fe
plugin framework
Seaven edited a comment on issue #2463: Add fe plugin framework
URL: https://github.com/apache/incubator-doris/pull/2463#issuecomment-567003853
for #2344
* Add install/unintall Plugin statement
* Add Show plugin statement
* Support install plugin through two ways:
* Built-in Plugin: use PluginMgr's register method.
* Dynamic Plugin: install by SQL statement, and the process:
1. check Plugin has already install?
2. download Plugin file from remote source or copy from local source
3. extract Plugin's .zip
4. read Plugin's plugin.properties, and check Plugin's Value
5. dynamic load .jar and init Plugin's main Class
6. invoke Plugin's init method
7. register Plugin into PluginMgr.
8. update meta
* Support FE Plugin dynamic uninstall process
1. check Plugin has install?
2. invoke Plugin's close method
3. delete Plugin from PluginMgr
4. update meta
* Add audit plugin interface
* Add plugin enable flags in Config
* Add plugin install path in Config, default plugin will install in ${DORIS_FE_PATH}/plugins
* Add audit plugin demo
The audit demo plugin will print query in logs, the impelement in test/plugin/AuditPluginDemo, and the plugin .zip in test/resources/auditplugindemo.zip
The usage:
```
// install plugin and show plugins;
mysql>
mysql> install plugin from "/home/users/hekai/auditplugindemo.zip";
Query OK, 0 rows affected (0.05 sec)
mysql>
mysql> show plugins;
+-------------------+-------+---------------+---------+-------------+------------------------+--------+---------------------------------------+
| Name | Type | Description | Version | JavaVersion | ClassName | SoName | Sources |
+-------------------+-------+---------------+---------+-------------+------------------------+--------+---------------------------------------+
| audit_plugin_demo | AUDIT | just for test | 0.11.0 | 1.8.31 | plugin.AuditPluginDemo | NULL | /home/users/hekai/auditplugindemo.zip |
+-------------------+-------+---------------+---------+-------------+------------------------+--------+---------------------------------------+
1 row in set (0.00 sec)
mysql> use test;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql>
mysql>
mysql>
mysql> show tables;
+----------------+
| Tables_in_test |
+----------------+
| baseall |
| bigtable |
+----------------+
2 rows in set (0.00 sec)
mysql> select * from baseall;
Empty set (0.01 sec)
mysql>
mysql>
mysql> show plugins;
+-------------------+-------+---------------+---------+-------------+------------------------+--------+---------------------------------------+
| Name | Type | Description | Version | JavaVersion | ClassName | SoName | Sources |
+-------------------+-------+---------------+---------+-------------+------------------------+--------+---------------------------------------+
| audit_plugin_demo | AUDIT | just for test | 0.11.0 | 1.8.31 | plugin.AuditPluginDemo | NULL | /home/users/hekai/auditplugindemo.zip |
+-------------------+-------+---------------+---------+-------------+------------------------+--------+---------------------------------------+
1 row in set (0.00 sec)
mysql>
mysql>
mysql>
mysql>
mysql> uninstall plugin audit_plugin_demo; Query OK, 0 rows affected (0.04 sec)
mysql> show plugins;
Empty set (0.00 sec)
mysql>
mysql>
```
then we can find in logs:
```
2020-03-12 17:30:54,940 INFO 175 [StmtExecutor.analyze():349] begin to analyze stmt: 14, forwarded stmt id: 0
2020-03-12 17:30:54,950 INFO 175 [PluginZip.downloadRemoteZip():104] download plugin zip from: http://bjhw-sys-rpm405d4b151.bjhw.baidu.com:8000/hktest/auditplugindemo.zip
2020-03-12 17:30:54,984 INFO 175 [AuditPluginDemo.init():38] this is audit plugin demo init
2020-03-12 17:30:55,041 INFO 175 [Catalog.installPlugin():6566] install plugin = audit_plugin_demo
2020-03-12 17:30:55,041 INFO 175 [AuditPluginDemo.exec():54] audit demo plugin log: event=2 masks=2 user=root, ip=10.146.209.30, query=install plugin from "http://bjhw-sys-rpm405d4b151.bjhw.baidu.com:8000/hktest/auditplugindemo.zip"
2020-03-12 17:31:02,756 INFO 83 [BDBJEJournal.getFinalizedJournalId():412] database names: 1
2020-03-12 17:31:02,817 INFO 175 [AuditPluginDemo.exec():54] audit demo plugin log: event=2 masks=1 user=root, ip=10.146.209.30, query=show plugins
2020-03-12 17:31:02,817 INFO 175 [StmtExecutor.analyze():349] begin to analyze stmt: 15, forwarded stmt id: 0
2020-03-12 17:31:02,836 INFO 175 [AuditPluginDemo.exec():54] audit demo plugin log: event=2 masks=2 user=root, ip=10.146.209.30
2020-03-12 18:59:11,735 INFO 98 [ReportHandler.taskReport():352] finished to handle task report from backend 10005, diff task num: 0. cost: 0 ms
2020-03-12 18:59:11,915 INFO 196 [AuditPluginDemo.exec():54] audit demo plugin log: event=2 masks=1 user=root, ip=10.146.209.30, query=select * from baseall
2020-03-12 18:59:11,915 INFO 196 [StmtExecutor.analyze():349] begin to analyze stmt: 43, forwarded stmt id: 0
2020-03-12 18:59:11,920 INFO 196 [QeProcessorImpl.registerQuery():68] register query id = 36e09bda9f054753-bec8d2157a9dfdda, job: -1
2020-03-12 18:59:11,928 INFO 196 [QeProcessorImpl.unregisterQuery():78] deregister query id 36e09bda9f054753-bec8d2157a9dfdda
2020-03-12 18:59:11,928 INFO 196 [AuditPluginDemo.exec():54] audit demo plugin log: event=2 masks=2 user=root, ip=10.146.209.30, query=select * from baseall
2020-03-12 18:59:12,605 INFO 38 [LoadBalancer.selectAlternativeTabletsForCluster():121] get number of low load paths: 1, with medium: HDD
```
And To do:
* for BE Plugin, need check valid through BE node. we need add rpc interface
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org