[jira] [Commented] (DRILL-7030) Make format plugins fully plugable

["Paul Rogers (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DRILL-7030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16762010#comment-16762010 ] 

Paul Rogers commented on DRILL-7030:
------------------------------------

Please also ensure that the unit testing tools available in exec are also available in contrib. In particular, a user should be able to write a unit test, run it from their IDE, and step through their plugin code. If change is needed, the needed build should be within the IDE, no need to invoke maven.

That is, contrib should enable the rapid edit-compile-debug cycle that we've created in exec.

> Make format plugins fully plugable
> ----------------------------------
>
>                 Key: DRILL-7030
>                 URL: https://issues.apache.org/jira/browse/DRILL-7030
>             Project: Apache Drill
>          Issue Type: Improvement
>    Affects Versions: 1.15.0
>            Reporter: Arina Ielchiieva
>            Priority: Major
>
> Discussion on the mailing list - [https://lists.apache.org/thread.html/0c7de9c23ee9a8e18f8548ae0a323284cf1311b9570bd77ba544f63d@%3Cdev.drill.apache.org%3E]
> {noformat}
> Before we were adding new formats / plugins into the exec module. Eventually we came up to the point that exec package size is growing and adding plugin and format contributions is better to separate out in the different module.
> Now we have contrib module where we add such contributions. Plugins are pluggable, there are added automatically by means of having drill-module.conf file which points to the scanning packages.
> Format plugins are using the same approach, the only problem is that they are not added into bootstrap-storage-plugins.json. So when adding new format plugin, in order for it to automatically appear in Drill Web UI, developer has to update bootstrap file which is in the exec module.
> My suggestion we implement some functionality that would merge format config with the bootstrap one. For example, each plugin would have to have bootstrap-format.json file with the information to which plugin format should be added (structure the same as in bootstrap-storage-plugins.json):
> Example:
> {
>   "storage":{
>     dfs: {
>       formats: {
>         "psv" : {
>           type: "msgpack",
>           extensions: [ "mp" ]
>         }
>       }
>     }
>   }
> }
> Then during Drill start up such bootstrap-format.json files will be merged with bootstrap-storage-plugins.json.{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)