You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by Charuta Rajopadhye <ch...@gmail.com> on 2017/08/29 10:26:23 UTC
Custom extension storage plugin development
Hi Team,
I need to develop an extension storage plugin for my data source.
I have been through this
<https://stackoverflow.com/questions/37557388/how-to-write-custom-storage-plugin-for-apache-drill/37646421#37646421>
and Apache Drill's documentation <https://drill.apache.org/docs/> but did
not find helpful references for my cause.
Thanks to the mailing list, i got hold of a few wiki pages:
https://github.com/paul-rogers/drill/wiki/Storage-Plugin-Configuration
*https://github.com/paul-rogers/drill/wiki/Storage-Plugin-Model
<https://github.com/paul-rogers/drill/wiki/Storage-Plugin-Model>*
and other related pages on the same link.
I have downloaded the drill source code, cross referenced the storage
plugin implementations in there and the information on the wiki pages,
learnt about the interfaces to implement, classes to use, configs to set up
etc.
This has made my understanding a smidgen better, but i am still unable to
get a clear picture of the workflow.
Can someone please guide me in this regard or provide a few more pointers?
Re: Custom extension storage plugin development
Posted by Charuta Rajopadhye <ch...@gmail.com>.
Thank you very much Muhammad and Paul, I'll go through the pcap plugin and
read the docs again.
Regards,
Charuta
On Tue, Aug 29, 2017 at 11:35 PM, Paul Rogers <pr...@mapr.com> wrote:
> Also look at the recently-added pcap format plugin. Seems the developers
> there found a minimum-effort path to doing all the necessary steps. The
> steps are more complex than we’d like, but pcap is as a good example as any
> for how to work your way through those steps.
>
> - Paul
>
> > On Aug 29, 2017, at 4:23 AM, Muhammad Gelbana <m....@gmail.com>
> wrote:
> >
> > I assume this page
> > <https://github.com/paul-rogers/drill/wiki/Storage-
> Plugin-Runtime-Interface>
> > can give you the overview you are looking for ? May be you can start with
> > asking specific questions.
> >
> > Thanks,
> > Gelbana
> >
> > On Tue, Aug 29, 2017 at 12:26 PM, Charuta Rajopadhye <
> charutar90@gmail.com>
> > wrote:
> >
> >> Hi Team,
> >>
> >> I need to develop an extension storage plugin for my data source.
> >> I have been through this
> >> <https://stackoverflow.com/questions/37557388/how-to-
> >> write-custom-storage-plugin-for-apache-drill/37646421#37646421>
> >> and Apache Drill's documentation <https://drill.apache.org/docs/> but
> did
> >> not find helpful references for my cause.
> >> Thanks to the mailing list, i got hold of a few wiki pages:
> >> https://github.com/paul-rogers/drill/wiki/Storage-Plugin-Configuration
> >> *https://github.com/paul-rogers/drill/wiki/Storage-Plugin-Model
> >> <https://github.com/paul-rogers/drill/wiki/Storage-Plugin-Model>*
> >> and other related pages on the same link.
> >> I have downloaded the drill source code, cross referenced the storage
> >> plugin implementations in there and the information on the wiki pages,
> >> learnt about the interfaces to implement, classes to use, configs to
> set up
> >> etc.
> >> This has made my understanding a smidgen better, but i am still unable
> to
> >> get a clear picture of the workflow.
> >> Can someone please guide me in this regard or provide a few more
> pointers?
> >>
>
>
Re: Custom extension storage plugin development
Posted by Paul Rogers <pr...@mapr.com>.
Also look at the recently-added pcap format plugin. Seems the developers there found a minimum-effort path to doing all the necessary steps. The steps are more complex than we’d like, but pcap is as a good example as any for how to work your way through those steps.
- Paul
> On Aug 29, 2017, at 4:23 AM, Muhammad Gelbana <m....@gmail.com> wrote:
>
> I assume this page
> <https://github.com/paul-rogers/drill/wiki/Storage-Plugin-Runtime-Interface>
> can give you the overview you are looking for ? May be you can start with
> asking specific questions.
>
> Thanks,
> Gelbana
>
> On Tue, Aug 29, 2017 at 12:26 PM, Charuta Rajopadhye <ch...@gmail.com>
> wrote:
>
>> Hi Team,
>>
>> I need to develop an extension storage plugin for my data source.
>> I have been through this
>> <https://stackoverflow.com/questions/37557388/how-to-
>> write-custom-storage-plugin-for-apache-drill/37646421#37646421>
>> and Apache Drill's documentation <https://drill.apache.org/docs/> but did
>> not find helpful references for my cause.
>> Thanks to the mailing list, i got hold of a few wiki pages:
>> https://github.com/paul-rogers/drill/wiki/Storage-Plugin-Configuration
>> *https://github.com/paul-rogers/drill/wiki/Storage-Plugin-Model
>> <https://github.com/paul-rogers/drill/wiki/Storage-Plugin-Model>*
>> and other related pages on the same link.
>> I have downloaded the drill source code, cross referenced the storage
>> plugin implementations in there and the information on the wiki pages,
>> learnt about the interfaces to implement, classes to use, configs to set up
>> etc.
>> This has made my understanding a smidgen better, but i am still unable to
>> get a clear picture of the workflow.
>> Can someone please guide me in this regard or provide a few more pointers?
>>
Re: Custom extension storage plugin development
Posted by Muhammad Gelbana <m....@gmail.com>.
I assume this page
<https://github.com/paul-rogers/drill/wiki/Storage-Plugin-Runtime-Interface>
can give you the overview you are looking for ? May be you can start with
asking specific questions.
Thanks,
Gelbana
On Tue, Aug 29, 2017 at 12:26 PM, Charuta Rajopadhye <ch...@gmail.com>
wrote:
> Hi Team,
>
> I need to develop an extension storage plugin for my data source.
> I have been through this
> <https://stackoverflow.com/questions/37557388/how-to-
> write-custom-storage-plugin-for-apache-drill/37646421#37646421>
> and Apache Drill's documentation <https://drill.apache.org/docs/> but did
> not find helpful references for my cause.
> Thanks to the mailing list, i got hold of a few wiki pages:
> https://github.com/paul-rogers/drill/wiki/Storage-Plugin-Configuration
> *https://github.com/paul-rogers/drill/wiki/Storage-Plugin-Model
> <https://github.com/paul-rogers/drill/wiki/Storage-Plugin-Model>*
> and other related pages on the same link.
> I have downloaded the drill source code, cross referenced the storage
> plugin implementations in there and the information on the wiki pages,
> learnt about the interfaces to implement, classes to use, configs to set up
> etc.
> This has made my understanding a smidgen better, but i am still unable to
> get a clear picture of the workflow.
> Can someone please guide me in this regard or provide a few more pointers?
>