You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by Charles Givre <cg...@gmail.com> on 2019/03/22 18:13:50 UTC
[DISCUSS] Including Features that Need Regular Updating?
Hello all,
I have a question regarding new Drill features. I have two UDFs which I’ve been considering submitting but these UDFs will require regular updating. The features in question are UDFs to do IP Geolocation and a User Agent Parser. The IP geolocation is dependent on the MaxMind database and associated libraries. Basically if it were to be included in Drill, every release we would have to update the MaxMind DB. (This is done in many tools that rely on it for IP Geolocation)
The other is the user agent parser. Likewise, the only updating it would need would be to update the pom.xml file to reflect the latest version of the UA parser. These are both very useful features for security analysis but I wanted to ask the Drill developer community if this is something we wanted to consider.
— C
Re: [DISCUSS] Including Features that Need Regular Updating?
Posted by Arina Yelchiyeva <ar...@gmail.com>.
I am not sure that library update should be included in the release process. On the contrary, I think it should be done via regular PR process. Of course, you always have to find a volunteer :)
Kind regards,
Arina
> On Mar 22, 2019, at 8:39 PM, Bob Rudis <bo...@rud.is> wrote:
>
> Two rly 👍 UDFs, too!
>
> Charles: many of the FOSS (or even non-FOSS) security tools we both likely tend to use in either of those contexts put the onus of updating the core DBs on the end-user (and try to go out of their way in the docs re: need to do that).
>
> I think as long as the version included with any given Drill release is stated along with verbiage re: need to update (perhaps even provide an update script) would be fine.
>
> One question I'd have for the MM DB is where would https://support.maxmind.com/geolite-faq/general/what-do-i-need-to-do-to-meet-the-geolite2-attribution-requirement/ have to go to meet the letter and spirit of the requirement? Would it need to be in the console? Just on the web site? At $WORK we license it for use in-product so it's not something we have to care abt but in-Drill would seem to be something that'd need to be ironed out.
>
>> On Mar 22, 2019, at 14:31, Charles Givre <cg...@gmail.com> wrote:
>>
>> Boaz,
>> If that’s the case… I’ll have 2 PRs for Drill v 1.17 ;-)
>>
>>> On Mar 22, 2019, at 14:25, Boaz Ben-Zvi <bo...@apache.org> wrote:
>>>
>>> Hi Charles,
>>>
>>> If these updates are only small simple tasks, it would not be a big issue to add them to the Drill Release Process (see [1]).
>>>
>>> BTW, most of the release work is automated via a script (see section 4 in [1]); so if these updates could be automated as well, it would be a trivial matter,
>>>
>>> Thanks for your useful contributions,
>>>
>>> -- Boaz
>>>
>>> [1] https://github.com/parthchandra/drill/wiki/Drill-Release-Process
>>>
>>>> On 3/22/19 11:13 AM, Charles Givre wrote:
>>>> Hello all,
>>>> I have a question regarding new Drill features. I have two UDFs which I’ve been considering submitting but these UDFs will require regular updating. The features in question are UDFs to do IP Geolocation and a User Agent Parser. The IP geolocation is dependent on the MaxMind database and associated libraries. Basically if it were to be included in Drill, every release we would have to update the MaxMind DB. (This is done in many tools that rely on it for IP Geolocation)
>>>>
>>>> The other is the user agent parser. Likewise, the only updating it would need would be to update the pom.xml file to reflect the latest version of the UA parser. These are both very useful features for security analysis but I wanted to ask the Drill developer community if this is something we wanted to consider.
>>>> — C
>>
>
Re: [DISCUSS] Including Features that Need Regular Updating?
Posted by Arina Yelchiyeva <ar...@gmail.com>.
I am not sure that library update should be included in the release process. On the contrary, I think it should be done via regular PR process. Of course, you always have to find a volunteer :)
Kind regards,
Arina
> On Mar 22, 2019, at 8:39 PM, Bob Rudis <bo...@rud.is> wrote:
>
> Two rly 👍 UDFs, too!
>
> Charles: many of the FOSS (or even non-FOSS) security tools we both likely tend to use in either of those contexts put the onus of updating the core DBs on the end-user (and try to go out of their way in the docs re: need to do that).
>
> I think as long as the version included with any given Drill release is stated along with verbiage re: need to update (perhaps even provide an update script) would be fine.
>
> One question I'd have for the MM DB is where would https://support.maxmind.com/geolite-faq/general/what-do-i-need-to-do-to-meet-the-geolite2-attribution-requirement/ have to go to meet the letter and spirit of the requirement? Would it need to be in the console? Just on the web site? At $WORK we license it for use in-product so it's not something we have to care abt but in-Drill would seem to be something that'd need to be ironed out.
>
>> On Mar 22, 2019, at 14:31, Charles Givre <cg...@gmail.com> wrote:
>>
>> Boaz,
>> If that’s the case… I’ll have 2 PRs for Drill v 1.17 ;-)
>>
>>> On Mar 22, 2019, at 14:25, Boaz Ben-Zvi <bo...@apache.org> wrote:
>>>
>>> Hi Charles,
>>>
>>> If these updates are only small simple tasks, it would not be a big issue to add them to the Drill Release Process (see [1]).
>>>
>>> BTW, most of the release work is automated via a script (see section 4 in [1]); so if these updates could be automated as well, it would be a trivial matter,
>>>
>>> Thanks for your useful contributions,
>>>
>>> -- Boaz
>>>
>>> [1] https://github.com/parthchandra/drill/wiki/Drill-Release-Process
>>>
>>>> On 3/22/19 11:13 AM, Charles Givre wrote:
>>>> Hello all,
>>>> I have a question regarding new Drill features. I have two UDFs which I’ve been considering submitting but these UDFs will require regular updating. The features in question are UDFs to do IP Geolocation and a User Agent Parser. The IP geolocation is dependent on the MaxMind database and associated libraries. Basically if it were to be included in Drill, every release we would have to update the MaxMind DB. (This is done in many tools that rely on it for IP Geolocation)
>>>>
>>>> The other is the user agent parser. Likewise, the only updating it would need would be to update the pom.xml file to reflect the latest version of the UA parser. These are both very useful features for security analysis but I wanted to ask the Drill developer community if this is something we wanted to consider.
>>>> — C
>>
>
Re: [DISCUSS] Including Features that Need Regular Updating?
Posted by Bob Rudis <bo...@rud.is>.
Two rly 👍 UDFs, too!
Charles: many of the FOSS (or even non-FOSS) security tools we both likely tend to use in either of those contexts put the onus of updating the core DBs on the end-user (and try to go out of their way in the docs re: need to do that).
I think as long as the version included with any given Drill release is stated along with verbiage re: need to update (perhaps even provide an update script) would be fine.
One question I'd have for the MM DB is where would https://support.maxmind.com/geolite-faq/general/what-do-i-need-to-do-to-meet-the-geolite2-attribution-requirement/ have to go to meet the letter and spirit of the requirement? Would it need to be in the console? Just on the web site? At $WORK we license it for use in-product so it's not something we have to care abt but in-Drill would seem to be something that'd need to be ironed out.
> On Mar 22, 2019, at 14:31, Charles Givre <cg...@gmail.com> wrote:
>
> Boaz,
> If that’s the case… I’ll have 2 PRs for Drill v 1.17 ;-)
>
>> On Mar 22, 2019, at 14:25, Boaz Ben-Zvi <bo...@apache.org> wrote:
>>
>> Hi Charles,
>>
>> If these updates are only small simple tasks, it would not be a big issue to add them to the Drill Release Process (see [1]).
>>
>> BTW, most of the release work is automated via a script (see section 4 in [1]); so if these updates could be automated as well, it would be a trivial matter,
>>
>> Thanks for your useful contributions,
>>
>> -- Boaz
>>
>> [1] https://github.com/parthchandra/drill/wiki/Drill-Release-Process
>>
>> On 3/22/19 11:13 AM, Charles Givre wrote:
>>> Hello all,
>>> I have a question regarding new Drill features. I have two UDFs which I’ve been considering submitting but these UDFs will require regular updating. The features in question are UDFs to do IP Geolocation and a User Agent Parser. The IP geolocation is dependent on the MaxMind database and associated libraries. Basically if it were to be included in Drill, every release we would have to update the MaxMind DB. (This is done in many tools that rely on it for IP Geolocation)
>>>
>>> The other is the user agent parser. Likewise, the only updating it would need would be to update the pom.xml file to reflect the latest version of the UA parser. These are both very useful features for security analysis but I wanted to ask the Drill developer community if this is something we wanted to consider.
>>> — C
>
Re: [DISCUSS] Including Features that Need Regular Updating?
Posted by Bob Rudis <bo...@rud.is>.
Two rly 👍 UDFs, too!
Charles: many of the FOSS (or even non-FOSS) security tools we both likely tend to use in either of those contexts put the onus of updating the core DBs on the end-user (and try to go out of their way in the docs re: need to do that).
I think as long as the version included with any given Drill release is stated along with verbiage re: need to update (perhaps even provide an update script) would be fine.
One question I'd have for the MM DB is where would https://support.maxmind.com/geolite-faq/general/what-do-i-need-to-do-to-meet-the-geolite2-attribution-requirement/ have to go to meet the letter and spirit of the requirement? Would it need to be in the console? Just on the web site? At $WORK we license it for use in-product so it's not something we have to care abt but in-Drill would seem to be something that'd need to be ironed out.
> On Mar 22, 2019, at 14:31, Charles Givre <cg...@gmail.com> wrote:
>
> Boaz,
> If that’s the case… I’ll have 2 PRs for Drill v 1.17 ;-)
>
>> On Mar 22, 2019, at 14:25, Boaz Ben-Zvi <bo...@apache.org> wrote:
>>
>> Hi Charles,
>>
>> If these updates are only small simple tasks, it would not be a big issue to add them to the Drill Release Process (see [1]).
>>
>> BTW, most of the release work is automated via a script (see section 4 in [1]); so if these updates could be automated as well, it would be a trivial matter,
>>
>> Thanks for your useful contributions,
>>
>> -- Boaz
>>
>> [1] https://github.com/parthchandra/drill/wiki/Drill-Release-Process
>>
>> On 3/22/19 11:13 AM, Charles Givre wrote:
>>> Hello all,
>>> I have a question regarding new Drill features. I have two UDFs which I’ve been considering submitting but these UDFs will require regular updating. The features in question are UDFs to do IP Geolocation and a User Agent Parser. The IP geolocation is dependent on the MaxMind database and associated libraries. Basically if it were to be included in Drill, every release we would have to update the MaxMind DB. (This is done in many tools that rely on it for IP Geolocation)
>>>
>>> The other is the user agent parser. Likewise, the only updating it would need would be to update the pom.xml file to reflect the latest version of the UA parser. These are both very useful features for security analysis but I wanted to ask the Drill developer community if this is something we wanted to consider.
>>> — C
>
Re: [DISCUSS] Including Features that Need Regular Updating?
Posted by Charles Givre <cg...@gmail.com>.
Boaz,
If that’s the case… I’ll have 2 PRs for Drill v 1.17 ;-)
> On Mar 22, 2019, at 14:25, Boaz Ben-Zvi <bo...@apache.org> wrote:
>
> Hi Charles,
>
> If these updates are only small simple tasks, it would not be a big issue to add them to the Drill Release Process (see [1]).
>
> BTW, most of the release work is automated via a script (see section 4 in [1]); so if these updates could be automated as well, it would be a trivial matter,
>
> Thanks for your useful contributions,
>
> -- Boaz
>
> [1] https://github.com/parthchandra/drill/wiki/Drill-Release-Process
>
> On 3/22/19 11:13 AM, Charles Givre wrote:
>> Hello all,
>> I have a question regarding new Drill features. I have two UDFs which I’ve been considering submitting but these UDFs will require regular updating. The features in question are UDFs to do IP Geolocation and a User Agent Parser. The IP geolocation is dependent on the MaxMind database and associated libraries. Basically if it were to be included in Drill, every release we would have to update the MaxMind DB. (This is done in many tools that rely on it for IP Geolocation)
>>
>> The other is the user agent parser. Likewise, the only updating it would need would be to update the pom.xml file to reflect the latest version of the UA parser. These are both very useful features for security analysis but I wanted to ask the Drill developer community if this is something we wanted to consider.
>> — C
Re: [DISCUSS] Including Features that Need Regular Updating?
Posted by Charles Givre <cg...@gmail.com>.
Boaz,
If that’s the case… I’ll have 2 PRs for Drill v 1.17 ;-)
> On Mar 22, 2019, at 14:25, Boaz Ben-Zvi <bo...@apache.org> wrote:
>
> Hi Charles,
>
> If these updates are only small simple tasks, it would not be a big issue to add them to the Drill Release Process (see [1]).
>
> BTW, most of the release work is automated via a script (see section 4 in [1]); so if these updates could be automated as well, it would be a trivial matter,
>
> Thanks for your useful contributions,
>
> -- Boaz
>
> [1] https://github.com/parthchandra/drill/wiki/Drill-Release-Process
>
> On 3/22/19 11:13 AM, Charles Givre wrote:
>> Hello all,
>> I have a question regarding new Drill features. I have two UDFs which I’ve been considering submitting but these UDFs will require regular updating. The features in question are UDFs to do IP Geolocation and a User Agent Parser. The IP geolocation is dependent on the MaxMind database and associated libraries. Basically if it were to be included in Drill, every release we would have to update the MaxMind DB. (This is done in many tools that rely on it for IP Geolocation)
>>
>> The other is the user agent parser. Likewise, the only updating it would need would be to update the pom.xml file to reflect the latest version of the UA parser. These are both very useful features for security analysis but I wanted to ask the Drill developer community if this is something we wanted to consider.
>> — C
Re: [DISCUSS] Including Features that Need Regular Updating?
Posted by Boaz Ben-Zvi <bo...@apache.org>.
Hi Charles,
If these updates are only small simple tasks, it would not be a big
issue to add them to the Drill Release Process (see [1]).
BTW, most of the release work is automated via a script (see section 4
in [1]); so if these updates could be automated as well, it would be a
trivial matter,
Thanks for your useful contributions,
-- Boaz
[1] https://github.com/parthchandra/drill/wiki/Drill-Release-Process
On 3/22/19 11:13 AM, Charles Givre wrote:
> Hello all,
> I have a question regarding new Drill features. I have two UDFs which I’ve been considering submitting but these UDFs will require regular updating. The features in question are UDFs to do IP Geolocation and a User Agent Parser. The IP geolocation is dependent on the MaxMind database and associated libraries. Basically if it were to be included in Drill, every release we would have to update the MaxMind DB. (This is done in many tools that rely on it for IP Geolocation)
>
> The other is the user agent parser. Likewise, the only updating it would need would be to update the pom.xml file to reflect the latest version of the UA parser. These are both very useful features for security analysis but I wanted to ask the Drill developer community if this is something we wanted to consider.
> — C
Re: [DISCUSS] Including Features that Need Regular Updating?
Posted by Boaz Ben-Zvi <bo...@apache.org>.
Hi Charles,
If these updates are only small simple tasks, it would not be a big
issue to add them to the Drill Release Process (see [1]).
BTW, most of the release work is automated via a script (see section 4
in [1]); so if these updates could be automated as well, it would be a
trivial matter,
Thanks for your useful contributions,
-- Boaz
[1] https://github.com/parthchandra/drill/wiki/Drill-Release-Process
On 3/22/19 11:13 AM, Charles Givre wrote:
> Hello all,
> I have a question regarding new Drill features. I have two UDFs which I’ve been considering submitting but these UDFs will require regular updating. The features in question are UDFs to do IP Geolocation and a User Agent Parser. The IP geolocation is dependent on the MaxMind database and associated libraries. Basically if it were to be included in Drill, every release we would have to update the MaxMind DB. (This is done in many tools that rely on it for IP Geolocation)
>
> The other is the user agent parser. Likewise, the only updating it would need would be to update the pom.xml file to reflect the latest version of the UA parser. These are both very useful features for security analysis but I wanted to ask the Drill developer community if this is something we wanted to consider.
> — C