Some initial thoughts on making all plugins reloadable

[Leif Hedstrom <zw...@apache.org>]

Hi all,

I started this slide show with some ideas how we can solve the “90%” of the problem (i.e. making all plugins reloadable, both code and configs). See the link at

https://www.icloud.com/keynote/AwBUCAESENdl8ooGRL1RjGFpC0Jt03saKd2FE7OokWOx4mDdeu2WkoGhPN-GT6KxDq0Sk01DmcUHUWsF7og8i8kJMCUCAQEEIMibNldjSDbtsm5RR0runUjgkB9WK2a2H5OJy1q3jOxs?redirectReason=notFound#Reloading_plugins <https://www.icloud.com/keynote/AwBUCAESENdl8ooGRL1RjGFpC0Jt03saKd2FE7OokWOx4mDdeu2WkoGhPN-GT6KxDq0Sk01DmcUHUWsF7og8i8kJMCUCAQEEIMibNldjSDbtsm5RR0runUjgkB9WK2a2H5OJy1q3jOxs?redirectReason=notFound#Reloading_plugins>



This is not complete yet, but looking for input / comments so we can maybe discuss this for the next summit. The big picture idea is that we

1) Load plugin.config and remap.config as a unit. Either file changes, they both reload.

2) These loaded units are ref-counted, on a  per-session basis. This means the same plugins / remap rules applies to all transaction on a session.

3) Plugins no longer have to worry about things such as ref-counting their configs / continuations, and we eliminate the (crazy :) time-delayed deletes of remap configurations.


This is not 100% compatible with current behavior (remap.config ls per transaction), but I think it’s worth that compromise. I think the above would be fairly straight forward, with a new struct (atomically swappable?) that holds both remap and plugins. I haven’t thought about all potential issues here, and this obviously still doesn’t solve the issue of gracefully restarting the core, but I think this could solve some of the most serious issues we have today (plugin code reload and plugin config reload).

Cheers,

— Leif

Re: Some initial thoughts on making all plugins reloadable

[Phil Sorber <so...@apache.org>]

On Tue, Jun 16, 2015 at 4:54 PM Leif Hedstrom <zw...@apache.org> wrote:

> Hi all,
>
> I started this slide show with some ideas how we can solve the “90%” of
> the problem (i.e. making all plugins reloadable, both code and configs).
> See the link at
>
>
> https://www.icloud.com/keynote/AwBUCAESENdl8ooGRL1RjGFpC0Jt03saKd2FE7OokWOx4mDdeu2WkoGhPN-GT6KxDq0Sk01DmcUHUWsF7og8i8kJMCUCAQEEIMibNldjSDbtsm5RR0runUjgkB9WK2a2H5OJy1q3jOxs?redirectReason=notFound#Reloading_plugins
> <
> https://www.icloud.com/keynote/AwBUCAESENdl8ooGRL1RjGFpC0Jt03saKd2FE7OokWOx4mDdeu2WkoGhPN-GT6KxDq0Sk01DmcUHUWsF7og8i8kJMCUCAQEEIMibNldjSDbtsm5RR0runUjgkB9WK2a2H5OJy1q3jOxs?redirectReason=notFound#Reloading_plugins
> >
>
>
>
> This is not complete yet, but looking for input / comments so we can maybe
> discuss this for the next summit. The big picture idea is that we
>
> 1) Load plugin.config and remap.config as a unit. Either file changes,
> they both reload.
>
> 2) These loaded units are ref-counted, on a  per-session basis. This means
> the same plugins / remap rules applies to all transaction on a session.
>
> 3) Plugins no longer have to worry about things such as ref-counting their
> configs / continuations, and we eliminate the (crazy :) time-delayed
> deletes of remap configurations.
>
>
> This is not 100% compatible with current behavior (remap.config ls per
> transaction), but I think it’s worth that compromise. I think the above
> would be fairly straight forward, with a new struct (atomically swappable?)
> that holds both remap and plugins. I haven’t thought about all potential
> issues here, and this obviously still doesn’t solve the issue of gracefully
> restarting the core, but I think this could solve some of the most serious
> issues we have today (plugin code reload and plugin config reload).
>
> Cheers,
>
> — Leif
>
>
We've talked offline about this, but I am generally in agreement here. I'll
also volunteer to give this some thorough testing in prod before 7.0.0.