Making Drill support graceful downgrades

[Jason Altekruse <al...@gmail.com>]

While generally Drill has been getting more and more stable with the last
few releases, sometimes users need to downgrade for a variety of reasons.
Currently we have Jackson configured to strictly expect only the current
version of a storage or format plugin configuration.

Here is a recent example of a user who ran into this issue:
https://issues.apache.org/jira/browse/DRILL-4231

I think we should either catch errors like this and suggest that users
check their configuration stored in zookeeper, or in the /tmp/ directory if
running in embedded mode.

Another possible change that could be made to help mitigate this issue
going forward is to change the Jackson configuration to ignore unrecognized
fields in the configuration file. As we will be adding additional
configuration options as we try to accommodate more diverse use cases I
think this could be a small but useful change.

Are there any downsides to changing the behavior in this way? The one I can
think is that someone that would update a configuration after a downgrade
would lose their changes to any new properties from the previously
installed newer version. Considering the behavior currently requires
blowing away the configuration to allow Drill to start up after a
downgrade, this doesn't seem like much of an issue.

Thoughts welcome, I'll open a JIRA if there is no opposition.

Re: Making Drill support graceful downgrades

[Jason Altekruse <al...@gmail.com>]

One additional thought on ignoring unrecognized fields, this would give the
behavior of ignoring values with mistyped field names. I think that right
now I will submit a JIRA to just catch the Jackson error and direct users
to check or clear out their configuration in zookeeper or /tmp/drill
directory.

I will probably open a JIRA to track a more complete solution if one is
needed in the future, but for now even the simple change would be helpful.

On Tue, Dec 29, 2015 at 12:10 PM, Jason Altekruse <al...@gmail.com>
wrote:

> While generally Drill has been getting more and more stable with the last
> few releases, sometimes users need to downgrade for a variety of reasons.
> Currently we have Jackson configured to strictly expect only the current
> version of a storage or format plugin configuration.
>
> Here is a recent example of a user who ran into this issue:
> https://issues.apache.org/jira/browse/DRILL-4231
>
> I think we should either catch errors like this and suggest that users
> check their configuration stored in zookeeper, or in the /tmp/ directory if
> running in embedded mode.
>
> Another possible change that could be made to help mitigate this issue
> going forward is to change the Jackson configuration to ignore unrecognized
> fields in the configuration file. As we will be adding additional
> configuration options as we try to accommodate more diverse use cases I
> think this could be a small but useful change.
>
> Are there any downsides to changing the behavior in this way? The one I
> can think is that someone that would update a configuration after a
> downgrade would lose their changes to any new properties from the
> previously installed newer version. Considering the behavior currently
> requires blowing away the configuration to allow Drill to start up after a
> downgrade, this doesn't seem like much of an issue.
>
> Thoughts welcome, I'll open a JIRA if there is no opposition.
>