You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@karaf.apache.org by Tim Ward <td...@telensa.com> on 2016/11/28 13:05:59 UTC
Configuration file handling?
I'm trying to work out how to handle configuration of a system deployed
to Karaf.
I can see that configuration items put into etc/<PID>.cfg end up being
passed to the @Activate method (or whatever), and that you can change
configuration either by editing the .cfg file or from the Karaf command
line (or, I'm guessing, from a JMX console). So that's all fine, I think.
The parts of the process that I don't yet understand are
(a) getting the .cfg file into etc in the first place
(b) what happens on upgrade.
Let's say the Java source files for the code are in git, and get built
into bundles using Eclipse, and the bundles are installed into Karaf by
some mechanism (I gather that there are some choices, such as simply
dropping the bundle files into the deploy directory). So the first
question is, how do the initial, default, states of the .cfg files get
from git into the etc directory (I'm hoping for a less error prone
answer than checking them out manually and copying them manually)?
Then, the life cycle of a configuration file in other contexts is typically
(1) when the software is first installed, the initial, default state of
the configuration file gets installed at the same time as part of the
same process
(2) the user then edits the configuration file to suit this particular
deployment
(3) a later upgrade to the software comes with a new version of the
configuration file containing some new items, and the upgrade process
must ensure that neither these new items nor the user changes at (2) get
lost.
So how is this managed?
What I've found so far is that one can create a "feature" and use
<configfile>. But the documentation I've seen doesn't appear to cope
with upgrade - I think it said that new versions of config files would
be silently discarded if an old version was already there? - which
doesn't meet the case (our Operations people get quite cross with
upgrades that do this). At the very very least there needs to be a clear
warning flagged up to the user that they need to do a manual merge.
And, what is the "URL" that one puts in a <configfile>, assuming that
there's a solution to the upgrade issue?
To summarise my questions:
(A) What are the options for getting initial, default config files from
git to etc?
(B) How do people cope with the upgrade issue?
(C) If features and <configfile> are part of the solution, what's the "URL"?
Thanks.
--
Tim Ward
Re: Configuration file handling?
Posted by Guillaume Nodet <gn...@apache.org>.
There is an append xml attribute you can set on the config. It will only
add new attributes, so that if there is an existing configuration, all its
properties will be kept, it will only be extended with the entries from the
feature.
2016-11-28 17:11 GMT+01:00 Tim Ward <td...@telensa.com>:
> OK, nearly there, thanks all so far!
>
> So <config/> updates the etc folder, and keeps all my data in one place,
> so that looks like the way to go ...
>
> ... give or take the upgrade problem. So what happens if
>
> (a) a new version of a feature is installed which sets a new config item
> which wasn't already in the etc file
> (b) a new version of a feature is installed which sets a config item which
> is already in the etc file with a different value
>
> ?
>
> If the answers are
>
> (a) the new config item gets written to the etc file
> (b) the existing config item doesn't get overwritten
>
> then that solves the problem, but I don't see that much detail in the
> documentation?
>
> Oh, and, I note that deleting the contents of the data directory is stated
> to clear your Karaf down to a known state. But in fact it doesn't do this,
> because etc is outside data, so garbage config files and items can get left
> behind. What do people normally do about this?
>
>
> On 28/11/2016 16:03, Jean-Baptiste Onofré wrote:
>
>> Actually, feature <config/> now populate the etc folder as well (since
>> Karaf 4.0.5 AFAIR).
>>
>> Regards
>> JB
>>
>> On 11/28/2016 02:50 PM, Christian Schneider wrote:
>>
>>> You already found the confiFile option for features. This is the most
>>> widely used option.
>>> The alternative is the config option which simply adds the config in
>>> config admin but not in etc.
>>>
>>> Both variants do not cover the upgrade case. A simply way is to just
>>> remove the old config to make sure the new default one is written. There
>>> is no built in mechanism to preserve user changes in karaf.
>>>
>>> Christian
>>>
>>> On 28.11.2016 14:05, Tim Ward wrote:
>>>
>>>> I'm trying to work out how to handle configuration of a system
>>>> deployed to Karaf.
>>>>
>>>> I can see that configuration items put into etc/<PID>.cfg end up being
>>>> passed to the @Activate method (or whatever), and that you can change
>>>> configuration either by editing the .cfg file or from the Karaf
>>>> command line (or, I'm guessing, from a JMX console). So that's all
>>>> fine, I think.
>>>>
>>>> The parts of the process that I don't yet understand are
>>>>
>>>> (a) getting the .cfg file into etc in the first place
>>>> (b) what happens on upgrade.
>>>>
>>>> Let's say the Java source files for the code are in git, and get built
>>>> into bundles using Eclipse, and the bundles are installed into Karaf
>>>> by some mechanism (I gather that there are some choices, such as
>>>> simply dropping the bundle files into the deploy directory). So the
>>>> first question is, how do the initial, default, states of the .cfg
>>>> files get from git into the etc directory (I'm hoping for a less error
>>>> prone answer than checking them out manually and copying them manually)?
>>>>
>>>> Then, the life cycle of a configuration file in other contexts is
>>>> typically
>>>>
>>>> (1) when the software is first installed, the initial, default state
>>>> of the configuration file gets installed at the same time as part of
>>>> the same process
>>>> (2) the user then edits the configuration file to suit this particular
>>>> deployment
>>>> (3) a later upgrade to the software comes with a new version of the
>>>> configuration file containing some new items, and the upgrade process
>>>> must ensure that neither these new items nor the user changes at (2)
>>>> get lost.
>>>>
>>>> So how is this managed?
>>>>
>>>> What I've found so far is that one can create a "feature" and use
>>>> <configfile>. But the documentation I've seen doesn't appear to cope
>>>> with upgrade - I think it said that new versions of config files would
>>>> be silently discarded if an old version was already there? - which
>>>> doesn't meet the case (our Operations people get quite cross with
>>>> upgrades that do this). At the very very least there needs to be a
>>>> clear warning flagged up to the user that they need to do a manual
>>>> merge.
>>>>
>>>> And, what is the "URL" that one puts in a <configfile>, assuming that
>>>> there's a solution to the upgrade issue?
>>>>
>>>> To summarise my questions:
>>>>
>>>> (A) What are the options for getting initial, default config files
>>>> from git to etc?
>>>> (B) How do people cope with the upgrade issue?
>>>> (C) If features and <configfile> are part of the solution, what's the
>>>> "URL"?
>>>>
>>>> Thanks.
>>>>
>>>>
>>>
>>>
>>
>
> --
> Tim Ward
>
>
--
------------------------
Guillaume Nodet
------------------------
Red Hat, Open Source Integration
Email: gnodet@redhat.com
Web: http://fusesource.com
Blog: http://gnodet.blogspot.com/
Re: Configuration file handling?
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
You can use the append attribute on feature config.
However, it won't override, it's only append. And it's the expected
behavior as if an user changes the properties, we don't want to override
the "custom" values.
Regards
JB
On 11/28/2016 05:11 PM, Tim Ward wrote:
> OK, nearly there, thanks all so far!
>
> So <config/> updates the etc folder, and keeps all my data in one place,
> so that looks like the way to go ...
>
> ... give or take the upgrade problem. So what happens if
>
> (a) a new version of a feature is installed which sets a new config item
> which wasn't already in the etc file
> (b) a new version of a feature is installed which sets a config item
> which is already in the etc file with a different value
>
> ?
>
> If the answers are
>
> (a) the new config item gets written to the etc file
> (b) the existing config item doesn't get overwritten
>
> then that solves the problem, but I don't see that much detail in the
> documentation?
>
> Oh, and, I note that deleting the contents of the data directory is
> stated to clear your Karaf down to a known state. But in fact it doesn't
> do this, because etc is outside data, so garbage config files and items
> can get left behind. What do people normally do about this?
>
> On 28/11/2016 16:03, Jean-Baptiste Onofr� wrote:
>> Actually, feature <config/> now populate the etc folder as well (since
>> Karaf 4.0.5 AFAIR).
>>
>> Regards
>> JB
>>
>> On 11/28/2016 02:50 PM, Christian Schneider wrote:
>>> You already found the confiFile option for features. This is the most
>>> widely used option.
>>> The alternative is the config option which simply adds the config in
>>> config admin but not in etc.
>>>
>>> Both variants do not cover the upgrade case. A simply way is to just
>>> remove the old config to make sure the new default one is written. There
>>> is no built in mechanism to preserve user changes in karaf.
>>>
>>> Christian
>>>
>>> On 28.11.2016 14:05, Tim Ward wrote:
>>>> I'm trying to work out how to handle configuration of a system
>>>> deployed to Karaf.
>>>>
>>>> I can see that configuration items put into etc/<PID>.cfg end up being
>>>> passed to the @Activate method (or whatever), and that you can change
>>>> configuration either by editing the .cfg file or from the Karaf
>>>> command line (or, I'm guessing, from a JMX console). So that's all
>>>> fine, I think.
>>>>
>>>> The parts of the process that I don't yet understand are
>>>>
>>>> (a) getting the .cfg file into etc in the first place
>>>> (b) what happens on upgrade.
>>>>
>>>> Let's say the Java source files for the code are in git, and get built
>>>> into bundles using Eclipse, and the bundles are installed into Karaf
>>>> by some mechanism (I gather that there are some choices, such as
>>>> simply dropping the bundle files into the deploy directory). So the
>>>> first question is, how do the initial, default, states of the .cfg
>>>> files get from git into the etc directory (I'm hoping for a less error
>>>> prone answer than checking them out manually and copying them
>>>> manually)?
>>>>
>>>> Then, the life cycle of a configuration file in other contexts is
>>>> typically
>>>>
>>>> (1) when the software is first installed, the initial, default state
>>>> of the configuration file gets installed at the same time as part of
>>>> the same process
>>>> (2) the user then edits the configuration file to suit this particular
>>>> deployment
>>>> (3) a later upgrade to the software comes with a new version of the
>>>> configuration file containing some new items, and the upgrade process
>>>> must ensure that neither these new items nor the user changes at (2)
>>>> get lost.
>>>>
>>>> So how is this managed?
>>>>
>>>> What I've found so far is that one can create a "feature" and use
>>>> <configfile>. But the documentation I've seen doesn't appear to cope
>>>> with upgrade - I think it said that new versions of config files would
>>>> be silently discarded if an old version was already there? - which
>>>> doesn't meet the case (our Operations people get quite cross with
>>>> upgrades that do this). At the very very least there needs to be a
>>>> clear warning flagged up to the user that they need to do a manual
>>>> merge.
>>>>
>>>> And, what is the "URL" that one puts in a <configfile>, assuming that
>>>> there's a solution to the upgrade issue?
>>>>
>>>> To summarise my questions:
>>>>
>>>> (A) What are the options for getting initial, default config files
>>>> from git to etc?
>>>> (B) How do people cope with the upgrade issue?
>>>> (C) If features and <configfile> are part of the solution, what's the
>>>> "URL"?
>>>>
>>>> Thanks.
>>>>
>>>
>>>
>>
>
>
--
Jean-Baptiste Onofr�
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: Configuration file handling?
Posted by Tim Ward <td...@telensa.com>.
OK, nearly there, thanks all so far!
So <config/> updates the etc folder, and keeps all my data in one place,
so that looks like the way to go ...
... give or take the upgrade problem. So what happens if
(a) a new version of a feature is installed which sets a new config item
which wasn't already in the etc file
(b) a new version of a feature is installed which sets a config item
which is already in the etc file with a different value
?
If the answers are
(a) the new config item gets written to the etc file
(b) the existing config item doesn't get overwritten
then that solves the problem, but I don't see that much detail in the
documentation?
Oh, and, I note that deleting the contents of the data directory is
stated to clear your Karaf down to a known state. But in fact it doesn't
do this, because etc is outside data, so garbage config files and items
can get left behind. What do people normally do about this?
On 28/11/2016 16:03, Jean-Baptiste Onofr� wrote:
> Actually, feature <config/> now populate the etc folder as well (since
> Karaf 4.0.5 AFAIR).
>
> Regards
> JB
>
> On 11/28/2016 02:50 PM, Christian Schneider wrote:
>> You already found the confiFile option for features. This is the most
>> widely used option.
>> The alternative is the config option which simply adds the config in
>> config admin but not in etc.
>>
>> Both variants do not cover the upgrade case. A simply way is to just
>> remove the old config to make sure the new default one is written. There
>> is no built in mechanism to preserve user changes in karaf.
>>
>> Christian
>>
>> On 28.11.2016 14:05, Tim Ward wrote:
>>> I'm trying to work out how to handle configuration of a system
>>> deployed to Karaf.
>>>
>>> I can see that configuration items put into etc/<PID>.cfg end up being
>>> passed to the @Activate method (or whatever), and that you can change
>>> configuration either by editing the .cfg file or from the Karaf
>>> command line (or, I'm guessing, from a JMX console). So that's all
>>> fine, I think.
>>>
>>> The parts of the process that I don't yet understand are
>>>
>>> (a) getting the .cfg file into etc in the first place
>>> (b) what happens on upgrade.
>>>
>>> Let's say the Java source files for the code are in git, and get built
>>> into bundles using Eclipse, and the bundles are installed into Karaf
>>> by some mechanism (I gather that there are some choices, such as
>>> simply dropping the bundle files into the deploy directory). So the
>>> first question is, how do the initial, default, states of the .cfg
>>> files get from git into the etc directory (I'm hoping for a less error
>>> prone answer than checking them out manually and copying them
>>> manually)?
>>>
>>> Then, the life cycle of a configuration file in other contexts is
>>> typically
>>>
>>> (1) when the software is first installed, the initial, default state
>>> of the configuration file gets installed at the same time as part of
>>> the same process
>>> (2) the user then edits the configuration file to suit this particular
>>> deployment
>>> (3) a later upgrade to the software comes with a new version of the
>>> configuration file containing some new items, and the upgrade process
>>> must ensure that neither these new items nor the user changes at (2)
>>> get lost.
>>>
>>> So how is this managed?
>>>
>>> What I've found so far is that one can create a "feature" and use
>>> <configfile>. But the documentation I've seen doesn't appear to cope
>>> with upgrade - I think it said that new versions of config files would
>>> be silently discarded if an old version was already there? - which
>>> doesn't meet the case (our Operations people get quite cross with
>>> upgrades that do this). At the very very least there needs to be a
>>> clear warning flagged up to the user that they need to do a manual
>>> merge.
>>>
>>> And, what is the "URL" that one puts in a <configfile>, assuming that
>>> there's a solution to the upgrade issue?
>>>
>>> To summarise my questions:
>>>
>>> (A) What are the options for getting initial, default config files
>>> from git to etc?
>>> (B) How do people cope with the upgrade issue?
>>> (C) If features and <configfile> are part of the solution, what's the
>>> "URL"?
>>>
>>> Thanks.
>>>
>>
>>
>
--
Tim Ward
Re: Configuration file handling?
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Actually, feature <config/> now populate the etc folder as well (since
Karaf 4.0.5 AFAIR).
Regards
JB
On 11/28/2016 02:50 PM, Christian Schneider wrote:
> You already found the confiFile option for features. This is the most
> widely used option.
> The alternative is the config option which simply adds the config in
> config admin but not in etc.
>
> Both variants do not cover the upgrade case. A simply way is to just
> remove the old config to make sure the new default one is written. There
> is no built in mechanism to preserve user changes in karaf.
>
> Christian
>
> On 28.11.2016 14:05, Tim Ward wrote:
>> I'm trying to work out how to handle configuration of a system
>> deployed to Karaf.
>>
>> I can see that configuration items put into etc/<PID>.cfg end up being
>> passed to the @Activate method (or whatever), and that you can change
>> configuration either by editing the .cfg file or from the Karaf
>> command line (or, I'm guessing, from a JMX console). So that's all
>> fine, I think.
>>
>> The parts of the process that I don't yet understand are
>>
>> (a) getting the .cfg file into etc in the first place
>> (b) what happens on upgrade.
>>
>> Let's say the Java source files for the code are in git, and get built
>> into bundles using Eclipse, and the bundles are installed into Karaf
>> by some mechanism (I gather that there are some choices, such as
>> simply dropping the bundle files into the deploy directory). So the
>> first question is, how do the initial, default, states of the .cfg
>> files get from git into the etc directory (I'm hoping for a less error
>> prone answer than checking them out manually and copying them manually)?
>>
>> Then, the life cycle of a configuration file in other contexts is
>> typically
>>
>> (1) when the software is first installed, the initial, default state
>> of the configuration file gets installed at the same time as part of
>> the same process
>> (2) the user then edits the configuration file to suit this particular
>> deployment
>> (3) a later upgrade to the software comes with a new version of the
>> configuration file containing some new items, and the upgrade process
>> must ensure that neither these new items nor the user changes at (2)
>> get lost.
>>
>> So how is this managed?
>>
>> What I've found so far is that one can create a "feature" and use
>> <configfile>. But the documentation I've seen doesn't appear to cope
>> with upgrade - I think it said that new versions of config files would
>> be silently discarded if an old version was already there? - which
>> doesn't meet the case (our Operations people get quite cross with
>> upgrades that do this). At the very very least there needs to be a
>> clear warning flagged up to the user that they need to do a manual merge.
>>
>> And, what is the "URL" that one puts in a <configfile>, assuming that
>> there's a solution to the upgrade issue?
>>
>> To summarise my questions:
>>
>> (A) What are the options for getting initial, default config files
>> from git to etc?
>> (B) How do people cope with the upgrade issue?
>> (C) If features and <configfile> are part of the solution, what's the
>> "URL"?
>>
>> Thanks.
>>
>
>
--
Jean-Baptiste Onofr�
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: Configuration file handling?
Posted by Guillaume Nodet <gn...@apache.org>.
2016-11-28 14:50 GMT+01:00 Christian Schneider <ch...@die-schneider.net>:
> You already found the confiFile option for features. This is the most
> widely used option.
> The alternative is the config option which simply adds the config in
> config admin but not in etc.
>
Fwiw, both should add the configuration to the etc/ folder.
The main difference is that <configfile/> use an external file which can be
any kind of file (an xml, or whatever) while <config/> use an inlined
properties file and write the config both to config admin and to the etc/
folder.
>
> Both variants do not cover the upgrade case. A simply way is to just
> remove the old config to make sure the new default one is written. There is
> no built in mechanism to preserve user changes in karaf.
>
> Christian
>
> On 28.11.2016 14:05, Tim Ward wrote:
>
>> I'm trying to work out how to handle configuration of a system deployed
>> to Karaf.
>>
>> I can see that configuration items put into etc/<PID>.cfg end up being
>> passed to the @Activate method (or whatever), and that you can change
>> configuration either by editing the .cfg file or from the Karaf command
>> line (or, I'm guessing, from a JMX console). So that's all fine, I think.
>>
>> The parts of the process that I don't yet understand are
>>
>> (a) getting the .cfg file into etc in the first place
>> (b) what happens on upgrade.
>>
>> Let's say the Java source files for the code are in git, and get built
>> into bundles using Eclipse, and the bundles are installed into Karaf by
>> some mechanism (I gather that there are some choices, such as simply
>> dropping the bundle files into the deploy directory). So the first question
>> is, how do the initial, default, states of the .cfg files get from git into
>> the etc directory (I'm hoping for a less error prone answer than checking
>> them out manually and copying them manually)?
>>
>> Then, the life cycle of a configuration file in other contexts is
>> typically
>>
>> (1) when the software is first installed, the initial, default state of
>> the configuration file gets installed at the same time as part of the same
>> process
>> (2) the user then edits the configuration file to suit this particular
>> deployment
>> (3) a later upgrade to the software comes with a new version of the
>> configuration file containing some new items, and the upgrade process must
>> ensure that neither these new items nor the user changes at (2) get lost.
>>
>> So how is this managed?
>>
>> What I've found so far is that one can create a "feature" and use
>> <configfile>. But the documentation I've seen doesn't appear to cope with
>> upgrade - I think it said that new versions of config files would be
>> silently discarded if an old version was already there? - which doesn't
>> meet the case (our Operations people get quite cross with upgrades that do
>> this). At the very very least there needs to be a clear warning flagged up
>> to the user that they need to do a manual merge.
>>
>> And, what is the "URL" that one puts in a <configfile>, assuming that
>> there's a solution to the upgrade issue?
>>
>> To summarise my questions:
>>
>> (A) What are the options for getting initial, default config files from
>> git to etc?
>> (B) How do people cope with the upgrade issue?
>> (C) If features and <configfile> are part of the solution, what's the
>> "URL"?
>>
>> Thanks.
>>
>>
>
> --
> Christian Schneider
> http://www.liquid-reality.de
>
> Open Source Architect
> http://www.talend.com
>
>
--
------------------------
Guillaume Nodet
------------------------
Red Hat, Open Source Integration
Email: gnodet@redhat.com
Web: http://fusesource.com
Blog: http://gnodet.blogspot.com/
Re: Configuration file handling?
Posted by Achim Nierbeck <bc...@googlemail.com>.
Hi Tim,
as Christian already stated, you have two different ways of configuring.
1) inisde the feature itself as <config> element
2) as external location, for example an URI to a resource located in a
Maven Repository. But here you can reference any valid URI pattern where a
corresponding URI handler is registered.
An example for such a use-case may also be found here [1], it's the feature
file for Pax-Web, it references an external file located in a Maven
Repository.
regards, Achim
[1] -
https://github.com/ops4j/org.ops4j.pax.web/blob/master/pax-web-features/src/main/resources/features.xml#L91
2016-11-28 16:22 GMT+01:00 Tim Ward <td...@telensa.com>:
> On 28/11/2016 15:14, Christian Schneider wrote:
>
>> On 28.11.2016 15:14, Tim Ward wrote:
>>
>>> Thanks - replies inline:
>>>
>>> On 28/11/2016 13:50, Christian Schneider wrote:
>>>
>>>> You already found the confiFile option for features. This is the most
>>>> widely used option.
>>>>
>>>
>>> So what is the "URL" in the <configFile> element, and how do my config
>>> files get there from git?
>>>
>> The URL typically is a maven url. See https://github.com/apache/kara
>> f-decanter/blob/master/assembly/src/main/feature/feature.xml#L241
>> The default configs are deployed there using build-helper-maven-plugin:
>> https://github.com/apache/karaf-decanter/blob/master/appende
>> r/kafka/pom.xml#L50-L69
>>
>
> Sorry, I'm not sure I get that.
>
> Where, physically, do the config files go? (I've never used a "Maven URL",
> and we use bndtools not Maven for building.)
>
> I was expecting something where the config files were somewhere inside the
> bundle or feature being deployed.
>
> --
> Tim Ward
>
>
--
Apache Member
Apache Karaf <http://karaf.apache.org/> Committer & PMC
OPS4J Pax Web <http://wiki.ops4j.org/display/paxweb/Pax+Web/> Committer &
Project Lead
blog <http://notizblog.nierbeck.de/>
Co-Author of Apache Karaf Cookbook <http://bit.ly/1ps9rkS>
Software Architect / Project Manager / Scrum Master
Re: Configuration file handling?
Posted by Tim Ward <td...@telensa.com>.
On 28/11/2016 15:14, Christian Schneider wrote:
> On 28.11.2016 15:14, Tim Ward wrote:
>> Thanks - replies inline:
>>
>> On 28/11/2016 13:50, Christian Schneider wrote:
>>> You already found the confiFile option for features. This is the
>>> most widely used option.
>>
>> So what is the "URL" in the <configFile> element, and how do my
>> config files get there from git?
> The URL typically is a maven url. See
> https://github.com/apache/karaf-decanter/blob/master/assembly/src/main/feature/feature.xml#L241
> The default configs are deployed there using
> build-helper-maven-plugin:
> https://github.com/apache/karaf-decanter/blob/master/appender/kafka/pom.xml#L50-L69
Sorry, I'm not sure I get that.
Where, physically, do the config files go? (I've never used a "Maven
URL", and we use bndtools not Maven for building.)
I was expecting something where the config files were somewhere inside
the bundle or feature being deployed.
--
Tim Ward
Re: Configuration file handling?
Posted by Christian Schneider <ch...@die-schneider.net>.
On 28.11.2016 15:14, Tim Ward wrote:
> Thanks - replies inline:
>
> On 28/11/2016 13:50, Christian Schneider wrote:
>> You already found the confiFile option for features. This is the most
>> widely used option.
>
> So what is the "URL" in the <configFile> element, and how do my config
> files get there from git?
The URL typically is a maven url. See
https://github.com/apache/karaf-decanter/blob/master/assembly/src/main/feature/feature.xml#L241
The default configs are deployed there using build-helper-maven-plugin:
https://github.com/apache/karaf-decanter/blob/master/appender/kafka/pom.xml#L50-L69
>> The alternative is the config option which simply adds the config in
>> config admin but not in etc.
>>
>> Both variants do not cover the upgrade case. A simply way is to just
>> remove the old config to make sure the new default one is written.
>> There is no built in mechanism to preserve user changes in karaf.
>
> "just remove the old config" suggests either a manual process (which I
> can't see going down well with Ops) or some non-trivial upgrade
> scripting (which I fear we're going to need anyway). And "just remove
> the old config" doesn't meet the case, as it'll result in a system
> that was working no longer working.
>
> What do people do in practice? - every non-trivial system with a
> lifetime of more than one version must have come across this upgrade
> issue. Write an upgrade script which detects (how?) that an incoming
> feature has an update for a config file, quarantines one version of
> the file somewhere, and tells the user in big red letters that they've
> got some manual merging to do? Do such scripts, or frameworks for
> writing them, exist, to save wheel-reinventing, as this must be a
> common question?
I know two different approaches:
1. Manual upgrades
2. Immutable server pattern: Install scripts that deploy code and the
actual configuration. In these cases the config is typically not changed
by the admin.
As such scripts depend a lot on the customer there is no out of the box
solution.
Christian
--
Christian Schneider
http://www.liquid-reality.de
Open Source Architect
http://www.talend.com
Re: Configuration file handling?
Posted by Tim Ward <td...@telensa.com>.
Thanks - replies inline:
On 28/11/2016 13:50, Christian Schneider wrote:
> You already found the confiFile option for features. This is the most
> widely used option.
So what is the "URL" in the <configFile> element, and how do my config
files get there from git?
> The alternative is the config option which simply adds the config in
> config admin but not in etc.
>
> Both variants do not cover the upgrade case. A simply way is to just
> remove the old config to make sure the new default one is written.
> There is no built in mechanism to preserve user changes in karaf.
"just remove the old config" suggests either a manual process (which I
can't see going down well with Ops) or some non-trivial upgrade
scripting (which I fear we're going to need anyway). And "just remove
the old config" doesn't meet the case, as it'll result in a system that
was working no longer working.
What do people do in practice? - every non-trivial system with a
lifetime of more than one version must have come across this upgrade
issue. Write an upgrade script which detects (how?) that an incoming
feature has an update for a config file, quarantines one version of the
file somewhere, and tells the user in big red letters that they've got
some manual merging to do? Do such scripts, or frameworks for writing
them, exist, to save wheel-reinventing, as this must be a common question?
> Christian
>
> On 28.11.2016 14:05, Tim Ward wrote:
>> I'm trying to work out how to handle configuration of a system
>> deployed to Karaf.
>>
>> I can see that configuration items put into etc/<PID>.cfg end up
>> being passed to the @Activate method (or whatever), and that you can
>> change configuration either by editing the .cfg file or from the
>> Karaf command line (or, I'm guessing, from a JMX console). So that's
>> all fine, I think.
>>
>> The parts of the process that I don't yet understand are
>>
>> (a) getting the .cfg file into etc in the first place
>> (b) what happens on upgrade.
>>
>> Let's say the Java source files for the code are in git, and get
>> built into bundles using Eclipse, and the bundles are installed into
>> Karaf by some mechanism (I gather that there are some choices, such
>> as simply dropping the bundle files into the deploy directory). So
>> the first question is, how do the initial, default, states of the
>> .cfg files get from git into the etc directory (I'm hoping for a less
>> error prone answer than checking them out manually and copying them
>> manually)?
>>
>> Then, the life cycle of a configuration file in other contexts is
>> typically
>>
>> (1) when the software is first installed, the initial, default state
>> of the configuration file gets installed at the same time as part of
>> the same process
>> (2) the user then edits the configuration file to suit this
>> particular deployment
>> (3) a later upgrade to the software comes with a new version of the
>> configuration file containing some new items, and the upgrade process
>> must ensure that neither these new items nor the user changes at (2)
>> get lost.
>>
>> So how is this managed?
>>
>> What I've found so far is that one can create a "feature" and use
>> <configfile>. But the documentation I've seen doesn't appear to cope
>> with upgrade - I think it said that new versions of config files
>> would be silently discarded if an old version was already there? -
>> which doesn't meet the case (our Operations people get quite cross
>> with upgrades that do this). At the very very least there needs to be
>> a clear warning flagged up to the user that they need to do a manual
>> merge.
>>
>> And, what is the "URL" that one puts in a <configfile>, assuming that
>> there's a solution to the upgrade issue?
>>
>> To summarise my questions:
>>
>> (A) What are the options for getting initial, default config files
>> from git to etc?
>> (B) How do people cope with the upgrade issue?
>> (C) If features and <configfile> are part of the solution, what's the
>> "URL"?
>>
>> Thanks.
>>
>
>
--
Tim Ward
Re: Configuration file handling?
Posted by Christian Schneider <ch...@die-schneider.net>.
You already found the confiFile option for features. This is the most
widely used option.
The alternative is the config option which simply adds the config in
config admin but not in etc.
Both variants do not cover the upgrade case. A simply way is to just
remove the old config to make sure the new default one is written. There
is no built in mechanism to preserve user changes in karaf.
Christian
On 28.11.2016 14:05, Tim Ward wrote:
> I'm trying to work out how to handle configuration of a system
> deployed to Karaf.
>
> I can see that configuration items put into etc/<PID>.cfg end up being
> passed to the @Activate method (or whatever), and that you can change
> configuration either by editing the .cfg file or from the Karaf
> command line (or, I'm guessing, from a JMX console). So that's all
> fine, I think.
>
> The parts of the process that I don't yet understand are
>
> (a) getting the .cfg file into etc in the first place
> (b) what happens on upgrade.
>
> Let's say the Java source files for the code are in git, and get built
> into bundles using Eclipse, and the bundles are installed into Karaf
> by some mechanism (I gather that there are some choices, such as
> simply dropping the bundle files into the deploy directory). So the
> first question is, how do the initial, default, states of the .cfg
> files get from git into the etc directory (I'm hoping for a less error
> prone answer than checking them out manually and copying them manually)?
>
> Then, the life cycle of a configuration file in other contexts is
> typically
>
> (1) when the software is first installed, the initial, default state
> of the configuration file gets installed at the same time as part of
> the same process
> (2) the user then edits the configuration file to suit this particular
> deployment
> (3) a later upgrade to the software comes with a new version of the
> configuration file containing some new items, and the upgrade process
> must ensure that neither these new items nor the user changes at (2)
> get lost.
>
> So how is this managed?
>
> What I've found so far is that one can create a "feature" and use
> <configfile>. But the documentation I've seen doesn't appear to cope
> with upgrade - I think it said that new versions of config files would
> be silently discarded if an old version was already there? - which
> doesn't meet the case (our Operations people get quite cross with
> upgrades that do this). At the very very least there needs to be a
> clear warning flagged up to the user that they need to do a manual merge.
>
> And, what is the "URL" that one puts in a <configfile>, assuming that
> there's a solution to the upgrade issue?
>
> To summarise my questions:
>
> (A) What are the options for getting initial, default config files
> from git to etc?
> (B) How do people cope with the upgrade issue?
> (C) If features and <configfile> are part of the solution, what's the
> "URL"?
>
> Thanks.
>
--
Christian Schneider
http://www.liquid-reality.de
Open Source Architect
http://www.talend.com