You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by Rahul Ravindran <ra...@yahoo.com> on 2012/11/21 20:23:42 UTC
Running multiple flume versions on the same box
Hi,
This is primarily to try and address a flume upgrade scenario in the case of any incompatible changes in future. I tried this with multiple processes of the same version, and it appeared to work. Are there any concerns on running multiple versions of flume on the same box (each with different agent configurations where there is no overlap of ports) ?
Thanks,
~Rahul.
Re: Running multiple flume versions on the same box
Posted by Roshan Naik <ro...@hortonworks.com>.
Inherently there should be no issue in Flume itself.
But you may want to make sure that your flume sinks (old v/s new) are not
configured in a such a way that they step on each other.
Best to do a trial simulation.
-roshan
On Wed, Nov 21, 2012 at 1:29 PM, Camp, Roy <rc...@ebay.com> wrote:
> We did this when upgrading from 0.9x to FlumeNG 1.3-SNAPSHOT. Used
> different ports and different logging/data directories. Worked great.****
>
> ** **
>
> Roy****
>
> ** **
>
> *From:* Rahul Ravindran [mailto:rahulrv@yahoo.com]
> *Sent:* Wednesday, November 21, 2012 11:24 AM
> *To:* User-flume
> *Subject:* Running multiple flume versions on the same box****
>
> ** **
>
> Hi,****
>
> This is primarily to try and address a flume upgrade scenario in the
> case of any incompatible changes in future. I tried this with multiple
> processes of the same version, and it appeared to work. Are there any
> concerns on running multiple versions of flume on the same box (each with
> different agent configurations where there is no overlap of ports) ? ****
>
> ** **
>
> Thanks,****
>
> ~Rahul.****
>
Re: Running multiple flume versions on the same box
Posted by Rahul Ravindran <ra...@yahoo.com>.
Absolutely. I don't expect to need to do this and I expect to be staging any changes prior to a production deployment.
This was a question from our operations on how non-compatible upgrades would be handled, and I needed to get an idea on how we may handle it, if this scenario, however rare does come up.
Thanks for all the info.
~Rahul.
________________________________
From: Mike Percy <mp...@apache.org>
To: user@flume.apache.org
Cc: Rahul Ravindran <ra...@yahoo.com>
Sent: Wednesday, November 21, 2012 2:24 PM
Subject: Re: Running multiple flume versions on the same box
There are no system level singletons or hard-coded file paths or ports if that is what you mean.
But in a production scenario, Flume should be resilient to failures since it will just buffer events in the channel at each agent. So why run simultaneous versions when doing minor version upgrades? (I can understand in an OG -> NG migration) If there is a problem just take it down and roll back; the rest of the system should be fine if you have done sufficient capacity planning (with channel sizes) and configuration to tolerate downtime - which I'd strongly recommend.
At the end of the day, it's always best to test new versions in staging any time you do a software upgrade, including with Flume.
Hope that helps.
Regards,
Mike
On Wed, Nov 21, 2012 at 1:29 PM, Camp, Roy <rc...@ebay.com> wrote:
We did this when upgrading from 0.9x to FlumeNG 1.3-SNAPSHOT. Used different ports and different logging/data directories. Worked great.
>
>Roy
>
>From:Rahul Ravindran [mailto:rahulrv@yahoo.com]
>Sent: Wednesday, November 21, 2012 11:24 AM
>To: User-flume
>Subject: Running multiple flume versions on the same box
>
>Hi,
> This is primarily to try and address a flume upgrade scenario in the case of any incompatible changes in future. I tried this with multiple processes of the same version, and it appeared to work. Are there any concerns on running multiple versions of flume on the same box (each with different agent configurations where there is no overlap of ports) ?
>
>Thanks,
>~Rahul.
Re: Running multiple flume versions on the same box
Posted by Mike Percy <mp...@apache.org>.
There are no system level singletons or hard-coded file paths or ports if
that is what you mean.
But in a production scenario, Flume should be resilient to failures since
it will just buffer events in the channel at each agent. So why run
simultaneous versions when doing minor version upgrades? (I can understand
in an OG -> NG migration) If there is a problem just take it down and roll
back; the rest of the system should be fine if you have done sufficient
capacity planning (with channel sizes) and configuration to tolerate
downtime - which I'd strongly recommend.
At the end of the day, it's always best to test new versions in staging any
time you do a software upgrade, including with Flume.
Hope that helps.
Regards,
Mike
On Wed, Nov 21, 2012 at 1:29 PM, Camp, Roy <rc...@ebay.com> wrote:
> We did this when upgrading from 0.9x to FlumeNG 1.3-SNAPSHOT. Used
> different ports and different logging/data directories. Worked great.****
>
> ** **
>
> Roy****
>
> ** **
>
> *From:* Rahul Ravindran [mailto:rahulrv@yahoo.com]
> *Sent:* Wednesday, November 21, 2012 11:24 AM
> *To:* User-flume
> *Subject:* Running multiple flume versions on the same box****
>
> ** **
>
> Hi,****
>
> This is primarily to try and address a flume upgrade scenario in the
> case of any incompatible changes in future. I tried this with multiple
> processes of the same version, and it appeared to work. Are there any
> concerns on running multiple versions of flume on the same box (each with
> different agent configurations where there is no overlap of ports) ? ****
>
> ** **
>
> Thanks,****
>
> ~Rahul.****
>
RE: Running multiple flume versions on the same box
Posted by "Camp, Roy" <rc...@ebay.com>.
We did this when upgrading from 0.9x to FlumeNG 1.3-SNAPSHOT. Used different ports and different logging/data directories. Worked great.
Roy
From: Rahul Ravindran [mailto:rahulrv@yahoo.com]
Sent: Wednesday, November 21, 2012 11:24 AM
To: User-flume
Subject: Running multiple flume versions on the same box
Hi,
This is primarily to try and address a flume upgrade scenario in the case of any incompatible changes in future. I tried this with multiple processes of the same version, and it appeared to work. Are there any concerns on running multiple versions of flume on the same box (each with different agent configurations where there is no overlap of ports) ?
Thanks,
~Rahul.