You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Bolke de Bruin <bd...@gmail.com> on 2018/01/26 08:41:41 UTC
Re: Installation of Airflow
Installation under a different user works fine in a virtualenv. You can also install system wide and run as normal user (that is how we run it currently).
Bolke
Verstuurd vanaf mijn iPad
> Op 25 jan. 2018 om 23:21 heeft Pullagura, Kalyan <kp...@paypal.com.INVALID> het volgende geschreven:
>
> Hi
> Did you guys ever figure out the following or your suggestions would be much appreciated.
>
>
> 1. Installing Airflow application without ‘root’ user. If using root user, not to use system libraries for all the required python modules instead redirecting to a different location.
> 2. Installing Airflow using a local user or having an Airflow package itself with all the installation and all I have to run is a rpm or some install file.
> The above should help us to get faster when installing workers across our environment.
>
> Regards,
> Kalyan.
>
>
Re: Installation of Airflow
Posted by Ash Berlin-Taylor <as...@firemirror.com>.
If you want to use system installed modules (such as os-packaged py libs) the other thing you can do is `pip install --user apache-airflow` which will install into ~/.local/bin and ~/.local/lib etc. This is automatically in the python search path ahead of the system paths for the user
(This is what we do)
-ash
> On 26 Jan 2018, at 08:41, Bolke de Bruin <bd...@gmail.com> wrote:
>
> Installation under a different user works fine in a virtualenv. You can also install system wide and run as normal user (that is how we run it currently).
>
> Bolke
>
> Verstuurd vanaf mijn iPad
>
>> Op 25 jan. 2018 om 23:21 heeft Pullagura, Kalyan <kp...@paypal.com.INVALID> het volgende geschreven:
>>
>> Hi
>> Did you guys ever figure out the following or your suggestions would be much appreciated.
>>
>>
>> 1. Installing Airflow application without ‘root’ user. If using root user, not to use system libraries for all the required python modules instead redirecting to a different location.
>> 2. Installing Airflow using a local user or having an Airflow package itself with all the installation and all I have to run is a rpm or some install file.
>> The above should help us to get faster when installing workers across our environment.
>>
>> Regards,
>> Kalyan.
>>
>>
Re: Installation of Airflow
Posted by Dennis O'Brien <de...@dennisobrien.net>.
Hi Kalyan,
We use conda for managing the python environment. Everything including
airflow is installed in this conda environment (via conda when available,
or pip otherwise). Then the path to that conda env is added to the PATH
environment variable. The same approach should work if you are using
virtualenv.
cheers,
Dennis
On Fri, Jan 26, 2018 at 7:57 AM Pullagura, Kalyan
<kp...@paypal.com.invalid> wrote:
> Hi Bolke,
> Thanks for the comments, yes we do the same currently. We install system
> wide and run the app. as local user. The main problem I am having in this
> model is that when I install the app. system wide, the python modules are
> having dependency with system libraries and I don’t want to
> upgrade/downgrade the system libraries because a python module might not
> like the version of system library we have on the system. I am lookin for
> ways to workaround this.
>
> Regards,
> Kalyan.
>
> On Jan 26, 2018, at 1:41 AM, Bolke de Bruin <bdbruin@gmail.com<mailto:
> bdbruin@gmail.com>> wrote:
>
> Installation under a different user works fine in a virtualenv. You can
> also install system wide and run as normal user (that is how we run it
> currently).
>
> Bolke
>
> Verstuurd vanaf mijn iPad
>
> Op 25 jan. 2018 om 23:21 heeft Pullagura, Kalyan
> <kp...@paypal.com.INVALID>> het
> volgende geschreven:
>
> Hi
> Did you guys ever figure out the following or your suggestions would be
> much appreciated.
>
>
> 1. Installing Airflow application without ‘root’ user. If using root
> user, not to use system libraries for all the required python modules
> instead redirecting to a different location.
> 2. Installing Airflow using a local user or having an Airflow package
> itself with all the installation and all I have to run is a rpm or some
> install file.
> The above should help us to get faster when installing workers across our
> environment.
>
> Regards,
> Kalyan.
>
>
>
Re: Installation of Airflow
Posted by "Pullagura, Kalyan" <kp...@paypal.com.INVALID>.
Hi Bolke,
Thanks for the comments, yes we do the same currently. We install system wide and run the app. as local user. The main problem I am having in this model is that when I install the app. system wide, the python modules are having dependency with system libraries and I don’t want to upgrade/downgrade the system libraries because a python module might not like the version of system library we have on the system. I am lookin for ways to workaround this.
Regards,
Kalyan.
On Jan 26, 2018, at 1:41 AM, Bolke de Bruin <bd...@gmail.com>> wrote:
Installation under a different user works fine in a virtualenv. You can also install system wide and run as normal user (that is how we run it currently).
Bolke
Verstuurd vanaf mijn iPad
Op 25 jan. 2018 om 23:21 heeft Pullagura, Kalyan <kp...@paypal.com.INVALID>> het volgende geschreven:
Hi
Did you guys ever figure out the following or your suggestions would be much appreciated.
1. Installing Airflow application without ‘root’ user. If using root user, not to use system libraries for all the required python modules instead redirecting to a different location.
2. Installing Airflow using a local user or having an Airflow package itself with all the installation and all I have to run is a rpm or some install file.
The above should help us to get faster when installing workers across our environment.
Regards,
Kalyan.