You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Taylor Edmiston <te...@gmail.com> on 2018/08/28 20:18:33 UTC
Running unit tests against SLUGIFY_USES_TEXT_UNIDECODE and
AIRFLOW_GPL_UNIDECODE (also is this broken?)
Since the release of 1.10, we now have the option to install Airflow with
either:
1. python-nvd3 <https://github.com/areski/python-nvd3> --> python-slugify
<https://github.com/un33k/python-slugify> --> text-unidecode
<https://github.com/kmike/text-unidecode/> (via env var
SLUGIFY_USES_TEXT_UNIDECODE=yes), or
2. python-nvd3 --> python-slugify --> unidecode
<https://github.com/avian2/unidecode> (via AIRFLOW_GPL_UNIDECODE=yes)
Currently on Travis CI we only test the former configuration. Does anyone
have a recommendation on how to go about testing the latter? Running an
entire second copy of the unit tests for one dependency feels a bit
redundant... maybe there's a small relevant subset of tests that could only
run for the alternative dependency config?
On a related note, I think this part of the install may be broken.
I've tried running a pip install under each config like so (using pyenv +
virtualenvwrapper):
Shell 1:
pyenv shell 3.6.5
mktmpenv
export SLUGIFY_USES_TEXT_UNIDECODE=yes
pip install apache-airflow
pip freeze > ~/a.txt
Shell 2:
pyenv shell 3.6.5
mktmpenv
export AIRFLOW_GPL_UNIDECODE=yes
pip install apache-airflow
pip freeze > ~/b.txt
Shell 3:
diff ~/a.txt ~/b.txt
(empty)
I would expect the former to have text-unidecode and the latter to have
Unidecode. *Can someone else attempt to reproduce this behavior?*
Additionally, I'm also experiencing this same behavior when trying to
install the underlying python-slugify package similarly as well. I've
opened an issue for that here -
https://github.com/un33k/python-slugify/issues/59.
Thank you,
Taylor
*Taylor Edmiston*
Blog <https://blog.tedmiston.com/> | CV
<https://stackoverflow.com/cv/taylor> | LinkedIn
<https://www.linkedin.com/in/tedmiston/> | AngelList
<https://angel.co/taylor> | Stack Overflow
<https://stackoverflow.com/users/149428/taylor-edmiston>
Re: Running unit tests against SLUGIFY_USES_TEXT_UNIDECODE and
AIRFLOW_GPL_UNIDECODE (also is this broken?)
Posted by Ash Berlin-Taylor <as...@apache.org>.
I don't think we strictly care about running the tests in both these circumstances - it is a flag that control which dep is installed two or three levels down as you say, and the project has it's own tests.
I'd rather we spent time on replacing python-nvd with something that means we don't have to force our users to install it in a slightly odd way, and instead can just `pip install apache-airflow` again.
The reason this setting might not make any difference to what is installed comes down to Python Wheels - i.e. binary packages. When pip installs a wheel (which it does for preference as it is quicker) it doesn't run that project's setup.py anymore, so this block https://github.com/un33k/python-slugify/blob/master/setup.py#L18-L21 <https://github.com/un33k/python-slugify/blob/master/setup.py#L18-L21> isn't run.
The fix if you want to make it run this block still is `pip install --no-binary=python-slugify apache-airflow`
> On 28 Aug 2018, at 21:18, Taylor Edmiston <te...@gmail.com> wrote:
>
> Since the release of 1.10, we now have the option to install Airflow with
> either:
>
> 1. python-nvd3 <https://github.com/areski/python-nvd3> --> python-slugify
> <https://github.com/un33k/python-slugify> --> text-unidecode
> <https://github.com/kmike/text-unidecode/> (via env var
> SLUGIFY_USES_TEXT_UNIDECODE=yes), or
> 2. python-nvd3 --> python-slugify --> unidecode
> <https://github.com/avian2/unidecode> (via AIRFLOW_GPL_UNIDECODE=yes)
>
> Currently on Travis CI we only test the former configuration. Does anyone
> have a recommendation on how to go about testing the latter? Running an
> entire second copy of the unit tests for one dependency feels a bit
> redundant... maybe there's a small relevant subset of tests that could only
> run for the alternative dependency config?
>
> On a related note, I think this part of the install may be broken.
>
> I've tried running a pip install under each config like so (using pyenv +
> virtualenvwrapper):
>
> Shell 1:
>
> pyenv shell 3.6.5
> mktmpenv
> export SLUGIFY_USES_TEXT_UNIDECODE=yes
> pip install apache-airflow
> pip freeze > ~/a.txt
>
> Shell 2:
>
> pyenv shell 3.6.5
> mktmpenv
> export AIRFLOW_GPL_UNIDECODE=yes
> pip install apache-airflow
> pip freeze > ~/b.txt
>
> Shell 3:
>
> diff ~/a.txt ~/b.txt
> (empty)
>
> I would expect the former to have text-unidecode and the latter to have
> Unidecode. *Can someone else attempt to reproduce this behavior?*
>
> Additionally, I'm also experiencing this same behavior when trying to
> install the underlying python-slugify package similarly as well. I've
> opened an issue for that here -
> https://github.com/un33k/python-slugify/issues/59.
>
> Thank you,
> Taylor
>
> *Taylor Edmiston*
> Blog <https://blog.tedmiston.com/> | CV
> <https://stackoverflow.com/cv/taylor> | LinkedIn
> <https://www.linkedin.com/in/tedmiston/> | AngelList
> <https://angel.co/taylor> | Stack Overflow
> <https://stackoverflow.com/users/149428/taylor-edmiston>
Re: Running unit tests against SLUGIFY_USES_TEXT_UNIDECODE and AIRFLOW_GPL_UNIDECODE (also is this broken?)
Posted by Bolke de Bruin <bd...@gmail.com>.
The dependency is so small, besides python-slugify executes tests with both that I wouldn’t bother. Probably it makes more sense to get rid of python-nvd3 and to be able to upgrade d3.
Verstuurd vanaf mijn iPad
> Op 28 aug. 2018 om 22:18 heeft Taylor Edmiston <te...@gmail.com> het volgende geschreven:
>
> Since the release of 1.10, we now have the option to install Airflow with
> either:
>
> 1. python-nvd3 <https://github.com/areski/python-nvd3> --> python-slugify
> <https://github.com/un33k/python-slugify> --> text-unidecode
> <https://github.com/kmike/text-unidecode/> (via env var
> SLUGIFY_USES_TEXT_UNIDECODE=yes), or
> 2. python-nvd3 --> python-slugify --> unidecode
> <https://github.com/avian2/unidecode> (via AIRFLOW_GPL_UNIDECODE=yes)
>
> Currently on Travis CI we only test the former configuration. Does anyone
> have a recommendation on how to go about testing the latter? Running an
> entire second copy of the unit tests for one dependency feels a bit
> redundant... maybe there's a small relevant subset of tests that could only
> run for the alternative dependency config?
>
> On a related note, I think this part of the install may be broken.
>
> I've tried running a pip install under each config like so (using pyenv +
> virtualenvwrapper):
>
> Shell 1:
>
> pyenv shell 3.6.5
> mktmpenv
> export SLUGIFY_USES_TEXT_UNIDECODE=yes
> pip install apache-airflow
> pip freeze > ~/a.txt
>
> Shell 2:
>
> pyenv shell 3.6.5
> mktmpenv
> export AIRFLOW_GPL_UNIDECODE=yes
> pip install apache-airflow
> pip freeze > ~/b.txt
>
> Shell 3:
>
> diff ~/a.txt ~/b.txt
> (empty)
>
> I would expect the former to have text-unidecode and the latter to have
> Unidecode. *Can someone else attempt to reproduce this behavior?*
>
> Additionally, I'm also experiencing this same behavior when trying to
> install the underlying python-slugify package similarly as well. I've
> opened an issue for that here -
> https://github.com/un33k/python-slugify/issues/59.
>
> Thank you,
> Taylor
>
> *Taylor Edmiston*
> Blog <https://blog.tedmiston.com/> | CV
> <https://stackoverflow.com/cv/taylor> | LinkedIn
> <https://www.linkedin.com/in/tedmiston/> | AngelList
> <https://angel.co/taylor> | Stack Overflow
> <https://stackoverflow.com/users/149428/taylor-edmiston>