You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Jeremy Lewi (JIRA)" <ji...@apache.org> on 2011/07/01 17:19:28 UTC
[jira] [Created] (AVRO-849) Python unittests don't work if avro is
already installed
Python unittests don't work if avro is already installed
--------------------------------------------------------
Key: AVRO-849
URL: https://issues.apache.org/jira/browse/AVRO-849
Project: Avro
Issue Type: Bug
Components: python
Affects Versions: 1.4.1
Reporter: Jeremy Lewi
Assignee: Jeremy Lewi
Priority: Minor
I think there is a problem running the python unittests if avro is already installed. If avro is already installed then when we import avro or its subpackages in the unittests we can end up importing the installed avro instead of the avro version being compiled because the installed version might be higher on the path.
One (slightly cumbersome way to fix this) would be to pass the the paths listed in test.path to our unittests (e.g by using an environment variable). The unittests could then adjust the path so these directories were highest on the path. The downside of this solution would be every unittest would need some boilerplate code to adjust the path.
Does anyone have suggestions about how to pass the values in test.path to our unittests? I don't know anything about ant, so any help would be appreciated.
Or does anyone have suggestions about a better solution which doesn't require adjusting the path in each unittest?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-849) Python unittests don't work if avro
is already installed
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087808#comment-13087808 ]
Doug Cutting commented on AVRO-849:
-----------------------------------
I'm confused. The values of test.path (lang/py/{src,test,lib}) become the value of PYTHONPATH, which should supercede anything installed, no?
> Python unittests don't work if avro is already installed
> --------------------------------------------------------
>
> Key: AVRO-849
> URL: https://issues.apache.org/jira/browse/AVRO-849
> Project: Avro
> Issue Type: Bug
> Components: python
> Affects Versions: 1.4.1
> Reporter: Jeremy Lewi
> Assignee: Jeremy Lewi
> Priority: Minor
> Attachments: AVRO-849.patch
>
>
> I think there is a problem running the python unittests if avro is already installed. If avro is already installed then when we import avro or its subpackages in the unittests we can end up importing the installed avro instead of the avro version being compiled because the installed version might be higher on the path.
> One (slightly cumbersome way to fix this) would be to pass the the paths listed in test.path to our unittests (e.g by using an environment variable). The unittests could then adjust the path so these directories were highest on the path. The downside of this solution would be every unittest would need some boilerplate code to adjust the path.
> Does anyone have suggestions about how to pass the values in test.path to our unittests? I don't know anything about ant, so any help would be appreciated.
> Or does anyone have suggestions about a better solution which doesn't require adjusting the path in each unittest?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-849) Python unittests don't work if avro is
already installed
Posted by "Jeremy Lewi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jeremy Lewi updated AVRO-849:
-----------------------------
Status: Patch Available (was: Open)
Here is a patch to address this issue. At the start of each unittest you add the code
{noformat}
import avro_fix_849
{noformat}
This causes the path to be adjusted based on the location of the unittest
> Python unittests don't work if avro is already installed
> --------------------------------------------------------
>
> Key: AVRO-849
> URL: https://issues.apache.org/jira/browse/AVRO-849
> Project: Avro
> Issue Type: Bug
> Components: python
> Affects Versions: 1.4.1
> Reporter: Jeremy Lewi
> Assignee: Jeremy Lewi
> Priority: Minor
>
> I think there is a problem running the python unittests if avro is already installed. If avro is already installed then when we import avro or its subpackages in the unittests we can end up importing the installed avro instead of the avro version being compiled because the installed version might be higher on the path.
> One (slightly cumbersome way to fix this) would be to pass the the paths listed in test.path to our unittests (e.g by using an environment variable). The unittests could then adjust the path so these directories were highest on the path. The downside of this solution would be every unittest would need some boilerplate code to adjust the path.
> Does anyone have suggestions about how to pass the values in test.path to our unittests? I don't know anything about ant, so any help would be appreciated.
> Or does anyone have suggestions about a better solution which doesn't require adjusting the path in each unittest?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-849) Python unittests don't work if avro is
already installed
Posted by "Jeremy Lewi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jeremy Lewi updated AVRO-849:
-----------------------------
Attachment: AVRO-849.patch
Here is a patch.
I created a module avro_849_fix.py which adjusts the path based on the location of the module. SO unittests just need to add the following at the top
{noformat}
import avro_849_fix
{noformat}
> Python unittests don't work if avro is already installed
> --------------------------------------------------------
>
> Key: AVRO-849
> URL: https://issues.apache.org/jira/browse/AVRO-849
> Project: Avro
> Issue Type: Bug
> Components: python
> Affects Versions: 1.4.1
> Reporter: Jeremy Lewi
> Assignee: Jeremy Lewi
> Priority: Minor
> Attachments: AVRO-849.patch
>
>
> I think there is a problem running the python unittests if avro is already installed. If avro is already installed then when we import avro or its subpackages in the unittests we can end up importing the installed avro instead of the avro version being compiled because the installed version might be higher on the path.
> One (slightly cumbersome way to fix this) would be to pass the the paths listed in test.path to our unittests (e.g by using an environment variable). The unittests could then adjust the path so these directories were highest on the path. The downside of this solution would be every unittest would need some boilerplate code to adjust the path.
> Does anyone have suggestions about how to pass the values in test.path to our unittests? I don't know anything about ant, so any help would be appreciated.
> Or does anyone have suggestions about a better solution which doesn't require adjusting the path in each unittest?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-849) Python unittests don't work if avro is
already installed
Posted by "Jeremy Lewi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jeremy Lewi updated AVRO-849:
-----------------------------
Status: Open (was: Patch Available)
Sorry I thought submit patch was the way to attach a patch.
> Python unittests don't work if avro is already installed
> --------------------------------------------------------
>
> Key: AVRO-849
> URL: https://issues.apache.org/jira/browse/AVRO-849
> Project: Avro
> Issue Type: Bug
> Components: python
> Affects Versions: 1.4.1
> Reporter: Jeremy Lewi
> Assignee: Jeremy Lewi
> Priority: Minor
>
> I think there is a problem running the python unittests if avro is already installed. If avro is already installed then when we import avro or its subpackages in the unittests we can end up importing the installed avro instead of the avro version being compiled because the installed version might be higher on the path.
> One (slightly cumbersome way to fix this) would be to pass the the paths listed in test.path to our unittests (e.g by using an environment variable). The unittests could then adjust the path so these directories were highest on the path. The downside of this solution would be every unittest would need some boilerplate code to adjust the path.
> Does anyone have suggestions about how to pass the values in test.path to our unittests? I don't know anything about ant, so any help would be appreciated.
> Or does anyone have suggestions about a better solution which doesn't require adjusting the path in each unittest?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-849) Python unittests don't work if avro
is already installed
Posted by "Jeremy Lewi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087872#comment-13087872 ]
Jeremy Lewi commented on AVRO-849:
----------------------------------
I'm not sure exactly how the path precedence is determined from PYTHONPATH but on my system it looks like "eggs" appear higher on the path then directories obtained from PYTHONPATH.
A simple test
{code}
export PYTHONPATH=/some/dir
python -c 'import sys; print "\n".join(sys.path)'
{code}
In my case the output is
{code}
/usr/lib/python2.7/site-packages/ipython-0.11-py2.7.egg
/usr/lib/python2.7/site-packages/avro-1.6.0_SNAPSHOT-py2.7.egg
/some/dir
/usr/lib64/python27.zip
/usr/lib64/python2.7
...
{code}
So at least in my case, the installed avro egg will take precedence over (lang/py/{src,test,lib}) directories added via PYTHONPATH.
> Python unittests don't work if avro is already installed
> --------------------------------------------------------
>
> Key: AVRO-849
> URL: https://issues.apache.org/jira/browse/AVRO-849
> Project: Avro
> Issue Type: Bug
> Components: python
> Affects Versions: 1.4.1
> Reporter: Jeremy Lewi
> Assignee: Jeremy Lewi
> Priority: Minor
> Attachments: AVRO-849.patch
>
>
> I think there is a problem running the python unittests if avro is already installed. If avro is already installed then when we import avro or its subpackages in the unittests we can end up importing the installed avro instead of the avro version being compiled because the installed version might be higher on the path.
> One (slightly cumbersome way to fix this) would be to pass the the paths listed in test.path to our unittests (e.g by using an environment variable). The unittests could then adjust the path so these directories were highest on the path. The downside of this solution would be every unittest would need some boilerplate code to adjust the path.
> Does anyone have suggestions about how to pass the values in test.path to our unittests? I don't know anything about ant, so any help would be appreciated.
> Or does anyone have suggestions about a better solution which doesn't require adjusting the path in each unittest?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-849) Python unittests don't work if avro
is already installed
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089704#comment-13089704 ]
Doug Cutting commented on AVRO-849:
-----------------------------------
This seems to be a well-known Python problem with no easy solution.
http://www.velocityreviews.com/forums/t716589-pythonpath-and-eggs.html
http://stackoverflow.com/questions/897792/pythons-sys-path-value
I'd be okay with the patch here if we change it to instead of 'avro_849_fix' to be something descriptive like 'avro-test-path' or somesuch.
> Python unittests don't work if avro is already installed
> --------------------------------------------------------
>
> Key: AVRO-849
> URL: https://issues.apache.org/jira/browse/AVRO-849
> Project: Avro
> Issue Type: Bug
> Components: python
> Affects Versions: 1.4.1
> Reporter: Jeremy Lewi
> Assignee: Jeremy Lewi
> Priority: Minor
> Attachments: AVRO-849.patch
>
>
> I think there is a problem running the python unittests if avro is already installed. If avro is already installed then when we import avro or its subpackages in the unittests we can end up importing the installed avro instead of the avro version being compiled because the installed version might be higher on the path.
> One (slightly cumbersome way to fix this) would be to pass the the paths listed in test.path to our unittests (e.g by using an environment variable). The unittests could then adjust the path so these directories were highest on the path. The downside of this solution would be every unittest would need some boilerplate code to adjust the path.
> Does anyone have suggestions about how to pass the values in test.path to our unittests? I don't know anything about ant, so any help would be appreciated.
> Or does anyone have suggestions about a better solution which doesn't require adjusting the path in each unittest?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-849) Python unittests don't work if avro is
already installed
Posted by "Jeremy Lewi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jeremy Lewi updated AVRO-849:
-----------------------------
Attachment: AVRO-849.patch
I renamed the module to "set_avro_test_path.py" to avoid using the JIRA issue number. I also removed all references to the JIRA # in the comments.
> Python unittests don't work if avro is already installed
> --------------------------------------------------------
>
> Key: AVRO-849
> URL: https://issues.apache.org/jira/browse/AVRO-849
> Project: Avro
> Issue Type: Bug
> Components: python
> Affects Versions: 1.4.1
> Reporter: Jeremy Lewi
> Assignee: Jeremy Lewi
> Priority: Minor
> Attachments: AVRO-849.patch, AVRO-849.patch
>
>
> I think there is a problem running the python unittests if avro is already installed. If avro is already installed then when we import avro or its subpackages in the unittests we can end up importing the installed avro instead of the avro version being compiled because the installed version might be higher on the path.
> One (slightly cumbersome way to fix this) would be to pass the the paths listed in test.path to our unittests (e.g by using an environment variable). The unittests could then adjust the path so these directories were highest on the path. The downside of this solution would be every unittest would need some boilerplate code to adjust the path.
> Does anyone have suggestions about how to pass the values in test.path to our unittests? I don't know anything about ant, so any help would be appreciated.
> Or does anyone have suggestions about a better solution which doesn't require adjusting the path in each unittest?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-849) Python unittests don't work if avro is
already installed
Posted by "Jeremy Lewi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jeremy Lewi updated AVRO-849:
-----------------------------
Status: Patch Available (was: Open)
> Python unittests don't work if avro is already installed
> --------------------------------------------------------
>
> Key: AVRO-849
> URL: https://issues.apache.org/jira/browse/AVRO-849
> Project: Avro
> Issue Type: Bug
> Components: python
> Affects Versions: 1.4.1
> Reporter: Jeremy Lewi
> Assignee: Jeremy Lewi
> Priority: Minor
> Attachments: AVRO-849.patch, AVRO-849.patch
>
>
> I think there is a problem running the python unittests if avro is already installed. If avro is already installed then when we import avro or its subpackages in the unittests we can end up importing the installed avro instead of the avro version being compiled because the installed version might be higher on the path.
> One (slightly cumbersome way to fix this) would be to pass the the paths listed in test.path to our unittests (e.g by using an environment variable). The unittests could then adjust the path so these directories were highest on the path. The downside of this solution would be every unittest would need some boilerplate code to adjust the path.
> Does anyone have suggestions about how to pass the values in test.path to our unittests? I don't know anything about ant, so any help would be appreciated.
> Or does anyone have suggestions about a better solution which doesn't require adjusting the path in each unittest?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira