You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mod_python-dev@quetz.apache.org by "Daniel J. Popowich" <dp...@astro.umass.edu> on 2007/05/17 19:19:23 UTC
sys.path and htaccess change?
In mod_python 3.2.x, if I set a directory to use mod_python with a
.htaccess file, I will find that directory in sys.path. That is, if:
/var/www/somedir/.htaccess
contains:
SetHandler python-program
then sys.path will have "/var/www/somedir" as element zero.
With 3.3.x this does not happen.
The problem? I have a handler with co-located python modules that get
imported (the handler does "import somemod" and somemomd is in the
same directory as the .htaccess file). This works fine with 3.2.x,
but when I upgraded to 3.3.1 I get import errors.
I searched the changelogs in the documentation, but I see no mention
of this change.
Thanks,
Daniel Popowich
---------------
http://www.astro.umass.edu/~dpopowich/python/mpservlets/
Re: sys.path and htaccess change?
Posted by Graham Dumpleton <gr...@gmail.com>.
I worked it out. As my change didn't disable the mpservlets caching
system, it was never triggering the module importer again to perform
its own checks. Thus, if I add:
klass, cached_mtime, servlets = self.class_cache.get(fname,
(None,None, []))
# XXX Pretend we didn't find it so mod_python importer used
klass, cached_mtime, servlets = (None,None, [])
# XXX
if not klass or mtime > cached_mtime:
In other words, a bit more work is required in mpservlets as with the
new module importer as it shouldn't really be supplying its own
caching system as that then defeats the mod_python importers own
caching and reloading system.
BTW, if you have program that can display DOT graphs, the graph data
was produced using:
PythonLogHandler ./_loghandler.py
and the following as the log handler.
from mod_python import apache
def loghandler(req):
try:
output = file('/tmp/request.dot', 'w')
output.write(apache.modules_graph(1))
finally:
output.close()
return apache.OK
Seeing the relationships between your modules and the data presented
there can be helpful sometimes in seeing what is happening and working
out how the module importer works.
Graham
On 18/05/07, Graham Dumpleton <gr...@gmail.com> wrote:
> On 18/05/07, Graham Dumpleton <gr...@gmail.com> wrote:
> > Just change this in your mpservlets package. The new importer can
> > handle arbitrary extensions so can load .mps files no problem. With
> > just this change your tutorial example works with no problems and it
> > uses 'import' to get stuff from handler directory.
> >
> > kundalini:/usr/local/src/mpservlets-1.1.6 grahamd$ diff
> > servlet.py.dist servlet.py
> > 1269,1270c1269,1271
> > < code = {}
> > < execfile(fname, code)
> > ---
> > > #code = {}
> > > #execfile(fname, code)
> > > module = apache.import_module(fname)
> > 1276c1277,1278
> > < klass = code[basename]
> > ---
> > > #klass = code[basename]
> > > klass = getattr(module, basename)
>
> Hmmm, although web pages are being displayed okay, there is something
> strange about the scenario in which apache.import_module() is being
> used which is meaning that depth searches for changes to child
> resulting in reloading from root down isn't working. Thus, if changing
> a module which isn't a root module, then I am still needing to restart
> Apache which I shouldn't have to with the new module importer. :-(
>
> For example, if I change _SitePage.py in attached graph, then the two
> .mps files and _SitePage.py should be reloaded on next request to
> either of the .mps files, but it isn't. Looks like I'll have to track
> down why as must be missing something very subtle, or obvious, which
> is causing the module importer not to behave as expected.
>
> Graham
>
>
Re: sys.path and htaccess change?
Posted by Graham Dumpleton <gr...@gmail.com>.
On 18/05/07, Graham Dumpleton <gr...@gmail.com> wrote:
> Just change this in your mpservlets package. The new importer can
> handle arbitrary extensions so can load .mps files no problem. With
> just this change your tutorial example works with no problems and it
> uses 'import' to get stuff from handler directory.
>
> kundalini:/usr/local/src/mpservlets-1.1.6 grahamd$ diff
> servlet.py.dist servlet.py
> 1269,1270c1269,1271
> < code = {}
> < execfile(fname, code)
> ---
> > #code = {}
> > #execfile(fname, code)
> > module = apache.import_module(fname)
> 1276c1277,1278
> < klass = code[basename]
> ---
> > #klass = code[basename]
> > klass = getattr(module, basename)
Hmmm, although web pages are being displayed okay, there is something
strange about the scenario in which apache.import_module() is being
used which is meaning that depth searches for changes to child
resulting in reloading from root down isn't working. Thus, if changing
a module which isn't a root module, then I am still needing to restart
Apache which I shouldn't have to with the new module importer. :-(
For example, if I change _SitePage.py in attached graph, then the two
.mps files and _SitePage.py should be reloaded on next request to
either of the .mps files, but it isn't. Looks like I'll have to track
down why as must be missing something very subtle, or obvious, which
is causing the module importer not to behave as expected.
Graham
Re: sys.path and htaccess change?
Posted by "Daniel J. Popowich" <dp...@comcast.net>.
Jorey Bump writes:
> Graham Dumpleton wrote:
>
> > The whole point of the changes which were made was to draw a well
> > defined line between the code modules used in the web application and
> > which reside in the document tree, or other specially specified areas
> > by way of mod_python module importer path, and the standard Python
> > sys.path locations.
>
> I think it's important to emphasize this point.
I don't have a problem with making a distinction, either. But I do
have a problem with behaviour like this changing (handler directory no
longer in sys.path) between minor upgrades. I probably have users
downloading mpservlets, launching the tutorial, saying to themselves
%#&@% piece of crap and moving on.
> > That there was no distinction before is what
> > caused all the issues with same named modules, loading of incorrect
> > modules, reloading of modules which shouldn't be reloaded etc etc.
>
> And it's also important to point out that discussion of these problems
> has become virtually nonexistent, whereas they used to dominate the list.
It was never clear to me that the problem was mod_python, but rather
publisher. The novices coming to mod_python would go to the tutorial
which pointed them to publisher and so in the mind of the novice:
mod_python == publisher is True. As they used publisher and pushed
its boundaries, importing issues became a major problem. I think it
worthy to note that the swarm of importing questions rarely (ever?)
involved psp or mpservlets, which had their own importers.
> > I remember when I first brought up this idea of 'import' working in
> > magic ways so things would just work as one expected, that there were
> > at least one person who thought it was a silly idea.
>
> That could well have been me. I viewed the issues as limitations, not
> necessarily weaknesses, and knew what to expect once I refined my
> approach. I was worried that changing the module importer could
> introduce more issues and force users to include proprietary mod_python
> code in their modules (excluding handlers, which are a special case).
I don't have a problem with mod_python tweaking import. It's well
known by python programmers (well, those who've been around awhile)
that this is a tool at their disposal. I would just like to see it
more **boldly** stated that it does. This knowledge may not make a
difference to those who are using mod_python indirectly (those coming
to it to use publisher, psp, mpservlets, vampire, etc.), but to those
who write and maintain these handlers, it's HUGE.
> Since I keep nearly all of my code outside of the document tree, I'm not
> necessarily the target beneficiary of the changes made to the module
> importer. But it has eliminated namespace collisions and reloading
> issues in my frontends, so I benefit from the improvements and agree
> that things are much better. 3.3 is easily my favorite version of
> mod_python.
I have found it a very powerful tool to be able to co-locate mps files
with regular python modules. In fact, I find it odd to separate them:
why put part of my applications code in one place and install the rest
in another?
Mpservlets does not allow files beginning with an underscore to be
requested (generates a 403 response), so it's great to be able to
place modules (eg, named _utilities.py) in the same directory as the
front-end mps files. The developer doesn't have to worry about
installing pieces of their package in site-packages or some such
place, so packaging is much easier. A prime example is the mpservlets
tutorial: everything is in one directory, very self-contained and easy
to install.
> > All I can suggest is you try it by starting with the changes to
> > mpservlets as I described and then simply see if your existing
> > applications which use mpservlets still work. Since the release we
> > have only had a few people even be slightly affected by the
> > differences, plus all the complaints we used to get about problems
> > with the old importer have pretty well vanished.
>
> I'm lucky enough that none of my applications are publicly
> distributed...
Yes, you are. :-)
> Daniel's in a rough spot, though, because he has to support all of the
> versions of mod_python that are out there (within reason). While we may
> find it trivial, the average user is unlikely to upgrade from a version
> included in the platform distribution (which could be dangerous if other
> packages were tweaked to rely on the state of mod_python when it was
> packaged). It sounds like he will still need to include his old caching
> system and disable it when it's not needed. Hopefully, this won't be too
> hard or require a fork.
I don't think there's really a problem with my importer co-existing
with 3.3 (although it would make sense for me to piggyback off
mod_python's, just like I do for sessions, cookies, etc., so I *do*
want to use the 3.3 importer at some point). I just need to figure
out how to fix the python module/mps file co-location problem in an
elegant way.
Dan
Re: sys.path and htaccess change?
Posted by Jorey Bump <li...@joreybump.com>.
Graham Dumpleton wrote:
> The whole point of the changes which were made was to draw a well
> defined line between the code modules used in the web application and
> which reside in the document tree, or other specially specified areas
> by way of mod_python module importer path, and the standard Python
> sys.path locations.
I think it's important to emphasize this point.
> That there was no distinction before is what
> caused all the issues with same named modules, loading of incorrect
> modules, reloading of modules which shouldn't be reloaded etc etc.
And it's also important to point out that discussion of these problems
has become virtually nonexistent, whereas they used to dominate the list.
> I remember when I first brought up this idea of 'import' working in
> magic ways so things would just work as one expected, that there were
> at least one person who thought it was a silly idea.
That could well have been me. I viewed the issues as limitations, not
necessarily weaknesses, and knew what to expect once I refined my
approach. I was worried that changing the module importer could
introduce more issues and force users to include proprietary mod_python
code in their modules (excluding handlers, which are a special case).
> In the end I had
> to give up on even trying to explain it because it would have taken me
> months just to get people to understand what I had in mind. Instead, I
> just wrote the code and I believe most would have to say the result in
> certainly better than what was there before and that code now just
> works as how one would expect it to work when one uses 'import'
> whereas it didn't always before.
Since I keep nearly all of my code outside of the document tree, I'm not
necessarily the target beneficiary of the changes made to the module
importer. But it has eliminated namespace collisions and reloading
issues in my frontends, so I benefit from the improvements and agree
that things are much better. 3.3 is easily my favorite version of
mod_python.
> All I can suggest is you try it by starting with the changes to
> mpservlets as I described and then simply see if your existing
> applications which use mpservlets still work. Since the release we
> have only had a few people even be slightly affected by the
> differences, plus all the complaints we used to get about problems
> with the old importer have pretty well vanished.
I'm lucky enough that none of my applications are publicly distributed,
so I was able to normalize all of my sytems and patch the code where
needed. If you're patching production code, I recommend restarting
apache after your changes (in spite of the new reloading behaviour) and
tailing the apache error log for several hours.
Daniel's in a rough spot, though, because he has to support all of the
versions of mod_python that are out there (within reason). While we may
find it trivial, the average user is unlikely to upgrade from a version
included in the platform distribution (which could be dangerous if other
packages were tweaked to rely on the state of mod_python when it was
packaged). It sounds like he will still need to include his old caching
system and disable it when it's not needed. Hopefully, this won't be too
hard or require a fork.
Re: sys.path and htaccess change?
Posted by "Daniel J. Popowich" <dp...@comcast.net>.
Graham Dumpleton writes:
> Correct, you were only changing the path related to modules managed by
> the new module importer and not those handled by the standard Python
> importer. Adding "~" like that would also only have been needed if in
> a subdirectory of the handler root directory an import was trying to
> be done with either 'import' or 'apache.import_module()' for a module
> in the handler root directory.
>
> I had assumed in suggesting that, that you had already converted
> mpservlets to use the new module importer and were triggering that
> specific case.
Graham,
Thanks for your detailed answers (as always). Very helpful. However,
something I said in my original email is, I think, clouding the issue,
at least for me as I read your responses (or perhaps I'm just not
being very clear in my questions): I had said in passing I was picking
up mod_python again and thought I'd rewrite my mpservlets importer to
use the new 3.3 apache.import_module() function. **I actually never
got that far.** I upgraded my apache to use mp3.3 and the first thing
I did was run my tutorial and what I got was import errors. So, to
sum up:
o I have never used apache.import_module(), old or new. I only use
the import builtin.
o I have NOT started rewriting mpservlets to use the new importer.
In fact, I have NOT touched any old code at all: just upgrade to
3.3 and run what I've always run.
o My configuration uses .htaccess with:
SetHandler mod_python
PythonHandler mod_python.servlet
PythonDebug on
o What I have is a servlet (mps file) that gets found just fine with
my importer, but when execfile is run on the file I get import
errors on the **imports INSIDE the mps file** because sys.path
does not contain the handler directory like it used to.
I have confirmed that adding "PythonOption mod_python.legacy.importer *"
"fixes" the problem.
So, if I'm not from ever-on going to use legacy switches, what in your
opinion is the solution? Are you suggesting that if I use the 3.3
importer to load my mps files then the imports inside the mps files
will automagically work?
Thanks,
Dan
PS
It should have been named 4.0. Having to turn on legacy switches or
rewrite code to handle changes in functionality screams: This Is Not A
Minor Upgrade.
PPS
For completeness here's the exception I get when I upgraded to 3.3 and
the first thing I did was load the homepage to my mps tutorial:
----------------------------------------------------------------------
MOD_PYTHON ERROR
ProcessId: 7363
Interpreter: 'io.astro.umass.edu'
ServerName: 'io.astro.umass.edu'
DocumentRoot: '/var/www/'
URI: '/~dpopowich/mpstutorial/index'
Location: None
Directory: '/home/dpopowich/public_html/mpstutorial/'
Filename: '/home/dpopowich/public_html/mpstutorial/index'
PathInfo: ''
Phase: 'PythonHandler'
Handler: 'mod_python.servlet'
Traceback (most recent call last):
File "/usr/lib/python2.4/site-packages/mod_python/importer.py", line 1537, in HandlerDispatch
default=default_handler, arg=req, silent=hlist.silent)
File "/usr/lib/python2.4/site-packages/mod_python/importer.py", line 1229, in _process_target
result = _execute_target(config, req, object, arg)
File "/usr/lib/python2.4/site-packages/mod_python/importer.py", line 1128, in _execute_target
result = object(arg)
File "/usr/lib/python2.4/site-packages/mod_python/servlet.py", line 1456, in handler
servlet = _sm.get_servlet(filename)
File "/usr/lib/python2.4/site-packages/mod_python/servlet.py", line 1353, in get_servlet
klass, servlets = self.__load_servlet(filename)
File "/usr/lib/python2.4/site-packages/mod_python/servlet.py", line 1270, in __load_servlet
execfile(fname, code)
File "/home/dpopowich/public_html/mpstutorial/index.mps", line 3, in ?
from _TutorialBase import *
ImportError: No module named _TutorialBase
----------------------------------------------------------------------
Re: sys.path and htaccess change?
Posted by Graham Dumpleton <gr...@gmail.com>.
On 18/05/07, Daniel J. Popowich <dp...@comcast.net> wrote:
>
> Graham Dumpleton writes:
> > And that is what the documentation I pointed you at states:
> >
> > """Note that with the new module importer, as directories associated
> > with Python*Handler directives are no longer being added automatically
> > to sys.path and they are instead used directly by the module importer
> > only when required, some existing code which expected to be able to
> > import modules in the handler root directory from a module in a
> > subdirectory may no longer work. In these situations it will be
> > necessary to set the mod_python module importer path to include '~' or
> > list '~' in the __mp_path__ attribute of the module performing the
> > import."""
>
> OK, I think it's slowly sinking in...when I put the ~ in, it didn't
> affect sys.path, so my import still wasn't going to work. Right?
Correct, you were only changing the path related to modules managed by
the new module importer and not those handled by the standard Python
importer. Adding "~" like that would also only have been needed if in
a subdirectory of the handler root directory an import was trying to
be done with either 'import' or 'apache.import_module()' for a module
in the handler root directory.
I had assumed in suggesting that, that you had already converted
mpservlets to use the new module importer and were triggering that
specific case.
> So, how the heck do I import the modules?
In just about all cases exactly how you do it now. Although, where you
were using your own module importing and caching system for working
under mod_python, you would need to adapt it to use the new importer.
> Do I have to use apache.import_module() now?
No. In fact there are now cases where before you had to use
apache.import_module() because of bugs but you now can use just
'import' and it will also work, thus allowing one to create sets of
modules that can also be used safely outside of mod_python in a test
framework. That you have an existing module importing and caching
system of your own is a special case but only the module importer code
in moservlets should need to change, not anything else.
> But I don't want to!
As far as using it in mpservlets itself, that would be a pity really
as one of the main reasons that support for importing files by full
path with an arbitrary extension was added was so that mpservlets
could make use of the new importer rather than use its own, and thus
benefit from some of the features the new module importer provided
such as root modules being imported automatically when a child was
changed thereby avoiding the need to restart Apache after anything but
simple changes. Thus, in the tutorial example, I could adjust
_SitePage and thereby notionally change the appearance of the whole
site without a restart, something that couldn't be avoided with how
mpservlets worked with previous versions of mod_python.
> > If one wants to be precise, the addition of the handler directory to
> > sys.path was actually done by the top level mod_python dispatch code
> > rather than the module importer itself.
>
> And this I don't understand at all. If this "feature" was in the old
> system, why would reimplementing an api function change this behaviour
> in the dispatch code? I would consider this a bug.
That it did add the handler directory to sys.path previously is what
caused a lot of the bugs documented in the page I referred to in the
first place. There was no choice but to change it else you couldn't
fix the existing bugs. You may have avoided the bugs by using your own
module importer, but the bugs still affected everyone else.
The whole point of the changes which were made was to draw a well
defined line between the code modules used in the web application and
which reside in the document tree, or other specially specified areas
by way of mod_python module importer path, and the standard Python
sys.path locations. That there was no distinction before is what
caused all the issues with same named modules, loading of incorrect
modules, reloading of modules which shouldn't be reloaded etc etc.
> I would also say
> this is WAY too much functionality change for a minor release update.
I did suggest that the release be called 4.0, but since there was a
backwards compatibility switch so people could reenable old behaviour
for module importer while they update their code the consensus was
that 3.3 was okay. Also, the whole thing was intended to be as
compatible as possible. In the end there was only a few cases where
people would actually have to make changes, the main one being where
people were relying on handler root direct being in sys.path so
imports from non mod_python managed modules could find a module in
that directory.
> > The other important bit is:
> >
> > """As a new feature in mod_python 3.3, when using the standard Python
> > 'import' statement to import a module, if the import is being done
> > from a module which was previously imported by the mod_python module
> > importer, it is equivalent to having called apache.import_module()
> > directly."""
> >
> > It is because of this that the directory doesn't need to be in
> > sys.path, as under the covers, when 'import' is being used by modules
> > imported by the mod_python module importer directly in some way, then
> > 'import' is equivalent to having used 'apache.import_module()' and
> > therefore it looks in places related to mod_python including stuff in
> > the separate mod_python importer path.
>
> And I would call this a bug, too. Why should using mod_python change
> the behaviour of the import builtin? What if I'm including a package
> that includes a package that includes a package? I'd have no idea
> what code is doing the work.
Once a module is imported which isn't being managed by mod_python,
then the mapping of 'import' to 'apache.import_module()' doesn't get
triggered. Thus, it applies only to stuff in the web application and
not to anything on sys.path.
FWIW, overriding 'import' in ways like this is not uncommon in Python
as this is how importing stuff from Zip files and unexpanded Python
eggs works. The stuff works and it steps out of the way when it
shouldn't be applied.
I remember when I first brought up this idea of 'import' working in
magic ways so things would just work as one expected, that there were
at least one person who thought it was a silly idea. In the end I had
to give up on even trying to explain it because it would have taken me
months just to get people to understand what I had in mind. Instead, I
just wrote the code and I believe most would have to say the result in
certainly better than what was there before and that code now just
works as how one would expect it to work when one uses 'import'
whereas it didn't always before.
> This is getting way too obfuscated for my simple ways.
All I can suggest is you try it by starting with the changes to
mpservlets as I described and then simply see if your existing
applications which use mpservlets still work. Since the release we
have only had a few people even be slightly affected by the
differences, plus all the complaints we used to get about problems
with the old importer have pretty well vanished. The only known issues
which currently exist are case sensitivity on Windows and a race
condition on imports causing a redundant reload. Both issues are
something that will be able to be fixed though.
Graham
Re: sys.path and htaccess change?
Posted by "Daniel J. Popowich" <dp...@comcast.net>.
Graham Dumpleton writes:
> And that is what the documentation I pointed you at states:
>
> """Note that with the new module importer, as directories associated
> with Python*Handler directives are no longer being added automatically
> to sys.path and they are instead used directly by the module importer
> only when required, some existing code which expected to be able to
> import modules in the handler root directory from a module in a
> subdirectory may no longer work. In these situations it will be
> necessary to set the mod_python module importer path to include '~' or
> list '~' in the __mp_path__ attribute of the module performing the
> import."""
OK, I think it's slowly sinking in...when I put the ~ in, it didn't
affect sys.path, so my import still wasn't going to work. Right? So,
how the heck do I import the modules? Do I have to use
apache.import_module() now? But I don't want to!
> If one wants to be precise, the addition of the handler directory to
> sys.path was actually done by the top level mod_python dispatch code
> rather than the module importer itself.
And this I don't understand at all. If this "feature" was in the old
system, why would reimplementing an api function change this behaviour
in the dispatch code? I would consider this a bug. I would also say
this is WAY too much functionality change for a minor release update.
> The other important bit is:
>
> """As a new feature in mod_python 3.3, when using the standard Python
> 'import' statement to import a module, if the import is being done
> from a module which was previously imported by the mod_python module
> importer, it is equivalent to having called apache.import_module()
> directly."""
>
> It is because of this that the directory doesn't need to be in
> sys.path, as under the covers, when 'import' is being used by modules
> imported by the mod_python module importer directly in some way, then
> 'import' is equivalent to having used 'apache.import_module()' and
> therefore it looks in places related to mod_python including stuff in
> the separate mod_python importer path.
And I would call this a bug, too. Why should using mod_python change
the behaviour of the import builtin? What if I'm including a package
that includes a package that includes a package? I'd have no idea
what code is doing the work.
This is getting way too obfuscated for my simple ways.
Dan
Re: sys.path and htaccess change?
Posted by Graham Dumpleton <gr...@gmail.com>.
On 18/05/07, Daniel J. Popowich <dp...@comcast.net> wrote:
>
> Graham Dumpleton writes:
> > > When I go to http://localhost/~dpopowich/py/syspath, a simple
> > > mpservlet that writes out the current value of sys.path, I do NOT see
> > > /home/dpopowich/public_html/py in the path.
> >
> > You will not see the path listed in sys.path. The mod_python module
> > importer path and the Python sys.path are deliberately kept separate
> > because of all the problems caused by each looking in the same
> > location.
>
> But I did see the path in sys.path before 3.3. Prior to 3.3, the
> directory that contained the SetHandler directive would be in
> sys.path. I have no idea if this behaviour in versions <3.3 is
> because of the old apache.import_module or some other "feature," but
> it's not there in 3.3.
And that is what the documentation I pointed you at states:
"""Note that with the new module importer, as directories associated
with Python*Handler directives are no longer being added automatically
to sys.path and they are instead used directly by the module importer
only when required, some existing code which expected to be able to
import modules in the handler root directory from a module in a
subdirectory may no longer work. In these situations it will be
necessary to set the mod_python module importer path to include '~' or
list '~' in the __mp_path__ attribute of the module performing the
import."""
This was immediately below the example line I referred you to go look for.
If one wants to be precise, the addition of the handler directory to
sys.path was actually done by the top level mod_python dispatch code
rather than the module importer itself.
The other important bit is:
"""As a new feature in mod_python 3.3, when using the standard Python
'import' statement to import a module, if the import is being done
from a module which was previously imported by the mod_python module
importer, it is equivalent to having called apache.import_module()
directly."""
It is because of this that the directory doesn't need to be in
sys.path, as under the covers, when 'import' is being used by modules
imported by the mod_python module importer directly in some way, then
'import' is equivalent to having used 'apache.import_module()' and
therefore it looks in places related to mod_python including stuff in
the separate mod_python importer path.
> > > And when I try to import
> > > a python module in that directory, I get import errors.
> >
> > If you are still using the old module importer in mpservlets to import
> > your .mps file then that would probably still occur.
>
> No. I do NOT use apache.import_module() to load .mps files. I rolled
> my own, using execfile on the filename coming from req.filename.
When I said 'old module importer' in that case, I meant your hand
rolled one in mpservlets, not the old mod_python one.
Graham
Re: sys.path and htaccess change?
Posted by "Daniel J. Popowich" <dp...@comcast.net>.
Graham Dumpleton writes:
> > When I go to http://localhost/~dpopowich/py/syspath, a simple
> > mpservlet that writes out the current value of sys.path, I do NOT see
> > /home/dpopowich/public_html/py in the path.
>
> You will not see the path listed in sys.path. The mod_python module
> importer path and the Python sys.path are deliberately kept separate
> because of all the problems caused by each looking in the same
> location.
But I did see the path in sys.path before 3.3. Prior to 3.3, the
directory that contained the SetHandler directive would be in
sys.path. I have no idea if this behaviour in versions <3.3 is
because of the old apache.import_module or some other "feature," but
it's not there in 3.3.
> > And when I try to import
> > a python module in that directory, I get import errors.
>
> If you are still using the old module importer in mpservlets to import
> your .mps file then that would probably still occur.
No. I do NOT use apache.import_module() to load .mps files. I rolled
my own, using execfile on the filename coming from req.filename.
Re: sys.path and htaccess change?
Posted by Graham Dumpleton <gr...@gmail.com>.
On 18/05/07, Daniel J. Popowich <dp...@comcast.net> wrote:
>
> Graham Dumpleton writes:
> > Read:
> >
> > http://www.modpython.org/live/current/doc-html/pyapi-apmeth.html
> >
> > Especially the area which starts just before:
> >
> > PythonOption mod_python.importer.path "['~']"
> >
> > Your particular issue is mentioned just after this example.
>
> Doesn't work, either in .htaccess or in <Directory> directives. This
> is what I have in my apache config:
>
> <Directory "/home/dpopowich/public_html/py">
> SetHandler mod_python
>
> PythonHandler mod_python.servlet
> PythonOption mod_python.importer.path "['~']"
> PythonDebug on
> </Directory>
>
> When I go to http://localhost/~dpopowich/py/syspath, a simple
> mpservlet that writes out the current value of sys.path, I do NOT see
> /home/dpopowich/public_html/py in the path.
You will not see the path listed in sys.path. The mod_python module
importer path and the Python sys.path are deliberately kept separate
because of all the problems caused by each looking in the same
location.
> And when I try to import
> a python module in that directory, I get import errors.
If you are still using the old module importer in mpservlets to import
your .mps file then that would probably still occur.
> However, if I put this in my apache global context:
>
> PythonOption mod_python.legacy.importer *
>
> this fixes my immediate problem.
>
> Is this the way to completely turn this subsystem OFF?
Yes, but the old importer will disappear at some point in the future.
> As a side note: I've been away from mod_python for a while and now am
> getting back into things and thought one project would be to update
> mpservlets to use the new importer I've heard so much about instead of
> the one I wrote for mpservlets. I must confess I'm a bit taken aback
> that it takes over six pages to document this one function
Considering the list of problems with the old importer came to about
12 pages, I thought it wasn't too bad. :-)
http://www.dscpl.com.au/wiki/ModPython/Articles/ModuleImportingIsBroken
> and a quick
> read has left me scratching my head and wondering if this is the way I
> want to go. I must confess I find it less than intuitive.
Just change this in your mpservlets package. The new importer can
handle arbitrary extensions so can load .mps files no problem. With
just this change your tutorial example works with no problems and it
uses 'import' to get stuff from handler directory.
kundalini:/usr/local/src/mpservlets-1.1.6 grahamd$ diff
servlet.py.dist servlet.py
1269,1270c1269,1271
< code = {}
< execfile(fname, code)
---
> #code = {}
> #execfile(fname, code)
> module = apache.import_module(fname)
1276c1277,1278
< klass = code[basename]
---
> #klass = code[basename]
> klass = getattr(module, basename)
Graham
Re: sys.path and htaccess change?
Posted by "Daniel J. Popowich" <dp...@comcast.net>.
Graham Dumpleton writes:
> Read:
>
> http://www.modpython.org/live/current/doc-html/pyapi-apmeth.html
>
> Especially the area which starts just before:
>
> PythonOption mod_python.importer.path "['~']"
>
> Your particular issue is mentioned just after this example.
Doesn't work, either in .htaccess or in <Directory> directives. This
is what I have in my apache config:
<Directory "/home/dpopowich/public_html/py">
SetHandler mod_python
PythonHandler mod_python.servlet
PythonOption mod_python.importer.path "['~']"
PythonDebug on
</Directory>
When I go to http://localhost/~dpopowich/py/syspath, a simple
mpservlet that writes out the current value of sys.path, I do NOT see
/home/dpopowich/public_html/py in the path. And when I try to import
a python module in that directory, I get import errors.
However, if I put this in my apache global context:
PythonOption mod_python.legacy.importer *
this fixes my immediate problem.
Is this the way to completely turn this subsystem OFF?
Dan
PS
As a side note: I've been away from mod_python for a while and now am
getting back into things and thought one project would be to update
mpservlets to use the new importer I've heard so much about instead of
the one I wrote for mpservlets. I must confess I'm a bit taken aback
that it takes over six pages to document this one function and a quick
read has left me scratching my head and wondering if this is the way I
want to go. I must confess I find it less than intuitive.
Re: sys.path and htaccess change?
Posted by Graham Dumpleton <gr...@gmail.com>.
Read:
http://www.modpython.org/live/current/doc-html/pyapi-apmeth.html
Especially the area which starts just before:
PythonOption mod_python.importer.path "['~']"
Your particular issue is mentioned just after this example.
Graham
On 18/05/07, Daniel J. Popowich <dp...@astro.umass.edu> wrote:
>
> In mod_python 3.2.x, if I set a directory to use mod_python with a
> .htaccess file, I will find that directory in sys.path. That is, if:
>
> /var/www/somedir/.htaccess
>
> contains:
>
> SetHandler python-program
>
> then sys.path will have "/var/www/somedir" as element zero.
>
> With 3.3.x this does not happen.
>
> The problem? I have a handler with co-located python modules that get
> imported (the handler does "import somemod" and somemomd is in the
> same directory as the .htaccess file). This works fine with 3.2.x,
> but when I upgraded to 3.3.1 I get import errors.
>
> I searched the changelogs in the documentation, but I see no mention
> of this change.
>
> Thanks,
>
> Daniel Popowich
> ---------------
> http://www.astro.umass.edu/~dpopowich/python/mpservlets/
>