Re: New importer is now the default in 3.3 trunk. (PART 3)

[Graham Dumpleton <gr...@dscpl.com.au>]

On 20/08/2006, at 8:14 PM, Graham Dumpleton wrote:

> On 20/08/2006, at 11:22 AM, Graham Dumpleton wrote:
>
>> FYI. Part 1 of some information about new module importer in 3.3.
>
> Part 2 of information about changes to new importer.

And now for the final part 3.

When using apache.import_module(), the old importer only allowed you
to provide the name of a module or submodule in a package. Thus:

   module1 = apache.import_module('module1')
   module2 = apache.import_module('package1.module2')

The new importer still allows this, except that the new importer will  
never
itself load packages and instead relies on builtin importer of Python to
do it. This means that packages must be somewhere on sys.path and are
not candidates for module reloading.

On some platforms (Linux but not MacOSX), you could also do use a slash
in the name to get a module from a subdirectory. Ie.,

   module3 = apache.import_module('prefix/module3')

I don't actually know whether the top level had to be setup as a  
package with
an __init__.py file or not.

In the new importer, you can use a slash as well, without needing a  
__init__.py
file in the subdirectory, and if it resides on the new module  
importers search
path it will use it. With the new module importer, this becomes a  
simplified way
of grouping modules together in a common directory, but where the common
directory name is the only thing visible at top level scope. Ie.,  
like a package
but not a package. As long as imports using 'import' amongst modules  
in that
common directory are local and don't try and reference back through  
the common
directory root, they will work.

Having mentioned 'import', an importer feature of the new module  
importer is
that you can use 'import' in place of apache.import_module() for file  
module
imports. The first place that will be searched for such modules is  
the same
directory as the file doing the importer. Thus there is no need to  
set PythonPath
to directories in the document tree like you had to with the old  
importer and in
fact if you do, you will possibly get a nasty warning in the error  
logs about setting
PythonPath to a directory which is actually a directory managed by  
the new
importer.

Note that 'import' will only use the new importer in this way if the  
module doing
the import was itself imported using the new module importer. Thus,  
if the
module was on sys.path somewhere, imports from those modules work like
before and don't use the new module importer.

In respect of 'import' looking in the same directory first,  
apache.import_module()
will also do this.

When specifying just a module name to apache.import_module() or with the
'import' statement where applicable, the module will be searched  
along a path
consisting of current directory, embedded __mp_path__ of that module and
value of mod_python.importer.path option.

One can also specify other directories to search for a specific  
import using
apache.import_module() by specifying them using the 'path' argument like
with the old importer.

   module5 = apache.import_module('module5', path=['/some/path/ 
modules'])

This will just be an additional place which is looked. If is  
necessary to specify
exactly where the module resides, the new module importer now allows  
a full path to
the module to be specified.

   module6 = apache.import_module('/some/path/modules/_module6.py')

The extension must be specified, and it doesn't actually have to be .py.
the mpservlets package for example might be updated to use the new
importer and use:

   module7 = apache.import_module('/some/path/module7.mps')

This gives some certainty about which module is imported.

Note that when specifying just the module name, the file must always  
have
a .py extension.

Rather than specifying a full path, it is also possible to specify a  
relative path.
This will be evaluated relative to the directory the file doing the  
import is located
in.

   module8 = apache.import_module('./module8.py')
   module9 = apache.import_module('../module9.py')
   module10 = apache.import_module('../modules/module10.py')

An odd little thing is that when a full or relative path name is  
supplied, any list of
directories specified with the 'path' argument to apache.import_module 
() as the
starting value of the embedded path __mp_path__. This was originally  
done to
support generated code from Cheetah templates, but not strictly be  
required now
given that mod_python.importer.path option exists.

All of the above already worked and existed in the new module  
importer. With the
last set of changes which made the new module importer the default,  
an old
feature was also resurrected. That is, one can also specify as a path  
to get a module
from the handler root directly, using '~/' prefix to path. Thus:

   module11 = apache.import_module('~/module11.py')

This is a short cut so that the handler root doesn't have to be set  
as an embedded
path using __mp_path__ or set explicitly in mod_python.importer.path.

BTW, the 'log' and 'debug' arguments to apache.import_module() are  
effectively
redundant now although they are still there. The arguments default to  
None which
means the importer will pick up the values itself by accessing the  
config table
object using apache.get_current_config(). Because these arguments are  
2nd
and 3rd and the 'path' is 4th, you should always use a keyword  
argument for 'path'
and simply skip 'log' and 'debug'. The 'log' and 'debug' arguments  
are kept for
backward compatibility only.

Okay, that is it for now. I'll have more to say about  
mod_python.importer.path in
a day or so when what I have posted so far has been digested. Because  
the
value of mod_python.importer.path is eval'd there are some cute backdoor
tricks that can be used to avoiding using absolute path names in it  
and these
tricks should perhaps be formalised and agreed that they be allowed.

Until next time. I'll stop bombarding you will mail now for the rest  
of my weekend
at least. :-)

Graham