Pickling/unpickling top-level functions, classes etc.

[Graham Dumpleton <gr...@dscpl.com.au>]

Nicolas

Are you okay with:

  http://issues.apache.org/jira/browse/MODPYTHON-81

  Pickling/unpickling top-level functions defined in published
  module no longer works in mod_python 3.2

being resolved as "Won't Fix" and then closed?

As I describe in:

  http://www.dscpl.com.au/articles/modpython-005.html

there are going to be various issues with pickling with any new importer
which doesn't keep stuff in sys.modules.

I don't see any solution for the issue as far as modules managed by the
mod_python importer. Simply means that if you want to pickle stuff like
that, has to be in module on standard sys.path and managed by normal
Python module import system.

Anyone else want to comment?

Graham

Re: Pickling/unpickling top-level functions, classes etc.

[Nicolas Lehuen <ni...@lehuen.com>]

OK, I'm +1 on the "Won't fix" status.

I'm not proficient enough in the way unpickling works, but the fact that you
cannot specify any namespace in which classes or functions names have to be
resolved makes it quite clear that dynamic imports + unpickling functions
and classes = not possible.

Regards,
Nicolas

2006/3/29, Deron Meranda <de...@gmail.com>:
>
> On 3/29/06, Graham Dumpleton <gr...@dscpl.com.au> wrote:
> > Are you okay with:
> >
> >   http://issues.apache.org/jira/browse/MODPYTHON-81
> >
> >   Pickling/unpickling top-level functions defined in published
> >   module no longer works in mod_python 3.2
> >
> > being resolved as "Won't Fix" and then closed?
>
> I agree that this seems to be something that is just not
> solvable without causing complete havoc with all the
> specialized import and reload functionality, or resulting
> in a solution that is too fragile.  It is just a limitation of
> the pickle mechanism.
>
> This of course doesn't mean that users wouldn't want to
> pickle these kinds of things.  But that the burden in those
> cases should be on them.  It may be possible to solve
> this for class instances (e.g., objects) by subclassing the
> Unpickler class and substituting a smarter find_class()
> method.  But as for globals, functions, etc., it looks like
> it may be much harder.
>
> The user may also be able to take advantage of the
> external object pickling (with persistent ids), but I
> haven't looked at them too closely.
>
> Regardless, there are lots of alternatives, so I have
> no problem with mod_python not solving this one
> (although the mod_python documentation should
> clearly emphasize these pickling limitiations).
> --
> Deron Meranda
>

Re: Pickling/unpickling top-level functions, classes etc.

[Deron Meranda <de...@gmail.com>]

On 3/29/06, Graham Dumpleton <gr...@dscpl.com.au> wrote:
> Are you okay with:
>
>   http://issues.apache.org/jira/browse/MODPYTHON-81
>
>   Pickling/unpickling top-level functions defined in published
>   module no longer works in mod_python 3.2
>
> being resolved as "Won't Fix" and then closed?

I agree that this seems to be something that is just not
solvable without causing complete havoc with all the
specialized import and reload functionality, or resulting
in a solution that is too fragile.  It is just a limitation of
the pickle mechanism.

This of course doesn't mean that users wouldn't want to
pickle these kinds of things.  But that the burden in those
cases should be on them.  It may be possible to solve
this for class instances (e.g., objects) by subclassing the
Unpickler class and substituting a smarter find_class()
method.  But as for globals, functions, etc., it looks like
it may be much harder.

The user may also be able to take advantage of the
external object pickling (with persistent ids), but I
haven't looked at them too closely.

Regardless, there are lots of alternatives, so I have
no problem with mod_python not solving this one
(although the mod_python documentation should
clearly emphasize these pickling limitiations).
--
Deron Meranda

Re: Pickling/unpickling top-level functions, classes etc.

[Jim Gallacher <jp...@jgassociates.ca>]

Graham Dumpleton wrote:
> Nicolas
> 
> Are you okay with:
> 
>   http://issues.apache.org/jira/browse/MODPYTHON-81
> 
>   Pickling/unpickling top-level functions defined in published
>   module no longer works in mod_python 3.2
> 
> being resolved as "Won't Fix" and then closed?
> 
> As I describe in:
> 
>   http://www.dscpl.com.au/articles/modpython-005.html
> 
> there are going to be various issues with pickling with any new importer
> which doesn't keep stuff in sys.modules.
> 
> I don't see any solution for the issue as far as modules managed by the
> mod_python importer. Simply means that if you want to pickle stuff like
> that, has to be in module on standard sys.path and managed by normal
> Python module import system.
> 
> Anyone else want to comment?

+1 for "Won't Fix"

Then it's pretty easy for us to just say "don't do it". We could offer a 
sub-optimal soltion and tell people "it might work", only to waste alot 
of time on the mailing list explaining why it doesn't.

Jim