release?

["Gregory (Grisha) Trubetskoy" <gr...@ispol.com>]

What do you folks think about gearing up for a release? Meaning not adding 
new features and focusing on the little details (like docs, tests, bugs).

The only major thing that I think needs to be finalized is FileSession - 
I'd like it to see all the locking issues resolved and either the code 
rolled in to Session.py or Session.py split into separate files by class, 
for consistency. I also think that if FileSession works well, it should be 
made the default session mechanism.

Anything else major that I'm missing? I'd like to start a little 
discussion on what would be a reasonable timeline and start making baby 
steps towards achieving it.

Cheers

Grisha

Re: release?

[Nicolas Lehuen <ni...@gmail.com>]

On 4/27/05, Gregory (Grisha) Trubetskoy <gr...@ispol.com> wrote:
> 
> What do you folks think about gearing up for a release? Meaning not adding
> new features and focusing on the little details (like docs, tests, bugs).
> 
> The only major thing that I think needs to be finalized is FileSession -
> I'd like it to see all the locking issues resolved and either the code
> rolled in to Session.py or Session.py split into separate files by class,
> for consistency. I also think that if FileSession works well, it should be
> made the default session mechanism.
> 
> Anything else major that I'm missing? I'd like to start a little
> discussion on what would be a reasonable timeline and start making baby
> steps towards achieving it.
> 
> Cheers
> 
> Grisha
> 

Woops, just when I've checked in a big set of changes ;-). I've just
fixed MODPYTHON-9, which was to me the most annoying bug that was left
in mod_python.publisher. While I was at it, I've fixed MODPYTHON-8, so
the locking mechanism of the publisher is much more lightweight.

I've done non-regression tests using the unit test suite (to which
I've added a few more tests to check for some special behaviour of the
publisher). Everything seems OK, but one or two +1s from people using
the publisher would be great.

I'm going to check in the last fixes to FileSession, if everybody
agrees with Jim's design. I think the fact that the grace period does
not protect against long running requests (requests running for more
than the grace period which is typically 120 seconds) should be a
documented feature, not a bug :). People running long request should
either not run them with a session context, or know what they are
doing.

Apart from that, yes, a 3.2.0 release would be great, there is quite a
lot of bug fixes that are awaiting to come to the real world now :).

Regards,

Nicolas

Re: release?

[dharana <dh...@dharana.net>]

I know I have little voice here but it would be nice to see a release in 
the near future particularly because of FileSession and the psp string= 
bugfix. If you need help testing the FileSession when the lock stuff is 
finished or just general testing I will gladly help. I've been using a 
FileSession without locking since I sent to the list without a problem. 
However it was just me and the server, nobody else.

Gregory (Grisha) Trubetskoy wrote:
> 
> What do you folks think about gearing up for a release? Meaning not 
> adding new features and focusing on the little details (like docs, 
> tests, bugs).
> 
> The only major thing that I think needs to be finalized is FileSession - 
> I'd like it to see all the locking issues resolved and either the code 
> rolled in to Session.py or Session.py split into separate files by 
> class, for consistency. I also think that if FileSession works well, it 
> should be made the default session mechanism.
> 
> Anything else major that I'm missing? I'd like to start a little 
> discussion on what would be a reasonable timeline and start making baby 
> steps towards achieving it.
> 
> Cheers
> 
> Grisha
> 
> 
> 

-- 
dharana

Re: release? FileSession.py

[Jim Gallacher <jg...@sympatico.ca>]

Nicolas Lehuen wrote:
> On 4/28/05, Nicolas Lehuen <ni...@gmail.com> wrote:
> 
>>On 4/27/05, dharana <dh...@dharana.net> wrote:
>>
>>>Jim Gallacher wrote:
>>>
>>>>Nicolas Lehuen wrote:
>>>>
>>>>[...]
>>>>
>>>>Attached is my latest and hopefully greatest. :)  This version uses the
>>>>grace_period idea to avoid race and/or deadlock issues and correctly
>>>>register the filesession_cleanup method to run at the end of a request.
>>
>>Thanks, I'll integrate this ASAP tomorrow morning (it's too late for now :).
> 
> 
> OK, this is done.
> 
> 
>>>>In the original code there was a comment (from dharana?) about a random
>>>>EOFError in the do_load method. I have not been able to reproduce this
>>>>error but I left the comment in anyway.
>>>
>>>No, it's not mine. I have yet to see it happen.
>>
>>It was mine. The bug is totally reproducible, on my machine at least.
>>I'll investigate this a little more tomorrow.
> 
> 
> Found it. It turns out that we need to open the session file in binary
> mode for reading and writing.
> 
> I've never really understood why, but on Win32 files can be opened in
> text mode, in which some control characters mean an EOF. If you try to
> write binary data in a text-mode file, and your binary data contains
> an EOF character, then you're out of luck. You have to open the file
> in binary mode, by adding an 'b' to the mode.
> 
> What lead to to find this was that when using the text-friendly
> pickling protocol 0, there was no problem. The bug only appeared when
> using the binary pickling protocol 2.
> 

Ah, yes. One of those cross-platform wrinkles. Glad the fix was simple.

Jim

Re: release? FileSession.py

[Nicolas Lehuen <ni...@gmail.com>]

Wooops, I've integrated FileSession.py into Session.py because I
thought the subject was settled. Anyway, it's not so difficult to
split it again. Only this time, I'll make sure that everyone agrees...

Regards
Nicolas

On 4/30/05, Jim Gallacher <jg...@sympatico.ca> wrote:
> Gregory (Grisha) Trubetskoy wrote:
> >
> > On Thu, 28 Apr 2005, Nicolas Lehuen wrote:
> >
> >> OK so now I think we have a pretty good implementation of FileSession :)
> >
> >
> > Excellent - now who's in favor of splitting Session into multiple files
> > versus rolling FileSession inti Session.py?
> 
> +1 for splitting.
> 
> > I'm thinking that if the Session.py is split, then same should be done
> > with Cookie.py for consistency and that's too much changes when we are
> > in a pre-release stage.
> 
> Agreed. Since this is mainly a bug fix release, except for FileSession,
>   I wouldn't shift things around too much.
> 
> > So I'm in favor of adding FileSession into Session.py for _this_ release
> > with the plan to consider splitting Session.py and Cookie.py later.
> 
> I'm not sure I like the idea of merging FileSession into Session and
> then splitting it back out in a future release. This just does not seem
> all that graceful. If the plan is to split Session, then start with
> FileSession as it will not impact any current code. When the ultimate
> future split occurs at least some application code will not be broken.
> 
> Regards,
> Jim
>

Re: release? FileSession.py

[Jim Gallacher <jg...@sympatico.ca>]

Gregory (Grisha) Trubetskoy wrote:
> 
> On Thu, 28 Apr 2005, Nicolas Lehuen wrote:
> 
>> OK so now I think we have a pretty good implementation of FileSession :)
> 
> 
> Excellent - now who's in favor of splitting Session into multiple files 
> versus rolling FileSession inti Session.py?

+1 for splitting.

> I'm thinking that if the Session.py is split, then same should be done 
> with Cookie.py for consistency and that's too much changes when we are 
> in a pre-release stage.

Agreed. Since this is mainly a bug fix release, except for FileSession, 
  I wouldn't shift things around too much.

> So I'm in favor of adding FileSession into Session.py for _this_ release 
> with the plan to consider splitting Session.py and Cookie.py later.

I'm not sure I like the idea of merging FileSession into Session and 
then splitting it back out in a future release. This just does not seem 
all that graceful. If the plan is to split Session, then start with 
FileSession as it will not impact any current code. When the ultimate 
future split occurs at least some application code will not be broken.

Regards,
Jim

Re: release? FileSession.py

["Gregory (Grisha) Trubetskoy" <gr...@ispol.com>]

On Thu, 28 Apr 2005, Nicolas Lehuen wrote:

> OK so now I think we have a pretty good implementation of FileSession :)

Excellent - now who's in favor of splitting Session into multiple files 
versus rolling FileSession inti Session.py?

I'm thinking that if the Session.py is split, then same should be done 
with Cookie.py for consistency and that's too much changes when we are in 
a pre-release stage.

So I'm in favor of adding FileSession into Session.py for _this_ release 
with the plan to consider splitting Session.py and Cookie.py later.

Grisha

Re: release? FileSession.py

[Nicolas Lehuen <ni...@gmail.com>]

On 4/28/05, Nicolas Lehuen <ni...@gmail.com> wrote:
> On 4/27/05, dharana <dh...@dharana.net> wrote:
> > Jim Gallacher wrote:
> > > Nicolas Lehuen wrote:
> > >
> > > [...]
> > >
> > > Attached is my latest and hopefully greatest. :)  This version uses the
> > > grace_period idea to avoid race and/or deadlock issues and correctly
> > > register the filesession_cleanup method to run at the end of a request.
> 
> Thanks, I'll integrate this ASAP tomorrow morning (it's too late for now :).

OK, this is done.

> > > In the original code there was a comment (from dharana?) about a random
> > > EOFError in the do_load method. I have not been able to reproduce this
> > > error but I left the comment in anyway.
> >
> > No, it's not mine. I have yet to see it happen.
> 
> It was mine. The bug is totally reproducible, on my machine at least.
> I'll investigate this a little more tomorrow.

Found it. It turns out that we need to open the session file in binary
mode for reading and writing.

I've never really understood why, but on Win32 files can be opened in
text mode, in which some control characters mean an EOF. If you try to
write binary data in a text-mode file, and your binary data contains
an EOF character, then you're out of luck. You have to open the file
in binary mode, by adding an 'b' to the mode.

What lead to to find this was that when using the text-friendly
pickling protocol 0, there was no problem. The bug only appeared when
using the binary pickling protocol 2.

OK so now I think we have a pretty good implementation of FileSession :)

Regards,
Nicolas

> > >
> > > Also, my comments for the filesession_cleanup function are rather
> > > verbose. Feel free to trim them as you see fit.
> > >
> > > Regards,
> > > Jim
> >
> > I have tested this version's performance which was the driving force
> > behind the initial suggestion. Performance-wise, this FileSession
> > version is as fast as the first rough one without locking, grace times,
> > etc. The difference is less than 1 req/s in the 165 req/s' range.
> >
> > I've switched my dev app to it and i will be using it daily from now on.
> >
> > --
> > dharana
> 
> Great ! Please give us your feedback on the behaviour of the new FileSession.
> 
> Regards,
> 
> Nicolas
>

Re: release? FileSession.py

[Nicolas Lehuen <ni...@gmail.com>]

On 4/27/05, dharana <dh...@dharana.net> wrote:
> Jim Gallacher wrote:
> > Nicolas Lehuen wrote:
> >
> > [...]
> >
> > Attached is my latest and hopefully greatest. :)  This version uses the
> > grace_period idea to avoid race and/or deadlock issues and correctly
> > register the filesession_cleanup method to run at the end of a request.

Thanks, I'll integrate this ASAP tomorrow morning (it's too late for now :).

> > In the original code there was a comment (from dharana?) about a random
> > EOFError in the do_load method. I have not been able to reproduce this
> > error but I left the comment in anyway.
>
> No, it's not mine. I have yet to see it happen.

It was mine. The bug is totally reproducible, on my machine at least.
I'll investigate this a little more tomorrow.

> >
> > Also, my comments for the filesession_cleanup function are rather
> > verbose. Feel free to trim them as you see fit.
> >
> > Regards,
> > Jim
> 
> I have tested this version's performance which was the driving force
> behind the initial suggestion. Performance-wise, this FileSession
> version is as fast as the first rough one without locking, grace times,
> etc. The difference is less than 1 req/s in the 165 req/s' range.
> 
> I've switched my dev app to it and i will be using it daily from now on.
> 
> --
> dharana

Great ! Please give us your feedback on the behaviour of the new FileSession.

Regards,

Nicolas

Re: release? FileSession.py

[dharana <dh...@dharana.net>]

Jim Gallacher wrote:
> Nicolas Lehuen wrote:
> 
> [...]
> 
> Attached is my latest and hopefully greatest. :)  This version uses the 
> grace_period idea to avoid race and/or deadlock issues and correctly 
> register the filesession_cleanup method to run at the end of a request.
> 
> In the original code there was a comment (from dharana?) about a random 
> EOFError in the do_load method. I have not been able to reproduce this 
> error but I left the comment in anyway.

No, it's not mine. I have yet to see it happen.

> 
> Also, my comments for the filesession_cleanup function are rather 
> verbose. Feel free to trim them as you see fit.
> 
> Regards,
> Jim

I have tested this version's performance which was the driving force 
behind the initial suggestion. Performance-wise, this FileSession 
version is as fast as the first rough one without locking, grace times, 
etc. The difference is less than 1 req/s in the 165 req/s' range.

I've switched my dev app to it and i will be using it daily from now on.

-- 
dharana

Re: release? FileSession.py

[Jim Gallacher <jg...@sympatico.ca>]

Nicolas Lehuen wrote:
> Jim,
> 
> Are you OK with this version of FileSession.py ? 

No, I moved beyond that version a while ago.

> I'm not sure I
> totally followed your thoughts since at a point the thread on
> FileSession became a bit blurry to me.

Hardly a suprise. It was getting kind of convoluted to me as well.

> If it's not OK, please send me
> a corrected version.

Attached is my latest and hopefully greatest. :)  This version uses the 
grace_period idea to avoid race and/or deadlock issues and correctly 
register the filesession_cleanup method to run at the end of a request.

In the original code there was a comment (from dharana?) about a random 
EOFError in the do_load method. I have not been able to reproduce this 
error but I left the comment in anyway.

Also, my comments for the filesession_cleanup function are rather 
verbose. Feel free to trim them as you see fit.

Regards,
Jim



> Regards,
> Nicolas
> 
> On 4/27/05, Jim Gallacher <jg...@sympatico.ca> wrote:
> 
>>Gregory (Grisha) Trubetskoy wrote:
>>
>>>What do you folks think about gearing up for a release? Meaning not
>>>adding new features and focusing on the little details (like docs,
>>>tests, bugs).
>>
>>+1
>>
>>
>>>The only major thing that I think needs to be finalized is FileSession -
>>>I'd like it to see all the locking issues resolved and either the code
>>>rolled in to Session.py or Session.py split into separate files by
>>>class, for consistency. I also think that if FileSession works well, it
>>>should be made the default session mechanism.
>>
>>I think the last patch I submitted (2005-04-21) takes care of the
>>locking issues, but I don't think it's been committed yet. There is
>>still one small change I'd like to make in the do_load method, but other
>>than that I think the code is in good shape.
>>
>>I like the idea of separate files for each class since it would make it
>>easy to add additional backends if there is such a desire in the future.
>>
>>
>>>Anything else major that I'm missing? I'd like to start a little
>>>discussion on what would be a reasonable timeline and start making baby
>>>steps towards achieving it.
>>>
>>
>>I'll be away until Sunday, but should be able to put together some
>>documentation for FileSession and do some additional testing early next
>>week.
>>
>>Regards,
>>Jim
>>

Re: release?

[Nicolas Lehuen <ni...@gmail.com>]

Jim,

Are you OK with this version of FileSession.py ? I'm not sure I
totally followed your thoughts since at a point the thread on
FileSession became a bit blurry to me. If it's not OK, please send me
a corrected version.

Regards,
Nicolas

On 4/27/05, Jim Gallacher <jg...@sympatico.ca> wrote:
> Gregory (Grisha) Trubetskoy wrote:
> >
> > What do you folks think about gearing up for a release? Meaning not
> > adding new features and focusing on the little details (like docs,
> > tests, bugs).
> 
> +1
> 
> > The only major thing that I think needs to be finalized is FileSession -
> > I'd like it to see all the locking issues resolved and either the code
> > rolled in to Session.py or Session.py split into separate files by
> > class, for consistency. I also think that if FileSession works well, it
> > should be made the default session mechanism.
> 
> I think the last patch I submitted (2005-04-21) takes care of the
> locking issues, but I don't think it's been committed yet. There is
> still one small change I'd like to make in the do_load method, but other
> than that I think the code is in good shape.
> 
> I like the idea of separate files for each class since it would make it
> easy to add additional backends if there is such a desire in the future.
> 
> > Anything else major that I'm missing? I'd like to start a little
> > discussion on what would be a reasonable timeline and start making baby
> > steps towards achieving it.
> >
> 
> I'll be away until Sunday, but should be able to put together some
> documentation for FileSession and do some additional testing early next
> week.
> 
> Regards,
> Jim
>

Re: release?

[Jim Gallacher <jg...@sympatico.ca>]

Gregory (Grisha) Trubetskoy wrote:
> 
> What do you folks think about gearing up for a release? Meaning not 
> adding new features and focusing on the little details (like docs, 
> tests, bugs).

+1

> The only major thing that I think needs to be finalized is FileSession - 
> I'd like it to see all the locking issues resolved and either the code 
> rolled in to Session.py or Session.py split into separate files by 
> class, for consistency. I also think that if FileSession works well, it 
> should be made the default session mechanism.

I think the last patch I submitted (2005-04-21) takes care of the 
locking issues, but I don't think it's been committed yet. There is 
still one small change I'd like to make in the do_load method, but other 
than that I think the code is in good shape.

I like the idea of separate files for each class since it would make it 
easy to add additional backends if there is such a desire in the future.

> Anything else major that I'm missing? I'd like to start a little 
> discussion on what would be a reasonable timeline and start making baby 
> steps towards achieving it.
> 

I'll be away until Sunday, but should be able to put together some 
documentation for FileSession and do some additional testing early next 
week.

Regards,
Jim

Re: release?

["Gregory (Grisha) Trubetskoy" <gr...@ispol.com>]

I'm thinking we're OK to require at least 2.3. If you are one of the users 
who still runs 2.2, then latest mod_python is probably not for you either.

Grisha

On Thu, 28 Apr 2005, Graham Dumpleton wrote:

>
> On 28/04/2005, at 1:18 AM, Gregory (Grisha) Trubetskoy wrote:
>
>> 
>> What do you folks think about gearing up for a release? Meaning not adding 
>> new features and focusing on the little details (like docs, tests, bugs).
>
> My only comment is that you need to make clear what the minimum version of
> Python that is required is. I have seen some changes that seem to rule out
> use of Python 2.2 any more. Thus, if Python 2.3 is the minimum then state
> it clearly and make sure it is tested for that version. If anything has
> been added that makes Python 2.4 mandatory, you are going to annoy a lot
> of people, less so if Python 2.3 is the minimum. We still run Python 2.2 on
> production servers where I work, but the one machine that runs mod_python
> is still Apache 1.3 and mod_python 2.7 so it isn't an issue, but doesn't
> mean others will not grumble.
>
> Graham
>

Re: release?

[Graham Dumpleton <gr...@dscpl.com.au>]

On 28/04/2005, at 1:18 AM, Gregory (Grisha) Trubetskoy wrote:

>
> What do you folks think about gearing up for a release? Meaning not 
> adding new features and focusing on the little details (like docs, 
> tests, bugs).

My only comment is that you need to make clear what the minimum version 
of
Python that is required is. I have seen some changes that seem to rule 
out
use of Python 2.2 any more. Thus, if Python 2.3 is the minimum then 
state
it clearly and make sure it is tested for that version. If anything has
been added that makes Python 2.4 mandatory, you are going to annoy a lot
of people, less so if Python 2.3 is the minimum. We still run Python 
2.2 on
production servers where I work, but the one machine that runs 
mod_python
is still Apache 1.3 and mod_python 2.7 so it isn't an issue, but doesn't
mean others will not grumble.

Graham

Re: release?

["Gregory (Grisha) Trubetskoy" <gr...@ispol.com>]

On Wed, 27 Apr 2005, Daniel Popowich wrote:

>
> Gregory (Grisha) Trubetskoy writes:
>>
>> What do you folks think about gearing up for a release? Meaning not adding
>> new features and focusing on the little details (like docs, tests, bugs).
>
> Great idea!
>
> Also...I have on a number of occasions attempted to donate mpservlets
> to the mainstream modpython distribution.  Assuming I merge my
> documentation into the standard format, what do you think about
> including this handler in this release?

I personally am guilty of not having taken the tame to look at mpservlets, 
so it's a +0 from me - but what do others on the list think? This might be 
the thing to do after this release.

Grisha

Re: release?

[Daniel Popowich <dp...@comcast.net>]

Gregory (Grisha) Trubetskoy writes:
> 
> What do you folks think about gearing up for a release? Meaning not adding 
> new features and focusing on the little details (like docs, tests, bugs).

Great idea!

Also...I have on a number of occasions attempted to donate mpservlets
to the mainstream modpython distribution.  Assuming I merge my
documentation into the standard format, what do you think about
including this handler in this release?

Daniel Popowich
-----------------------------------------------
http://home.comcast.net/~d.popowich/mpservlets/