You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@perl.apache.org by Joe Schaefer <jo...@sunstarsys.com> on 2003/06/02 08:09:18 UTC
[rfc] APR::Table & polymorphic values
I wanted to get some feedback on how to
add APR::Table support for apreq-2's "polymorphic"
value strings.
The issue is that apreq-2 uses apr_tables,
but adds additional metadata to each table
value. The current APR::Table glue uses
newSVpv((char *)val, 0)
to convert the char* to an SV*. That's not
totally adequate for apreq-2, since it forgets
about the metadata stored in front of val.
For apreq-2, what I'd like to have happen
is for a callback to do the type conversion
instead of newSVpv. The callback would be
attached to the APR::Table object, so the
XS glue for APR::Table would need a
patch to use the callback in place of the
newSVpv calls.
It looks to me like mg_ptr is NULL for the
TIEHASH magic associated to APR::Table
objects. Are there any objections to a patch
that uses a callback stored there?
Can anyone recommend a better approach?
--
Joe Schaefer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [rfc] APR::Table & polymorphic values
Posted by Joe Schaefer <jo...@sunstarsys.com>.
Stas Bekman <st...@stason.org> writes:
> The real problem is that it's probably a bad idea to rely on
> non-public APIs, since these behaviors can change at will.
Agreed.
> Perhaps after all, using either a sub-class, or an optional
> callback will be a more robust solution.
The simplest solution I can think of is to add a magic
callback to the APR::Table object.
--
Joe Schaefer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [rfc] APR::Table & polymorphic values
Posted by Joe Schaefer <jo...@sunstarsys.com>.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [rfc] APR::Table & polymorphic values
Posted by Stas Bekman <st...@stason.org>.
Joe Schaefer wrote:
> Stas Bekman <st...@stason.org> writes:
>
>
>>Joe Schaefer wrote:
>>
>>>Stas Bekman <st...@stason.org> writes:
>>>[...]
>>>
>>>
>>>>Still, once user tries to modify the data, what happens to meta-data?
>>>>When COW is performed, does it copy the whole thing or just whatever
>>>>was pointed by SvCUR?
>>>
>>>Doesn't matter. The point is that if there's an Apache::Request::Table
>>>that subclasses APR::Table, it doesn't have to *reimplement everything*
>>>in APR::Table. It can just fixup the return values before they are
>>>returned, like this:
>>> package Apache::Request::Table;
>>> use base 'APR::Table';
>>> sub get {
>>> if (wantarray) {
>>> return map {fixup_sv($_)} &APR::Table::get; }
>>> else {
>>> return fixup_sv(scalar &APR::Table::get);
>>> }
>>> }
>>>The only XSUB we'd need to maintain in the subclass is fixup_sv().
>>>But if APR::Table uses newSVpvn, Apache::Request::Table would need
>>>to *completely reimplent all* the APR::Table XSUBS.
>>
>>So we really are concerned about FETCH, and never about STORE?
>
>
> For APR::Tables, that's right- the copy/merge callbacks (which I'm
> confident apr will adopt) will take care to add some appropriate
> metadata during STORE.
cool
>>What if you use mpxs_apr_table_do to fixup the elements on the first
>>FETCH? Would that be a bad approach?
>
>
> Not sure what you have in mind.
you can run filter on all pairs in the table, see t/response/TestAPR/table.pm
>>The problem is that I still don't understand what do you need that
>>meta data for and how is it used.
>
>
> 1) the perl-glue for apreq will finally be able to handle embedded
> nul characters,
> 2) we can duplicate CGI.pm's overloading of params, since file-uploads
> are just params with a non-NULL bucket-brigade attached.
> 3) we can do lots of cool stuff with cookie values, too.
Any examples of how is it used?
> Here's another idea that is safe, and should work for everyone:
> just add a magic pointer to the original (char *) and keep using
> newSVpv():
>
>
> MP_INLINE static SV *mpxs_apr_table_str_2sv(pTHX_ const char *str)
> {
> SV *sv = newSVpv(str, 0);
> sv_magic(sv, Nullsv, PERL_MAGIC_ext, str, 0);
> return sv_2mortal(sv);
> }
>
> This way Apache::Request::Table can get at the original string
> via mg->mg_ptr, so it just needs to maintain an sv_fixup XSUB.
Can you ensure that apr won't relocate the string? why not pointing at the
newly copied sv's PV?
I can't see where the meta data is in str. Is it before str?
> If this looks ok, I'll try to conjure up a patch :-)
--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [rfc] APR::Table & polymorphic values
Posted by Joe Schaefer <jo...@sunstarsys.com>.
Stas Bekman <st...@stason.org> writes:
> Joe Schaefer wrote:
> > Stas Bekman <st...@stason.org> writes:
> > [...]
> >
> >>Still, once user tries to modify the data, what happens to meta-data?
> >>When COW is performed, does it copy the whole thing or just whatever
> >>was pointed by SvCUR?
> > Doesn't matter. The point is that if there's an Apache::Request::Table
> > that subclasses APR::Table, it doesn't have to *reimplement everything*
> > in APR::Table. It can just fixup the return values before they are
> > returned, like this:
> > package Apache::Request::Table;
> > use base 'APR::Table';
> > sub get {
> > if (wantarray) {
> > return map {fixup_sv($_)} &APR::Table::get; }
> > else {
> > return fixup_sv(scalar &APR::Table::get);
> > }
> > }
> > The only XSUB we'd need to maintain in the subclass is fixup_sv().
> > But if APR::Table uses newSVpvn, Apache::Request::Table would need
> > to *completely reimplent all* the APR::Table XSUBS.
>
> So we really are concerned about FETCH, and never about STORE?
For APR::Tables, that's right- the copy/merge callbacks (which I'm
confident apr will adopt) will take care to add some appropriate
metadata during STORE.
> What if you use mpxs_apr_table_do to fixup the elements on the first
> FETCH? Would that be a bad approach?
Not sure what you have in mind.
> The problem is that I still don't understand what do you need that
> meta data for and how is it used.
1) the perl-glue for apreq will finally be able to handle embedded
nul characters,
2) we can duplicate CGI.pm's overloading of params, since file-uploads
are just params with a non-NULL bucket-brigade attached.
3) we can do lots of cool stuff with cookie values, too.
Here's another idea that is safe, and should work for everyone:
just add a magic pointer to the original (char *) and keep using
newSVpv():
MP_INLINE static SV *mpxs_apr_table_str_2sv(pTHX_ const char *str)
{
SV *sv = newSVpv(str, 0);
sv_magic(sv, Nullsv, PERL_MAGIC_ext, str, 0);
return sv_2mortal(sv);
}
This way Apache::Request::Table can get at the original string
via mg->mg_ptr, so it just needs to maintain an sv_fixup XSUB.
If this looks ok, I'll try to conjure up a patch :-)
--
Joe Schaefer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [rfc] APR::Table & polymorphic values
Posted by Stas Bekman <st...@stason.org>.
Joe Schaefer wrote:
> Stas Bekman <st...@stason.org> writes:
>
>
> [...]
>
>
>>Still, once user tries to modify the data, what happens to meta-data?
>>When COW is performed, does it copy the whole thing or just whatever
>>was pointed by SvCUR?
>
>
> Doesn't matter. The point is that if there's an Apache::Request::Table
> that subclasses APR::Table, it doesn't have to *reimplement everything*
> in APR::Table. It can just fixup the return values before they are
> returned, like this:
>
> package Apache::Request::Table;
> use base 'APR::Table';
>
> sub get {
> if (wantarray) {
> return map {fixup_sv($_)} &APR::Table::get;
> }
> else {
> return fixup_sv(scalar &APR::Table::get);
> }
> }
>
> The only XSUB we'd need to maintain in the subclass is fixup_sv().
> But if APR::Table uses newSVpvn, Apache::Request::Table would need
> to *completely reimplent all* the APR::Table XSUBS.
So we really are concerned about FETCH, and never about STORE?
What if you use mpxs_apr_table_do to fixup the elements on the first FETCH?
Would that be a bad approach? The problem is that I still don't understand
what do you need that meta data for and how is it used.
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [rfc] APR::Table & polymorphic values
Posted by Joe Schaefer <jo...@sunstarsys.com>.
Stas Bekman <st...@stason.org> writes:
[...]
> Still, once user tries to modify the data, what happens to meta-data?
> When COW is performed, does it copy the whole thing or just whatever
> was pointed by SvCUR?
Doesn't matter. The point is that if there's an Apache::Request::Table
that subclasses APR::Table, it doesn't have to *reimplement everything*
in APR::Table. It can just fixup the return values before they are
returned, like this:
package Apache::Request::Table;
use base 'APR::Table';
sub get {
if (wantarray) {
return map {fixup_sv($_)} &APR::Table::get;
}
else {
return fixup_sv(scalar &APR::Table::get);
}
}
The only XSUB we'd need to maintain in the subclass is fixup_sv().
But if APR::Table uses newSVpvn, Apache::Request::Table would need
to *completely reimplent all* the APR::Table XSUBS.
--
Joe Schaefer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [rfc] APR::Table & polymorphic values
Posted by Stas Bekman <st...@stason.org>.
Joe Schaefer wrote:
> Stas Bekman <st...@stason.org> writes:
>
> [...]
>
>
>>http://archive.develooper.com/perl5-porters@perl.org/msg96172.html
>
>
> Thanks for the link!
>
>
>>I haven't read the whole thread, but saw Tim Bunce suggesting later in
>>this thread not to use SvFAKE:
>>http://archive.develooper.com/perl5-porters@perl.org/msg95102.html
>
>
> Using SvLEN = 0 seems to be the cleanest solution. Unfortunately,
> setting SvLEN = 0 prevents correct sv_2iv conversion w/ 5.8. The bug
> is caused by testing for SvLEN > 0 instead of SvCUR > 0 in Perl_sv_2iv.
> I don't know if bleadperl has fixed this, but it's probably behaved
> this way for quite a while.
Please perlbug this to p5p.
>>May be all you need is to manipulate the pointers. So in normal case
>>SVPV will point to the first char, in apreq case it'll point to the
>>char the real data starts from. Am I shooting far off the target?
>
>
> In the normal case:
>
> sv = newSVpv(data,0);
>
> perl mallocs space for a copy and does a strcpy(SvPVX(sv), data).
> That's not good enough for apreq- we really need SvPVX(sv) == data
> in order to get at the metadata in front of &data[0].
The real problem is that it's probably a bad idea to rely on non-public APIs,
since these behaviors can change at will. Perhaps after all, using either a
sub-class, or an optional callback will be a more robust solution.
>>Besides that, once SvFAKE what happens if users try to modify it?
>
>
> SvFAKE | SvREADONLY is what perl uses to mark shared sv's, so I thought
> I could borrow those flags here. Unfortunately that doesn't seem to
> work very well with the tiehash table API (t/modperl/env generates
> lots of "Attempt to free non-existent shared string ..." warnings).
And isn't all this stuff quite new in 5.8, and doesn't really work with 5.6?
Still, once user tries to modify the data, what happens to meta-data? When COW
is performed, does it copy the whole thing or just whatever was pointed by SvCUR?
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [rfc] APR::Table & polymorphic values
Posted by Joe Schaefer <jo...@sunstarsys.com>.
Stas Bekman <st...@stason.org> writes:
[...]
> http://archive.develooper.com/perl5-porters@perl.org/msg96172.html
Thanks for the link!
> I haven't read the whole thread, but saw Tim Bunce suggesting later in
> this thread not to use SvFAKE:
> http://archive.develooper.com/perl5-porters@perl.org/msg95102.html
Using SvLEN = 0 seems to be the cleanest solution. Unfortunately,
setting SvLEN = 0 prevents correct sv_2iv conversion w/ 5.8. The bug
is caused by testing for SvLEN > 0 instead of SvCUR > 0 in Perl_sv_2iv.
I don't know if bleadperl has fixed this, but it's probably behaved
this way for quite a while.
> May be all you need is to manipulate the pointers. So in normal case
> SVPV will point to the first char, in apreq case it'll point to the
> char the real data starts from. Am I shooting far off the target?
In the normal case:
sv = newSVpv(data,0);
perl mallocs space for a copy and does a strcpy(SvPVX(sv), data).
That's not good enough for apreq- we really need SvPVX(sv) == data
in order to get at the metadata in front of &data[0].
> Besides that, once SvFAKE what happens if users try to modify it?
SvFAKE | SvREADONLY is what perl uses to mark shared sv's, so I thought
I could borrow those flags here. Unfortunately that doesn't seem to
work very well with the tiehash table API (t/modperl/env generates
lots of "Attempt to free non-existent shared string ..." warnings).
--
Joe Schaefer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [rfc] APR::Table & polymorphic values
Posted by Stas Bekman <st...@stason.org>.
Joe Schaefer wrote:
> Stas Bekman <st...@stason.org> writes:
>
> [...]
>
>
>>The question is whether this hit (e.g. magic retrieval) is wanted for
>>the general case. APR::Table is
>
>
> OK- how about using SvFAKE for the APR::Table return values instead?
> That way everybody wins- APR::Tables uses copy-on-write for the
> returned SV's (which is faster than doing newSVpv), and apreq can
> get the metadata back by interrogating those SV's (since they're
> not copies, so SvPVX(sv) still points to the original data[1]).
I've checked that it's in 5.6.0 (The minimum that we need to support), though
I'm not sure if it's working right. I haven't used it myself, but there is a
lot of info in the thread starting from this post:
http://archive.develooper.com/perl5-porters@perl.org/msg96172.html
I haven't read the whole thread, but saw Tim Bunce suggesting later in this
thread not to use SvFAKE:
http://archive.develooper.com/perl5-porters@perl.org/msg95102.html
May be all you need is to manipulate the pointers. So in normal case SVPV will
point to the first char, in apreq case it'll point to the char the real data
starts from. Am I shooting far off the target?
Besides that, once SvFAKE what happens if users try to modify it?
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [rfc] APR::Table & polymorphic values
Posted by Joe Schaefer <jo...@sunstarsys.com>.
Stas Bekman <st...@stason.org> writes:
[...]
> The question is whether this hit (e.g. magic retrieval) is wanted for
> the general case. APR::Table is
OK- how about using SvFAKE for the APR::Table return values instead?
That way everybody wins- APR::Tables uses copy-on-write for the
returned SV's (which is faster than doing newSVpv), and apreq can
get the metadata back by interrogating those SV's (since they're
not copies, so SvPVX(sv) still points to the original data[1]).
--
Joe Schaefer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [rfc] APR::Table & polymorphic values
Posted by Joe Schaefer <jo...@sunstarsys.com>.
Stas Bekman <st...@stason.org> writes:
[...]
> but you don't store apreq_value_t in the apr_table, you store only
> apreq_value_t->data. Do I miss some trick with data[1], as explained
> here: http://www.eskimo.com/~scs/C-faq/q2.6.html ?
Yup, that's the trick.
--
Joe Schaefer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [rfc] APR::Table & polymorphic values
Posted by Stas Bekman <st...@stason.org>.
Joe Schaefer wrote:
> Stas Bekman <st...@stason.org> writes:
>
>
>>Joe, what meta data are you talking about? Is it that new
>>things that you've merged into apr tables?
>
>
> Yup.
Is it committed already? I must have been looking at the old implementation then.
> Whenever someone does a table lookup in apreq-2, the
> returned character string is always the last attribute of an
> apreq_value_t struct, (which is also the last attribute of
> either an apreq_param_t or an apreq_cookie_t struct).
>
> For apreq-2, it would be nice if the c2perl conversion glue
> was able to take this metadata into account.
>>Can you please point me to the source you are talking about?
>
>
> See httpd-apreq-2/src/apreq.h for the definition of apreq_value_t .
> All the parsers in apreq_parsers.c add param->v.data to the
> request body table, the cookie parser in apreq_cookie.c adds
> cookie->v.data to the jar->cookies table.
but you don't store apreq_value_t in the apr_table, you store only
apreq_value_t->data. Do I miss some trick with data[1], as explained here:
http://www.eskimo.com/~scs/C-faq/q2.6.html ?
>>Also how about using a sub-class of APR::Table which
>>overrides get/set to read/write this meta data?
>
>
> Yes, that's a very good approach, and may be what
> we wind up needing for apreq-2.
>
>
>>It's probably more maitenance overhead, but will be faster
>>code-wise, no?
>
>
> Yes to the maintenance, since a lot of the APR::Table
> XSUBs would need overriding. Dunno about the performance
> question- it's just a few extra pointer dereferences and
> another C-function call, which may be faster than perl's
> OO dispatch.
The question is whether this hit (e.g. magic retrieval) is wanted for the
general case. APR::Table is
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [rfc] APR::Table & polymorphic values
Posted by Joe Schaefer <jo...@sunstarsys.com>.
Stas Bekman <st...@stason.org> writes:
> Joe, what meta data are you talking about? Is it that new
> things that you've merged into apr tables?
Yup. Whenever someone does a table lookup in apreq-2, the
returned character string is always the last attribute of an
apreq_value_t struct, (which is also the last attribute of
either an apreq_param_t or an apreq_cookie_t struct).
For apreq-2, it would be nice if the c2perl conversion glue
was able to take this metadata into account.
> Can you please point me to the source you are talking about?
See httpd-apreq-2/src/apreq.h for the definition of apreq_value_t .
All the parsers in apreq_parsers.c add param->v.data to the
request body table, the cookie parser in apreq_cookie.c adds
cookie->v.data to the jar->cookies table.
> Also how about using a sub-class of APR::Table which
> overrides get/set to read/write this meta data?
Yes, that's a very good approach, and may be what
we wind up needing for apreq-2.
> It's probably more maitenance overhead, but will be faster
> code-wise, no?
Yes to the maintenance, since a lot of the APR::Table
XSUBs would need overriding. Dunno about the performance
question- it's just a few extra pointer dereferences and
another C-function call, which may be faster than perl's
OO dispatch.
--
Joe Schaefer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [rfc] APR::Table & polymorphic values
Posted by Stas Bekman <st...@stason.org>.
Joe Schaefer wrote:
> I wanted to get some feedback on how to
> add APR::Table support for apreq-2's "polymorphic"
> value strings.
>
> The issue is that apreq-2 uses apr_tables,
> but adds additional metadata to each table
> value. The current APR::Table glue uses
>
> newSVpv((char *)val, 0)
>
> to convert the char* to an SV*. That's not
> totally adequate for apreq-2, since it forgets
> about the metadata stored in front of val.
>
> For apreq-2, what I'd like to have happen
> is for a callback to do the type conversion
> instead of newSVpv. The callback would be
> attached to the APR::Table object, so the
> XS glue for APR::Table would need a
> patch to use the callback in place of the
> newSVpv calls.
>
> It looks to me like mg_ptr is NULL for the
> TIEHASH magic associated to APR::Table
> objects. Are there any objections to a patch
> that uses a callback stored there?
>
> Can anyone recommend a better approach?
Joe, what meta data are you talking about? Is it that new things that you've
merged into apr tables? Can you please point me to the source you are talking
about?
Also how about using a sub-class of APR::Table which overrides get/set to
read/write this meta data? It's probably more maitenance overhead, but will be
faster code-wise, no?
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org