You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@perl.apache.org by Stas Bekman <st...@stason.org> on 2004/12/18 18:08:26 UTC
[mp2] pool object dependant methods insanity
[the previous thread was too long, starting a new one]
*** what methods need to be protected from dependent blocks going out
*** of scope ***
the following looks only at the first level (i.e. methods taking a
pool object):
-------------------------------------------------------------
*** returning objects ***
APR::Brigade:
- mpxs_apr_brigade_create
APR::Bucket:
- apr_bucket_alloc_create
- mpxs_APR__Bucket_setaside
APR::Finfo:
- mpxs_APR__Finfo_stat
APR::IpSubnet:
- mpxs_apr_ipsubnet_create
APR::Pool:
- mpxs_apr_pool_create (not sure about this one)
Apache::RequestUtil:
- mpxs_Apache__RequestRec_new
APR::Table:
- apr_table_copy
- apr_table_overlay
- apr_table_make
APR::ThreadMutex
- mpxs_apr_thread_mutex_create
*** returning strings ***
Apache::ServerUtil
- mpxs_Apache__ServerUtil_server_root_relative (once this is
supported, we no longer need to double copy the string)
Apache::URI:
- ap_construct_server
- ap_construct_url
APR::URI
- mpxs_apr_uri_parse
Apache::Util
- ap_ht_time
- escape_path
-------------------------------------------------------------
as you can see most of these already use perl wrappers, so I can't quite
see how this can be automated.
The biggest problem, that this is not enough. If the returned object (by
one of the above methods) is used to create other objects/SVs,
the latter should be made dependant on the former too. e.g.:
APR::Bucket::eos_create(APR::Bucket::alloc_create(APR::Pool->new)
the bucket allocator goes out of scope, before eos bucket, potentially
wiped out eos internals while there are still in use.
so if we try to identify all those, I think we are going to come up with a
huge list of methods to fix. And most likely there will be 3rd and 4th
levels as well.
Besides the problem of writing all this code, I think it may add a
significant memory and CPU overhead.
Ideas for a better solution? We certainly can't leave things like there
are, for example at the moment this perfectly well written code:
$pool = APR::Pool->new();
my $mutex = APR::ThreadMutex->new($pool);
will crash with apr built "--enable-pool-debug
CPPFLAGS="-DAPR_BUCKET_DEBUG" (or in a real world application), since
$pool goes out of scope before the mutex is destroyed.
--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [mp2] pool object dependant methods insanity
Posted by Stas Bekman <st...@stason.org>.
Ok, here is the first batch. All the relevant APR::Table methods were
pretty similar, so I was able to use macros for all 3. I've first tests
and I saw all 3 methods broken. Only make() was segfaulting, the rest were
giving corrupted data. - I have found a nice way to get the data
corrupted, by allocating a new pool and overwriting the data, making an
old table corrupted. (but with this patch all tests pass).
I guess I should start committing those, and then refactor any re-usable
macros, like MPXS_POOL_MAGIC(obj)
Index: todo/release
===================================================================
--- todo/release (revision 123029)
+++ todo/release (working copy)
@@ -69,9 +69,9 @@
- mpxs_Apache__RequestRec_new
APR::Table:
- - apr_table_copy
- - apr_table_overlay
- - apr_table_make
+ V apr_table_copy
+ V apr_table_overlay
+ V apr_table_make
APR::ThreadMutex
- mpxs_apr_thread_mutex_create
Index: src/modules/perl/modperl_common_util.c
===================================================================
--- src/modules/perl/modperl_common_util.c (revision 123029)
+++ src/modules/perl/modperl_common_util.c (working copy)
@@ -69,7 +69,7 @@
/* Prefetch magic requires perl 5.8 */
#if ((PERL_REVISION == 5) && (PERL_VERSION >= 8))
- sv_magic(hv, NULL, PERL_MAGIC_ext, Nullch, -1);
+ sv_magicext(hv, NULL, PERL_MAGIC_ext, NULL, Nullch, -1);
SvMAGIC(hv)->mg_virtual = (MGVTBL *)&modperl_table_magic_prefetch;
SvMAGIC(hv)->mg_flags |= MGf_COPY;
Index: xs/maps/apr_functions.map
===================================================================
--- xs/maps/apr_functions.map (revision 123029)
+++ xs/maps/apr_functions.map (working copy)
@@ -244,10 +244,13 @@
MODULE=APR::Table
apr_table_clear
- apr_table_copy | | t, p
- apr_table_make
+~ apr_table_copy
+ mpxs_APR__Table_copy
+~apr_table_make
+ mpxs_APR__Table_make
apr_table_overlap
- apr_table_overlay | | base, overlay, p
+~apr_table_overlay
+ mpxs_APR__Table_overlay
apr_table_compress
apr_table_add
-apr_table_addn
Index: xs/APR/Table/APR__Table.h
===================================================================
--- xs/APR/Table/APR__Table.h (revision 123029)
+++ xs/APR/Table/APR__Table.h (working copy)
@@ -17,6 +17,42 @@
#define mpxs_APR__Table_DELETE apr_table_unset
#define mpxs_APR__Table_CLEAR apr_table_clear
+#if ((PERL_REVISION == 5) && (PERL_VERSION >= 8))
+ /* modperl_hash_tie already attached another _ext magic under
+ * 5.8+, so must use sv_magicext to have two magics with the
+ * type */
+#define MPXS_POOL_MAGIC(obj) \
+ sv_magicext(SvRV(obj), p_sv, PERL_MAGIC_ext, NULL, Nullch, -1)
+#else
+#define MPXS_POOL_MAGIC(obj) \
+ sv_magic(SvRV(obj), p_sv, PERL_MAGIC_ext, Nullch, -1)
+#endif
+
+#define MPXS_DO_TABLE_N_MAGIC_RETURN(call) \
+ apr_pool_t *p = mp_xs_sv2_APR__Pool(p_sv); \
+ apr_table_t *t = call; \
+ SV *t_sv = modperl_hash_tie(aTHX_ "APR::Table", Nullsv, t); \
+ MPXS_POOL_MAGIC(t_sv); \
+ return t_sv;
+
+static MP_INLINE SV *mpxs_APR__Table_make(pTHX_ SV *p_sv, int nelts)
+{
+ MPXS_DO_TABLE_N_MAGIC_RETURN(apr_table_make(p, nelts));
+}
+
+
+static MP_INLINE SV *mpxs_APR__Table_copy(pTHX_ apr_table_t *base, SV *p_sv)
+{
+ MPXS_DO_TABLE_N_MAGIC_RETURN(apr_table_copy(p, base));
+}
+
+static MP_INLINE SV *mpxs_APR__Table_overlay(pTHX_ apr_table_t *base,
+ apr_table_t *overlay, SV *p_sv)
+{
+ MPXS_DO_TABLE_N_MAGIC_RETURN(apr_table_overlay(p, overlay, base));
+}
+
+
typedef struct {
SV *cv;
apr_hash_t *filter;
Index: xs/tables/current/ModPerl/FunctionTable.pm
===================================================================
--- xs/tables/current/ModPerl/FunctionTable.pm (revision 123029)
+++ xs/tables/current/ModPerl/FunctionTable.pm (working copy)
@@ -2,7 +2,7 @@
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
# ! WARNING: generated by ModPerl::ParseSource/0.01
-# ! Fri Dec 17 21:23:11 2004
+# ! Tue Dec 21 22:12:37 2004
# ! do NOT edit, any changes will be lost !
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
@@ -5921,6 +5921,76 @@
]
},
{
+ 'return_type' => 'SV *',
+ 'name' => 'mpxs_APR__Table_copy',
+ 'attr' => [
+ 'static',
+ '__inline__'
+ ],
+ 'args' => [
+ {
+ 'type' => 'PerlInterpreter *',
+ 'name' => 'my_perl'
+ },
+ {
+ 'type' => 'apr_table_t *',
+ 'name' => 'base'
+ },
+ {
+ 'type' => 'SV *',
+ 'name' => 'p_sv'
+ }
+ ]
+ },
+ {
+ 'return_type' => 'SV *',
+ 'name' => 'mpxs_APR__Table_make',
+ 'attr' => [
+ 'static',
+ '__inline__'
+ ],
+ 'args' => [
+ {
+ 'type' => 'PerlInterpreter *',
+ 'name' => 'my_perl'
+ },
+ {
+ 'type' => 'SV *',
+ 'name' => 'p_sv'
+ },
+ {
+ 'type' => 'int',
+ 'name' => 'nelts'
+ }
+ ]
+ },
+ {
+ 'return_type' => 'SV *',
+ 'name' => 'mpxs_APR__Table_overlay',
+ 'attr' => [
+ 'static',
+ '__inline__'
+ ],
+ 'args' => [
+ {
+ 'type' => 'PerlInterpreter *',
+ 'name' => 'my_perl'
+ },
+ {
+ 'type' => 'apr_table_t *',
+ 'name' => 'base'
+ },
+ {
+ 'type' => 'apr_table_t *',
+ 'name' => 'overlay'
+ },
+ {
+ 'type' => 'SV *',
+ 'name' => 'p_sv'
+ }
+ ]
+ },
+ {
'return_type' => 'char *',
'name' => 'mpxs_APR__URI_port',
'args' => [
Index: t/lib/TestAPRlib/table.pm
===================================================================
--- t/lib/TestAPRlib/table.pm (revision 123029)
+++ t/lib/TestAPRlib/table.pm (working copy)
@@ -17,7 +17,7 @@
our $filter_count;
sub num_of_tests {
- my $tests = 50;
+ my $tests = 56;
# tied hash values() for a table w/ multiple values for the same
# key
@@ -295,6 +295,59 @@
ok t_cmp($foo[0], 'one, two, three');
ok t_cmp($bar[0], 'beer');
}
+
+
+ # temp pool objects.
+ # testing here that the temp pool object doesn't go out of scope
+ # before the object based on it was freed. the following tests
+ # were previously segfaulting when using apr1/httpd2.1 built w/
+ # --enable-pool-debug CPPFLAGS="-DAPR_BUCKET_DEBUG",
+ # the affected methods are:
+ # - make
+ # - copy
+ # - overlay
+ {
+ my $table = APR::Table::make(APR::Pool->new, 10);
+ $table->set($_ => $_) for 1..20;
+ ok t_cmp $table->get(20), 20, "no segfault";
+
+ my $table_copy = $table->copy(APR::Pool->new);
+ {
+ # verify that the temp pool used to create $table_copy was
+ # not freed, by allocating a new table to fill with a
+ # different data. if that former pool was freed
+ # $table_copy will now contain bogus data (and may
+ # segfault)
+ my $table = APR::Table::make(APR::Pool->new, 50);
+ $table->set($_ => $_) for 'a'..'z';
+ ok t_cmp $table->get('z'), 'z', "helper test";
+
+ }
+ ok t_cmp $table_copy->get(20), 20, "no segfault/valid data";
+
+ my $table2 = APR::Table::make(APR::Pool->new, 1);
+ $table2->set($_**2 => $_**2) for 1..20;
+ my $overlay = $table_copy->overlay($table2, APR::Pool->new);
+ {
+ # see the comment for above's:
+ # $table_copy = $table->copy(APR::Pool->new);
+ my $table = APR::Table::make(APR::Pool->new, 50);
+ $table->set($_ => $_) for 'aa'..'za';
+ ok t_cmp $table->get('za'), 'za', "helper test";
+
+ }
+ ok t_cmp $overlay->get(20), 20, "no segfault/valid data";
+ }
+ {
+ {
+ my $p = APR::Pool->new;
+ $p->cleanup_register(sub { "whatever" });
+ $table = APR::Table::make($p, 10)
+ };
+ $table->set(a => 5);
+ ok t_cmp $table->get("a"), 5, "no segfault";
+ }
+
}
sub my_filter {
--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [mp2] pool object dependant methods insanity
Posted by Stas Bekman <st...@stason.org>.
Stas Bekman wrote:
> while I can easily reproduce the problem with
> APR::Bucket::alloc_create(APR::Pool->new);
> the case with setaside is not clear. I've tried the following:
> + # setaside + alloc_create on out-of-scope pools
> + {
> + my $data = "A" x 10;
> + my $orig = $data;
> + #$ba = APR::Bucket::alloc_create(APR::Pool->new);
> + my $b = APR::Bucket->new($ba, $data);
> + my $status = $b->setaside(APR::Pool->new);
> + ok t_cmp $status, 0, "setaside status";
>
> + # try to overwrite the temp pool data
> + require APR::Table;
> + my $table = APR::Table::make(APR::Pool->new, 50);
> + $table->set($_ => $_) for 'aa'..'za';
> +
> + # now test that we are still OK
> + $b->read(my $read);
> + ok !t_cmp($read, $data,
> + "data inside the setaside bucket is unaffected by " .
> + "changes to the Perl variable it's created from");
> + $b->destroy;
> + }
> +
> APR::Bucket::alloc_destroy($ba);
>
> }
>
> but it doesn't break. Further looking at the source code,
>
> $b->setaside(APR::Pool->new);
>
> doesn't replace $b to a new bucket allocation. I think this is why I
> can't reproduce the problem.
That wasn't correct.
Here is the real reason why the problem can't be created with a temp pool:
Index: xs/APR/Bucket/APR__Bucket.h
===================================================================
--- xs/APR/Bucket/APR__Bucket.h (revision 123827)
+++ xs/APR/Bucket/APR__Bucket.h (working copy)
@@ -112,8 +112,11 @@
if (GIMME_V == G_VOID && rc != APR_SUCCESS) {
modperl_croak(aTHX_ rc, "APR::Bucket::setaside");
}
-
- //mpxs_add_pool_magic(b_sv, p_sv);
-
+
+ /* No need to call mpxs_add_pool_magic(b_sv, p_sv); since
+ * pool_bucket_cleanup is called by apr_bucket_pool_make (called
+ * by modperl_bucket_sv_setaside) if the pool goes out of scope,
+ * copying the data to the heap.
+ */
return rc;
}
so setaside does not need a special handling. I live the test that would
have failed if it wasn't for apr internal's backup. so if it ever removed
the test should catch it.
--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: Rename APR::Bucket::alloc_create
Posted by Stas Bekman <st...@stason.org>.
Joe Schaefer wrote:
> Stas Bekman <st...@stason.org> writes:
>
>
>>Joe Schaefer wrote:
>>
>>>Stas Bekman <st...@stason.org> writes:
>>>
>>>
>>>>while I can easily reproduce the problem with
>>>>APR::Bucket::alloc_create(APR::Pool->new);
>>>
>>>Would it be possible to rename this sub APR::BucketAlloc::create?
>>>When I first named it that, I wasn't aware of the APR::BucketAlloc
>>>package.
>>
>>doh! I think you are right. I've missed that bit. I hope our users
>>will excuse us.
>
>
> +1. The holidays have left me a few weeks behind modperl's
> trunk. I'll commit the change once I'm caught up (which
> may take a few more days), but feel free to take care of
> it for me if you'd rather not wait for me to catch up.
I hope I'll manage to do that shortly. I plan to polish the protocols.pod
as it's a bit out of date, so this will be a prerequisite.
>>also it looks that ba has the pool instead, so it was probably
>>a waste to demand the pool object when $ba is passed anyway.
>
>
> The pool argument to APR::BucketAlloc::create required,
> I think. If there's an APR::BucketAlloc::pool accessor,
> it's just retrieving a pointer to the pool that first
> created it.
Agreed. It might still work most of the time, but may be it's not what the
user want (though the extra pool argument could have been made optional).
--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: Rename APR::Bucket::alloc_create
Posted by Joe Schaefer <jo...@sunstarsys.com>.
Stas Bekman <st...@stason.org> writes:
> Joe Schaefer wrote:
>> Stas Bekman <st...@stason.org> writes:
>>
>>>while I can easily reproduce the problem with
>>>APR::Bucket::alloc_create(APR::Pool->new);
>> Would it be possible to rename this sub APR::BucketAlloc::create?
>> When I first named it that, I wasn't aware of the APR::BucketAlloc
>> package.
>
> doh! I think you are right. I've missed that bit. I hope our users
> will excuse us.
+1. The holidays have left me a few weeks behind modperl's
trunk. I'll commit the change once I'm caught up (which
may take a few more days), but feel free to take care of
it for me if you'd rather not wait for me to catch up.
> also it looks that ba has the pool instead, so it was probably
> a waste to demand the pool object when $ba is passed anyway.
The pool argument to APR::BucketAlloc::create required,
I think. If there's an APR::BucketAlloc::pool accessor,
it's just retrieving a pointer to the pool that first
created it.
--
Joe Schaefer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: Rename APR::Bucket::alloc_create (was Re: [mp2] pool object dependant
methods insanity)
Posted by Stas Bekman <st...@stason.org>.
Joe Schaefer wrote:
> Stas Bekman <st...@stason.org> writes:
>
>
>>while I can easily reproduce the problem with
>>APR::Bucket::alloc_create(APR::Pool->new);
>
>
> Would it be possible to rename this sub APR::BucketAlloc::create?
> When I first named it that, I wasn't aware of the APR::BucketAlloc
> package.
doh! I think you are right. I've missed that bit. I hope our users will
excuse us.
also it looks that ba has the pool instead, so it was probably a waste to
demand the pool object when $ba is passed anyway.
--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Rename APR::Bucket::alloc_create (was Re: [mp2] pool object
dependant methods insanity)
Posted by Joe Schaefer <jo...@sunstarsys.com>.
Stas Bekman <st...@stason.org> writes:
> while I can easily reproduce the problem with
> APR::Bucket::alloc_create(APR::Pool->new);
Would it be possible to rename this sub APR::BucketAlloc::create?
When I first named it that, I wasn't aware of the APR::BucketAlloc
package.
--
Joe Schaefer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [mp2] pool object dependant methods insanity
Posted by Stas Bekman <st...@stason.org>.
while I can easily reproduce the problem with
APR::Bucket::alloc_create(APR::Pool->new);
the case with setaside is not clear. I've tried the following:
Index: t/lib/TestAPRlib/bucket.pm
===================================================================
--- t/lib/TestAPRlib/bucket.pm (revision 123293)
+++ t/lib/TestAPRlib/bucket.pm (working copy)
@@ -14,7 +14,7 @@
use APR::BucketType ();
sub num_of_tests {
- return 18;
+ return 20;
}
sub test {
@@ -160,12 +160,33 @@
$data =~ s/^..../BBBB/;
$b->read(my $read);
ok !t_cmp($read, $data,
- "data inside the setaside bucket is uaffected by " .
+ "data inside the setaside bucket is unaffected by " .
"changes to the Perl variable it's created from");
$b->destroy;
}
+ # setaside + alloc_create on out-of-scope pools
+ {
+ my $data = "A" x 10;
+ my $orig = $data;
+ #$ba = APR::Bucket::alloc_create(APR::Pool->new);
+ my $b = APR::Bucket->new($ba, $data);
+ my $status = $b->setaside(APR::Pool->new);
+ ok t_cmp $status, 0, "setaside status";
+ # try to overwrite the temp pool data
+ require APR::Table;
+ my $table = APR::Table::make(APR::Pool->new, 50);
+ $table->set($_ => $_) for 'aa'..'za';
+
+ # now test that we are still OK
+ $b->read(my $read);
+ ok !t_cmp($read, $data,
+ "data inside the setaside bucket is unaffected by " .
+ "changes to the Perl variable it's created from");
+ $b->destroy;
+ }
+
APR::Bucket::alloc_destroy($ba);
}
but it doesn't break. Further looking at the source code,
$b->setaside(APR::Pool->new);
doesn't replace $b to a new bucket allocation. I think this is why I can't
reproduce the problem.
Also it looks like we aren't handling scopes:
modperl_bucket.c:
/* XXX: need to deal with PerlInterpScope */
#ifdef USE_ITHREADS
svbucket->perl = aTHX;
#endif
but if we handle them we may need to tie an enourmous amount of perl
interpreters just to deal with each bucket. That sounds a like a bad
design flow. (which affects pool cleanups as well).
--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [mp2] pool object dependant methods insanity
Posted by Stas Bekman <st...@stason.org>.
Joe Schaefer wrote:
> Stas Bekman <st...@stason.org> writes:
>
>
>>Joe Schaefer wrote:
>>
>>> $bucket->setaside($pool) means
>
>
>>>the apr_bucket_t attached to this APR::Bucket object needs to
>>>survive as long as the underlying apr_pool_t. However, if
>>>that apr_pool_t is owned by some APR::Pool object, that object
>>>should [*] survive at least as long as the APR::Bucket object does.
>
>
> [...]
>
>
>>So what do we hang the pool on? $bucket?
>
>
> +1.
>
>
>>That doesn't sound right, since the perl object may go away (thus
>>destroying the pool if noone else references it), but the underlying
>>bucket can't see that. so it's probably not enough, no?
>
>
> It is enough, if you're sticking to the text above.
> The pool is going to die at some point; the application
> (not mp2) has a bug if the bucket is used beyond that point.
Right. Thanks for that idea, Joe.
BTW, I've found a nice way to write tests for this dependency issue (not
only this particular method). Instead of trying to hope for a random
segfault, I simply create a new pool and use it to create some huge apr
table or something and then verify that the object created with the
previous temp pool is still valid, if the pool was freed too early, it'll
be corrupred, 100% certainly giving either wrong data or segfaults,
regardless whether --enable-pool-debug is in effect.
--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [mp2] pool object dependant methods insanity
Posted by Joe Schaefer <jo...@sunstarsys.com>.
Stas Bekman <st...@stason.org> writes:
> Joe Schaefer wrote:
>> $bucket->setaside($pool) means
>> the apr_bucket_t attached to this APR::Bucket object needs to
>> survive as long as the underlying apr_pool_t. However, if
>> that apr_pool_t is owned by some APR::Pool object, that object
>> should [*] survive at least as long as the APR::Bucket object does.
[...]
>
> So what do we hang the pool on? $bucket?
+1.
> That doesn't sound right, since the perl object may go away (thus
> destroying the pool if noone else references it), but the underlying
> bucket can't see that. so it's probably not enough, no?
It is enough, if you're sticking to the text above.
The pool is going to die at some point; the application
(not mp2) has a bug if the bucket is used beyond that point.
--
Joe Schaefer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [mp2] pool object dependant methods insanity
Posted by Stas Bekman <st...@stason.org>.
Joe Schaefer wrote:
> Stas Bekman <st...@stason.org> writes:
>
>
>>Joe Schaefer wrote:
>
>
> [...]
>
>
>>> $bucket->setaside($pool);
>>>that means "this bucket may need to live as long as this pool does".
>>>If we change the semantics to "this pool needs to live as long as
>>>this bucket does", IMO we've made a mistake. If you agree, then I
>>>think mpxs_APR__Bucket_setaside can be removed from the list.
>>
>>I haven't analized the actual methods yet, other than checking that
>>they return an object.
>>
>>I agree that this is a tricky issue with this method. But if you don't
>>tie the pool along with the setaside data, how is it going to be
>>preserved if the pool will be gone? the setaside data will become
>>corrupted, no?
>
>
> Sure, that could happen. Thinking about it a bit more, perhaps your
> first guess is right. Restating the semantics with a cleaner
> separation of (Perl) variable from (C) value:
>
> $bucket->setaside($pool) means
>
> the apr_bucket_t attached to this APR::Bucket object needs to
> survive as long as the underlying apr_pool_t. However, if
> that apr_pool_t is owned by some APR::Pool object, that object
> should [*] survive at least as long as the APR::Bucket object does.
>
> Agreed? If so, then setaside() does belong on the list.
>
> [*] - "should", not "must", because people can always explicitly
> destroy the pool.
So what do we hang the pool on? $bucket? That doesn't sound right, since
the perl object may go away (thus destroying the pool if noone else
references it), but the underlying bucket can't see that. so it's probably
not enough, no? Should we do it anyway?
--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [mp2] pool object dependant methods insanity
Posted by Stas Bekman <st...@stason.org>.
Joe Schaefer wrote:
>>> $bucket->setaside($pool);
>>>that means "this bucket may need to live as long as this pool does".
>>>If we change the semantics to "this pool needs to live as long as
>>>this bucket does", IMO we've made a mistake. If you agree, then I
>>>think mpxs_APR__Bucket_setaside can be removed from the list.
>>
>>I haven't analized the actual methods yet, other than checking that
>>they return an object.
>>
>>I agree that this is a tricky issue with this method. But if you don't
>>tie the pool along with the setaside data, how is it going to be
>>preserved if the pool will be gone? the setaside data will become
>>corrupted, no?
>
>
> Sure, that could happen. Thinking about it a bit more, perhaps your
> first guess is right. Restating the semantics with a cleaner
> separation of (Perl) variable from (C) value:
>
> $bucket->setaside($pool) means
>
> the apr_bucket_t attached to this APR::Bucket object needs to
> survive as long as the underlying apr_pool_t. However, if
> that apr_pool_t is owned by some APR::Pool object, that object
> should [*] survive at least as long as the APR::Bucket object does.
>
> Agreed? If so, then setaside() does belong on the list.
>
> [*] - "should", not "must", because people can always explicitly
> destroy the pool.
+1
--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [mp2] pool object dependant methods insanity
Posted by Joe Schaefer <jo...@sunstarsys.com>.
Stas Bekman <st...@stason.org> writes:
> Joe Schaefer wrote:
[...]
>> $bucket->setaside($pool);
>> that means "this bucket may need to live as long as this pool does".
>> If we change the semantics to "this pool needs to live as long as
>> this bucket does", IMO we've made a mistake. If you agree, then I
>> think mpxs_APR__Bucket_setaside can be removed from the list.
>
> I haven't analized the actual methods yet, other than checking that
> they return an object.
>
> I agree that this is a tricky issue with this method. But if you don't
> tie the pool along with the setaside data, how is it going to be
> preserved if the pool will be gone? the setaside data will become
> corrupted, no?
Sure, that could happen. Thinking about it a bit more, perhaps your
first guess is right. Restating the semantics with a cleaner
separation of (Perl) variable from (C) value:
$bucket->setaside($pool) means
the apr_bucket_t attached to this APR::Bucket object needs to
survive as long as the underlying apr_pool_t. However, if
that apr_pool_t is owned by some APR::Pool object, that object
should [*] survive at least as long as the APR::Bucket object does.
Agreed? If so, then setaside() does belong on the list.
[*] - "should", not "must", because people can always explicitly
destroy the pool.
--
Joe Schaefer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [mp2] pool object dependant methods insanity
Posted by Stas Bekman <st...@stason.org>.
> I haven't analized the actual methods yet, other than checking that they
> return an object.
Sorry, I wanted to say: I haven't analyzed the actual methods yet, other
than checking that they take a pool as an object and return something to
the user.
--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [mp2] pool object dependant methods insanity
Posted by Stas Bekman <st...@stason.org>.
Joe Schaefer wrote:
> Stas Bekman <st...@stason.org> writes:
>
>
>>The things aren't as bad as I thought there are. Here is an update on
>>this issue. The key difference to my previous post is: only objects
>>that have DESTROY method may need to be dealt with.
>
>
> +1, my only comment (for now) on your list is:
>
>
>> - mpxs_APR__Bucket_setaside
>
>
> The semantics for apr_bucket_setaside don't require the
> perl wrapper to be modified, because when someone writes
>
> $bucket->setaside($pool);
>
> that means "this bucket may need to live as long as this pool
> does". If we change the semantics to "this pool needs to
> live as long as this bucket does", IMO we've made a mistake.
> If you agree, then I think mpxs_APR__Bucket_setaside can
> be removed from the list.
I haven't analized the actual methods yet, other than checking that they
return an object.
I agree that this is a tricky issue with this method. But if you don't tie
the pool along with the setaside data, how is it going to be preserved if
the pool will be gone? the setaside data will become corrupted, no?
--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [mp2] pool object dependant methods insanity
Posted by Joe Schaefer <jo...@sunstarsys.com>.
Stas Bekman <st...@stason.org> writes:
> The things aren't as bad as I thought there are. Here is an update on
> this issue. The key difference to my previous post is: only objects
> that have DESTROY method may need to be dealt with.
+1, my only comment (for now) on your list is:
> - mpxs_APR__Bucket_setaside
The semantics for apr_bucket_setaside don't require the
perl wrapper to be modified, because when someone writes
$bucket->setaside($pool);
that means "this bucket may need to live as long as this pool
does". If we change the semantics to "this pool needs to
live as long as this bucket does", IMO we've made a mistake.
If you agree, then I think mpxs_APR__Bucket_setaside can
be removed from the list.
--
Joe Schaefer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [mp2] pool object dependant methods insanity
Posted by Stas Bekman <st...@stason.org>.
The things aren't as bad as I thought there are. Here is an update on this
issue. The key difference to my previous post is: only objects that have
DESTROY method may need to be dealt with. Below is the analysis of the
relevant classes and methods.
* we need to deal with a situation where an object is used to
construct another object, but it's then auto-DESTROYed by perl
rendering the object that used it corrupted.
the solution is to make the newly created objects refer to the
underlying object via magic attachment.
only objects using objects that have DESTROY are effected, so for
example in the case of:
APR::Bucket::eos_create(APR::Bucket::alloc_create(APR::Pool->new)
the object returned by eos_create shouldn't be affected, since
alloc_create()'s object doesn't have perl's DESTROY. so only the
pool object is an issue here (i.e. alloc_create needs special
handling)
relevant objects with DESTROY : Apache::SubRequest,
APR::ThreadMutex, APR::UUID, APR::Pool
=================
=== APR::Pool ===
=================
*** returning objects ***
APR::Brigade:
- mpxs_apr_brigade_create
APR::Bucket:
- apr_bucket_alloc_create
- mpxs_APR__Bucket_setaside
APR::Finfo:
- mpxs_APR__Finfo_stat
APR::IpSubnet:
- mpxs_apr_ipsubnet_create
APR::Pool:
- mpxs_apr_pool_create (not sure about this one)
Apache::RequestUtil:
- mpxs_Apache__RequestRec_new
APR::Table:
- apr_table_copy
- apr_table_overlay
- apr_table_make
APR::ThreadMutex
- mpxs_apr_thread_mutex_create
*** returning strings ***
Apache::ServerUtil
- mpxs_Apache__ServerUtil_server_root_relative (once this is
supported, we no longer need to double copy the string)
Apache::URI:
- ap_construct_server
- ap_construct_url
APR::URI
- mpxs_apr_uri_parse
Apache::Util
- ap_ht_time
- escape_path
==========================
=== Apache::SubRequest ===
==========================
no method uses the object in a way related to the issue in hand
=================
=== APR::UUID ===
=================
no method uses the object in a way related to the issue in hand
========================
=== APR::ThreadMutex ===
========================
no method uses the object in a way related to the issue in hand
--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [mp2] pool object dependant methods insanity
Posted by Stas Bekman <st...@stason.org>.
mpxs_Apache__RequestRec_new is another problematic method. it segfaults if
adding a magic:
Index: xs/Apache/RequestUtil/Apache__RequestUtil.h
===================================================================
--- xs/Apache/RequestUtil/Apache__RequestUtil.h (revision 123255)
+++ xs/Apache/RequestUtil/Apache__RequestUtil.h (working copy)
@@ -54,17 +54,21 @@
*/
static MP_INLINE
-request_rec *mpxs_Apache__RequestRec_new(SV *classname,
- conn_rec *c,
- apr_pool_t *base_pool)
+SV *mpxs_Apache__RequestRec_new(pTHX_ SV *classname,
+ conn_rec *c,
+ SV *base_pool_sv)
{
- apr_pool_t *p;
+ apr_pool_t *p, *base_pool;
request_rec *r;
server_rec *s = c->base_server;
-
+ SV *r_sv;
+
/* see: httpd-2.0/server/protocol.c:ap_read_request */
- if (!base_pool) {
+ if (base_pool_sv) {
+ base_pool = mp_xs_sv2_APR__Pool(base_pool_sv);
+ }
+ else {
base_pool = c->pool;
}
@@ -113,7 +117,13 @@
r->assbackwards = 1;
r->protocol = "UNKNOWN";
- return r;
+ r_sv = sv_setref_pv(NEWSV(0, 0), "Apache::RequestRec", (void*)r);
+
+ if (base_pool_sv) {
+ mpxs_add_pool_magic(r_sv, base_pool_sv);
+ }
+
+ return r_sv;
}
static MP_INLINE
I'm also thinking whether we should do that magic decorating only for
custom pools or for native pools as well. I think this should be done only
for custom pools, since native pools destruction timing is not under our
control anyway.
--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [mp2] pool object dependant methods insanity
Posted by Stas Bekman <st...@stason.org>.
> APR::Pool:
> - mpxs_apr_pool_create (not sure about this one)
Joe, any ideas how to solve this tricky one? This is the APR::Pool->new
itself, when called as $p->new:
Index: xs/APR/Pool/APR__Pool.h
===================================================================
--- xs/APR/Pool/APR__Pool.h (revision 123255)
+++ xs/APR/Pool/APR__Pool.h (working copy)
@@ -106,7 +106,7 @@
/* XXX: should we make it a new global tracing category
* MOD_PERL_TRACE=p for tracing pool management? */
-#define MP_POOL_TRACE_DO 0
+#define MP_POOL_TRACE_DO 1
#if MP_POOL_TRACE_DO && defined(MP_TRACE)
#define MP_POOL_TRACE modperl_trace
@@ -198,6 +198,10 @@
MP_POOL_TRACE(MP_FUNC, "sub-pool p: 0x%lx, sv: 0x%lx, rv: 0x%lx",
(unsigned long)child_pool, sv, rv);
+ if (parent_pool) {
+ mpxs_add_pool_magic(rv, parent_pool_obj);
+ }
+
return rv;
}
}
Index: t/lib/TestAPRlib/pool.pm
===================================================================
--- t/lib/TestAPRlib/pool.pm (revision 123255)
+++ t/lib/TestAPRlib/pool.pm (working copy)
@@ -11,7 +11,7 @@
use APR::Table ();
sub num_of_tests {
- return 74;
+ return 75;
}
sub test {
@@ -387,9 +387,13 @@
ok 1;
}
+ # out-of-scope pools
+ {
+ my $sp = APR::Pool->new->new;
+ # the parent temp pool must stick around
+ ok t_cmp(1, ancestry_count($sp), "parent pool is still alive");
+ }
-
-
# other stuff
{
my $p = APR::Pool->new;
It doesn't quite work, since
1) APR::Pool already uses _ext magic (but we can't use: sv_magicext as it
needs to work with 5.6.x).
2) even if I use sv_magicext it segfaults in mg_free in:
#define MP_APR_POOL_SV_DROPS_OWNERSHIP(acct) STMT_START { \
dTHXa(acct->perl); \
mg_free(acct->sv);
ideas? Should we somehow integrate that object as an alternative to just
hanging off a string magic in MP_APR_POOL_SV_TAKES_OWNERSHIP(acct_sv, pool)?
btw, this will be committed soon (but it's already used in the patch above)
Index: xs/modperl_xs_util.h
===================================================================
--- xs/modperl_xs_util.h (revision 123255)
+++ xs/modperl_xs_util.h (working copy)
@@ -104,4 +104,20 @@
MARK++; \
}
+/* several methods need to ensure that the pool that they take as an
+ * object doesn't go out of scope before the object that they return,
+ * since if this happens, the data contained in the later object
+ * becomes corrupted. this macro is used in various xs files where
+ * it's needed */
+#if ((PERL_REVISION == 5) && (PERL_VERSION >= 8))
+ /* modperl_hash_tie already attached another _ext magic under
+ * 5.8+, so must use sv_magicext to have two magics with the
+ * type */
+#define mpxs_add_pool_magic(obj, pool_obj) \
+ sv_magicext(SvRV(obj), pool_obj, PERL_MAGIC_ext, NULL, Nullch, -1)
+#else
+#define mpxs_add_pool_magic(obj) \
+ sv_magic(SvRV(obj), pool_obj, PERL_MAGIC_ext, Nullch, -1)
+#endif
+
#endif /* MODPERL_XS_H */
--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [mp2] pool object dependant methods insanity
Posted by Stas Bekman <st...@stason.org>.
Stas Bekman wrote:
>
>> I still have a problem with APR::Pool->new->new, trying to figure out
>> how to hang APR::Pool magic on APR::Pool objects themselves. At the
>> moment things crash badly, after mg_free call.
>
>
> This case:
>
> APR::Pool->new->new;
>
> was really tricky, as we have a special tricky case here.
special tricky cases are usually really tricky :)
but it seems to be working now. tested with 5.6.1, 5.8.6, prefork and
worker and committed. will run the smoke now to see if we are good.
--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [mp2] pool object dependant methods insanity
Posted by Stas Bekman <st...@stason.org>.
> I still have a problem with APR::Pool->new->new, trying to figure out
> how to hang APR::Pool magic on APR::Pool objects themselves. At the
> moment things crash badly, after mg_free call.
This case:
APR::Pool->new->new;
was really tricky, as we have a special tricky case here. We hang the
parent object's pool on the child, and when the child is getting
destroyed, the parent pool gets destroyed during mg_free, which internally
(via libapr) nukes the child pool, but APR::Pool doesn't know about this
(since it already verified that it has to free the child, but the rug was
pulled after that) and contines freeing the child. So by manually
unwinding the magic and postponing the moment the parent goes out of
scope, this seems to work:
+#define MP_APR_POOL_SV_DROPS_OWNERSHIP_RUN(acct) STMT_START { \
+ MAGIC *mg = mg_find(acct->sv, PERL_MAGIC_ext); \
+ if (mg && mg->mg_obj) { \
+ sv_2mortal(mg->mg_obj); \
+ mg->mg_obj = Nullsv; \
+ mg->mg_flags &= ~MGf_REFCOUNTED; \
+ } \
+ mg_free(acct->sv); \
+ SvIVX(acct->sv) = 0; \
+} STMT_END
This APR::Pool is really a marvel of twisted logic, and is getting more
and more complicated. It just shows how hard it is to mix two unrelated
memory management systems.
--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [mp2] pool object dependant methods insanity
Posted by Stas Bekman <st...@stason.org>.
Joe Schaefer wrote:
> Stas Bekman <st...@stason.org> writes:
>
> [...]
>
>
>>As most of the time internal mp magic doesn't use the mg_obj slot, I'm
>>trying to just to hang the pool object there. that will solve the problem of
>>double magic
>
>
> +1- I used that fact in certain places within apreq2 to avoid it,
> so I think it can work.
Yes, it seems to work fine.
>>(I still can't figure how using perl public API one finds more than
>>one magic of the same type)
>
>
> You can't use mg_find, but you could walk the SvMAGIC linked-list
> manually (it's a public struct). I think people used mg_ptr as
> a tag string to differentiate different ext-magics, but I have not
> actually seen it done in practice.
Manipulating internal structs manually sounds scary, but it's not the
first time modperl does something like that. I just hoped that mg_find
could be used to some iterate over magic chain.
>>and will save some memory too.
>>
>>#define mpxs_add_pool_magic(obj, pool_obj) \
>> if (mpxs_pool_is_custom(SvRV(pool_obj))) { \
>> MAGIC *mg = mg_find(SvRV(obj), PERL_MAGIC_ext); \
>> if (mg) { \
>> if (mg->mg_obj == Nullsv) { \
>> mg->mg_obj = SvREFCNT_inc(pool_obj); \
>> mg->mg_flags |= MGf_REFCOUNTED; \
>> } \
>> else { \
>> Perl_croak(aTHX_ "Fixme: don't know how to " \
>> "handle non-Nullsv magic"); \
>> } \
>> } \
>> else { \
>> mpxs_add_pool_magic_doit(obj, pool_obj); \
>> } \
>> }
>
>
> Looks good!
there was a buglet in it but now fixed.
I still have a problem with APR::Pool->new->new, trying to figure out how
to hang APR::Pool magic on APR::Pool objects themselves. At the moment
things crash badly, after mg_free call.
The only other remaining problem is the one with $bucket->setaside($pool)
which I think has an implementation problem:
http://marc.theaimsgroup.com/?l=apache-modperl-dev&m=110391757729115&w=2
I'd like to hear your thoughts on that one, Joe.
--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [mp2] pool object dependant methods insanity
Posted by Joe Schaefer <jo...@sunstarsys.com>.
Stas Bekman <st...@stason.org> writes:
[...]
> As most of the time internal mp magic doesn't use the mg_obj slot, I'm
> trying to just to hang the pool object there. that will solve the problem of
> double magic
+1- I used that fact in certain places within apreq2 to avoid it,
so I think it can work.
> (I still can't figure how using perl public API one finds more than
> one magic of the same type)
You can't use mg_find, but you could walk the SvMAGIC linked-list
manually (it's a public struct). I think people used mg_ptr as
a tag string to differentiate different ext-magics, but I have not
actually seen it done in practice.
> and will save some memory too.
>
> #define mpxs_add_pool_magic(obj, pool_obj) \
> if (mpxs_pool_is_custom(SvRV(pool_obj))) { \
> MAGIC *mg = mg_find(SvRV(obj), PERL_MAGIC_ext); \
> if (mg) { \
> if (mg->mg_obj == Nullsv) { \
> mg->mg_obj = SvREFCNT_inc(pool_obj); \
> mg->mg_flags |= MGf_REFCOUNTED; \
> } \
> else { \
> Perl_croak(aTHX_ "Fixme: don't know how to " \
> "handle non-Nullsv magic"); \
> } \
> } \
> else { \
> mpxs_add_pool_magic_doit(obj, pool_obj); \
> } \
> }
Looks good!
--
Joe Schaefer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [mp2] pool object dependant methods insanity
Posted by Stas Bekman <st...@stason.org>.
Joe Schaefer wrote:
> Stas Bekman <st...@stason.org> writes:
>
>
>>Joe Schaefer wrote:
>>
>>>Stas Bekman <st...@stason.org> writes:
>>>
>>>
>>>>One more thought on this subject. We should probably create the
>>>>dependency only on custom pools, since doing so on things like
>>>>$r->pool is just a waste of CPU and memory, since the underlying C
>>>>object is not controlled by mp2 in this case.
>>>
>>>+1. Btw, Stas, IIRC 5.6.x's sv_magic allows multiple ext magic,
>>>so theoretically, #ifdef'ing your original 5.6.1 solution
>>>with 5.8.x's sv_magicext should work ok.
>>
>>Really? That's great. so what needs to be figured out is how to
>>resolve the other problems I've posted about in this thread. Mainly
>>it's about the usage of the currently existing magic. (All kind of
>>segfaults happen now)
>
>
>
> Ordering them (I'd guess) so mg_find() never confuses the
> pool's placeholder magic with whatever real ext-magic it's
> expecting. But I suspect you've mucked around enough with
> this and didn't see anything improve. Sorry I can't offer
> more yet. Once I'm caught up, I'll definitely help out here.
As most of the time internal mp magic doesn't use the mg_obj slot, I'm
trying to just to hang the pool object there. that will solve the problem
of double magic (I still can't figure how using perl public API one finds
more than one magic of the same type) and will save some memory too.
#define mpxs_add_pool_magic(obj, pool_obj) \
if (mpxs_pool_is_custom(SvRV(pool_obj))) { \
MAGIC *mg = mg_find(SvRV(obj), PERL_MAGIC_ext); \
if (mg) { \
if (mg->mg_obj == Nullsv) { \
mg->mg_obj = SvREFCNT_inc(pool_obj); \
mg->mg_flags |= MGf_REFCOUNTED; \
} \
else { \
Perl_croak(aTHX_ "Fixme: don't know how to " \
"handle non-Nullsv magic"); \
} \
} \
else { \
mpxs_add_pool_magic_doit(obj, pool_obj); \
} \
}
--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [mp2] pool object dependant methods insanity
Posted by Joe Schaefer <jo...@sunstarsys.com>.
Stas Bekman <st...@stason.org> writes:
> Joe Schaefer wrote:
>> Stas Bekman <st...@stason.org> writes:
>>
>>>One more thought on this subject. We should probably create the
>>>dependency only on custom pools, since doing so on things like
>>>$r->pool is just a waste of CPU and memory, since the underlying C
>>> object is not controlled by mp2 in this case.
>> +1. Btw, Stas, IIRC 5.6.x's sv_magic allows multiple ext magic,
>> so theoretically, #ifdef'ing your original 5.6.1 solution
>> with 5.8.x's sv_magicext should work ok.
>
> Really? That's great. so what needs to be figured out is how to
> resolve the other problems I've posted about in this thread. Mainly
> it's about the usage of the currently existing magic. (All kind of
> segfaults happen now)
Ordering them (I'd guess) so mg_find() never confuses the
pool's placeholder magic with whatever real ext-magic it's
expecting. But I suspect you've mucked around enough with
this and didn't see anything improve. Sorry I can't offer
more yet. Once I'm caught up, I'll definitely help out here.
--
Joe Schaefer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [mp2] pool object dependant methods insanity
Posted by Stas Bekman <st...@stason.org>.
Joe Schaefer wrote:
> Stas Bekman <st...@stason.org> writes:
>
>
>>One more thought on this subject. We should probably create the
>>dependency only on custom pools, since doing so on things like
>>$r->pool is just a waste of CPU and memory, since the underlying C
>>object is not controlled by mp2 in this case.
>
>
> +1. Btw, Stas, IIRC 5.6.x's sv_magic allows multiple ext magic,
> so theoretically, #ifdef'ing your original 5.6.1 solution
> with 5.8.x's sv_magicext should work ok.
Really? That's great. so what needs to be figured out is how to resolve
the other problems I've posted about in this thread. Mainly it's about the
usage of the currently existing magic. (All kind of segfaults happen now)
--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [mp2] pool object dependant methods insanity
Posted by Joe Schaefer <jo...@sunstarsys.com>.
Stas Bekman <st...@stason.org> writes:
> One more thought on this subject. We should probably create the
> dependency only on custom pools, since doing so on things like
> $r->pool is just a waste of CPU and memory, since the underlying C
> object is not controlled by mp2 in this case.
+1. Btw, Stas, IIRC 5.6.x's sv_magic allows multiple ext magic,
so theoretically, #ifdef'ing your original 5.6.1 solution
with 5.8.x's sv_magicext should work ok.
--
Joe Schaefer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: [mp2] pool object dependant methods insanity
Posted by Stas Bekman <st...@stason.org>.
One more thought on this subject. We should probably create the dependency
only on custom pools, since doing so on things like $r->pool is just a
waste of CPU and memory, since the underlying C object is not controlled
by mp2 in this case.
--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org