You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Amir Caspi <ce...@3phase.com> on 2019/04/08 21:41:20 UTC
uninitialized value $( in Util.pm line 1595
Kevin et al,
I am getting errors in maillog relating to an uninitialized $( in Util.pm, line 1595:
Mar 24 03:28:35 server spamd[27149]: Use of uninitialized value $( in concatenation (.) or string at /usr/share/perl5/vendor_perl/Mail/SpamAssassin/Util.pm line 1595.
I saw an existing bugzilla (https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7629) for this error occurring on lines 1510 and 1595... it looks like 1510 has been patched but 1595 is still happening. I posted in that BZ but it seems to not have much activity so I thought I'd probe the list to see if others might have an idea how to patch it...
Following the discussion in that BZ, I'm guessing this is happening because the spamc user is a member of multiple groups:
util: setuid: ruid=1000 euid=1000 rgid= 10 190 egid= 10 190
So when line 1595 is trying to find the gid, it's getting something formatted differently than what it wants. When spamc is run for a user who is only a member of a single group, this error doesn't occur, so it seems clear that it's related to being part of multiple groups.
Line 1510 was solved by including a split statement... could that help here, too? Unfortunately I'm not sufficiently expert to figure out the proper syntax and where to include the split.
(This is in CentOS 7, fully updated.)
Cheers and thanks!
--- Amir
Re: uninitialized value $( in Util.pm line 1595
Posted by "@lbutlr" <kr...@kreme.com>.
On 8 Apr 2019, at 16:38, @lbutlr <kr...@kreme.com> wrote:
> On 8 Apr 2019, at 15:41, Amir Caspi <ce...@3phase.com> wrote:
>> I'm guessing this is happening because the spamc user is a member of multiple groups
>
> Is it usually too have a spam user?
spamc user, even.
--
If women wear a pair of pants, a pair of glasses, and a pair of
earrings, why don't they wear a pair of bras?
Re: uninitialized value $( in Util.pm line 1595
Posted by Amir Caspi <ce...@3phase.com>.
On Apr 8, 2019, at 4:38 PM, @lbutlr <kr...@kreme.com> wrote:
> Is it usually too have a spam user?
My system runs SA per-user, using sendmail MTA, procmail LDA "glue" calling spamc with the -u option. So spamc will setuid to the calling user. Any user who is in more than one group will experience this problem.
Cheers.
--- Amir
Re: uninitialized value $( in Util.pm line 1595
Posted by "@lbutlr" <kr...@kreme.com>.
On 8 Apr 2019, at 15:41, Amir Caspi <ce...@3phase.com> wrote:
> I'm guessing this is happening because the spamc user is a member of multiple groups
Is it usually too have a spam user?
--
"He loves Nature in spite of what it did to him." - Forrest Tucker
Re: uninitialized value $( in Util.pm line 1595
Posted by "@lbutlr" <kr...@kreme.com>.
On 10 Apr 2019, at 07:29, @lbutlr <kr...@kreme.com> wrote:
> I mean, sure, it's mostly above my head, but reading the process you guys have gone through is just fascinating and tickles my nerd bones.
Oh, and then this just came across my screen; it feels shockingly apropos.
<https://gizmodo.com/even-random-paint-splatters-can-be-valid-computer-code-1833910587>
McMillen discovered that when random paint splatters were processed by OCR software—character-scanning tools that are typically used to turn analog words into digital text—it resulted in valid Perl code 93 percent of the time. Not necessarily useful Perl code, but code that still properly executes.
--
Gods don't like people not doing much work. People who aren't busy all
the time might start to think.
Re: uninitialized value $( in Util.pm line 1595
Posted by "@lbutlr" <kr...@kreme.com>.
On 9 Apr 2019, at 15:01, Bill Cole <sa...@billmail.scconsult.com> wrote:
> I was finally able to reproduce this and in doing so I nailed down a definitive fact: this is a bug in the RedHat-built perl.
Man, I love threads like this.
I mean, sure, it's mostly above my head, but reading the process you guys have gone through is just fascinating and tickles my nerd bones.
--
Be careful what you wish for. You never know who will be listening. Or
what, for that matter.
Re: uninitialized value $( in Util.pm line 1595
Posted by Bill Cole <sa...@billmail.scconsult.com>.
On 9 Apr 2019, at 1:31, Amir Caspi wrote:
> Could it be that the assignment to $( and $) is not being done
> numerically? But trying to force $( numeric by adding 0 didn't help
> anything...
That matches my testing...
> Also, from reading perldoc, it looks like $( is supposed to be the old
> GID, i.e., the one we're leaving, while $) is supposed to be the
> effective GID, i.e., the one we're joining.
Not exactly.
They are the "real" and "effective" groups. If we were in a situation
where we might want to revert to the real primary GID later, we might
just set the effective GID to the new primary group, but we want to drop
any privileges bound to the original group, so we want to change both.
Hence the POSIX::setgid() call.
> But the assignment to $( is the new GID, not the old one. Am I
> misreading?
Not exactly, but the Perl docs don't tell the story of THIS code.
If you trace back why this is being done, it boils down to the pyzor
client (a distinct helper process) needing a pipe to talk to a spamd
child that both processes can talk to and that isn't a potential weak
point with privilege. The point of this code is to reduce privileges
irretrievably, not preserve an identity that can be returned to.
> I doubt that's the issue of course, since setting it to $oldpgid
> doesn't resolve it either.
>
> Somehow it appears the assignment isn't working properly, or is
> somehow not surviving the return from the subfunction.
>
> Let me know what else to test...
I have bad news...
I was finally able to reproduce this and in doing so I nailed down a
definitive fact: this is a bug in the RedHat-built perl. The fact that
assignment to $( was not setting $! but was resulting in $( being a
broken string with a null where the effective GID should be already
pointed at this as the problem, and when I used an alternative Perl
environment with identical SA code on the machine where I had reproduced
the breakage, the problem went away.
We had a similar bug recently
(https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7591) also due to
the RH custom perl build. Unfortunately, it appears that there's either
a bug in Perl itself OR in RH's use of "-D_FORTIFY_SOURCE=2" when
building Perl, a compiler feature that is intended to prevent some
common security-relevant coding errors but which carries the warning
"some conforming programs might fail."
--
Bill Cole
bill@scconsult.com or billcole@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Available For Hire: https://linkedin.com/in/billcole
Re: uninitialized value $( in Util.pm line 1595
Posted by Amir Caspi <ce...@3phase.com>.
On Apr 8, 2019, at 9:46 PM, Bill Cole <sa...@billmail.scconsult.com> wrote:
> I think it's right now... I'm still a bit puzzled by how a null apparently got into the first position of $(, but I hope it was an aberration.
>
> Got it. Try the new patch. I'll spin up my Centos test VM tomorrow and try to reproduce this again.
Sadly, no help. I applied the patch and I can see the revised util log msg, but the error is still there... it's now moved to line 1596 since your patch added a line:
Apr 9 05:05:43 kismet spamd[30175]: get_user_groups: uid is 1000
Apr 9 05:05:43 kismet spamd[30175]: get_user_groups: added 4 (adm) to group list which is now: 1000 4
Apr 9 05:05:43 kismet spamd[30175]: get_user_groups: added 10 (wheel) to group list which is now: 1000 4 10
Apr 9 05:05:43 kismet spamd[30175]: get_user_groups: added 190 (systemd-journal) to group list which is now: 1000 4 10 190
Apr 9 05:05:43 kismet spamd[30175]: util: changing real primary gid from 0 to 1000 and supplemental groups from 4 10 190 to 4 10 190 to match effective uid 1000
Apr 9 05:05:43 kismet spamd[30175]: util: POSIX::setgid(1000) set errno to
Apr 9 05:05:43 kismet spamd[30175]: util: assignment $) = 1000 4 10 190 set errno to
Apr 9 05:05:43 kismet spamd[30175]: util: changing real uid from 0 to match effective uid 1000
Apr 9 05:05:43 kismet spamd[30175]: Use of uninitialized value $( in concatenation (.) or string at /usr/share/perl5/vendor_perl/Mail/SpamAssassin/Util.pm line 1596, <GEN11> line 34.
Apr 9 05:05:43 kismet spamd[30175]: Use of uninitialized value $) in concatenation (.) or string at /usr/share/perl5/vendor_perl/Mail/SpamAssassin/Util.pm line 1596, <GEN11> line 34.
Could it be that the assignment to $( and $) is not being done numerically? But trying to force $( numeric by adding 0 didn't help anything...
Also, from reading perldoc, it looks like $( is supposed to be the old GID, i.e., the one we're leaving, while $) is supposed to be the effective GID, i.e., the one we're joining. But the assignment to $( is the new GID, not the old one. Am I misreading? I doubt that's the issue of course, since setting it to $oldpgid doesn't resolve it either.
Somehow it appears the assignment isn't working properly, or is somehow not surviving the return from the subfunction.
Let me know what else to test...
Cheers.
--- Amir
Re: uninitialized value $( in Util.pm line 1595
Posted by Bill Cole <sa...@billmail.scconsult.com>.
On 8 Apr 2019, at 21:33, Amir Caspi wrote:
> On Apr 8, 2019, at 7:06 PM, Bill Cole
> <sa...@billmail.scconsult.com> wrote:
>> I believe the issue is with group 0. I'm working on it...
Mistakes were made...
>> Have you tested with a user who is NOT in group 0?
>
> I'm a bit confused. spamd is running setuid root so it starts in
> group 0, but spamc is called with -u so spamd does a setuid and setgid
> to the calling user. In the example I showed, user centos (uid 1000)
> is not in group 0, as you can see from the output of 'id -a'. So none
> of the target groups are 0.
Right.
> But, I do see something strange -- it looks like the supplementary
> groups (4, 10, 190) are being added to both the "real primary gid" as
> well as the target group list, because I see that util is changing
> "from 0 4 10 190" to "1000 and supplemental" ... why were 4, 10, and
> 190 added to the "current" gid? (I'll note: root is ONLY in group 0,
> and is not in any of those three other groups.)
Indeed that was extremely odd, and is what was confusing me about group
0. I believe that the new patch I just added to the Bugzilla ticket
fixes that. It is a *REPLACEMENT* for the earlier patch.
> Is that the source of the error, or something related but different?
It was apparently the source of the error, indicating that the way I was
handling the $( and $) strings was slightly off.
I think it's right now... I'm still a bit puzzled by how a null
apparently got into the first position of $(, but I hope it was an
aberration.
> But, to recap: the target user, centos (uid 1000), is not in group 0.
Got it. Try the new patch. I'll spin up my Centos test VM tomorrow and
try to reproduce this again.
--
Bill Cole
bill@scconsult.com or billcole@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Available For Hire: https://linkedin.com/in/billcole
Re: uninitialized value $( in Util.pm line 1595
Posted by Amir Caspi <ce...@3phase.com>.
On Apr 8, 2019, at 7:06 PM, Bill Cole <sa...@billmail.scconsult.com> wrote:
> I believe the issue is with group 0. I'm working on it...
> Have you tested with a user who is NOT in group 0?
I'm a bit confused. spamd is running setuid root so it starts in group 0, but spamc is called with -u so spamd does a setuid and setgid to the calling user. In the example I showed, user centos (uid 1000) is not in group 0, as you can see from the output of 'id -a'. So none of the target groups are 0.
But, I do see something strange -- it looks like the supplementary groups (4, 10, 190) are being added to both the "real primary gid" as well as the target group list, because I see that util is changing "from 0 4 10 190" to "1000 and supplemental" ... why were 4, 10, and 190 added to the "current" gid? (I'll note: root is ONLY in group 0, and is not in any of those three other groups.)
Is that the source of the error, or something related but different?
But, to recap: the target user, centos (uid 1000), is not in group 0.
> (tangential coincidence: As I was reading your message, you appeared on my TV. )
Hah! Given the hour, I can guess what episode. ;-) I swear I'm not stalking you! ;-)
Thanks!
--- Amir
Re: uninitialized value $( in Util.pm line 1595
Posted by Bill Cole <sa...@billmail.scconsult.com>.
On 8 Apr 2019, at 20:04, Amir Caspi wrote:
> Does this help?
Yes!
I believe the issue is with group 0. I'm working on it...
Have you tested with a user who is NOT in group 0?
(tangential coincidence: As I was reading your message, you appeared on
my TV. )
--
Bill Cole
bill@scconsult.com or billcole@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Available For Hire: https://linkedin.com/in/billcole
Re: uninitialized value $( in Util.pm line 1595
Posted by Amir Caspi <ce...@3phase.com>.
On Apr 8, 2019, at 5:37 PM, Bill Cole <sa...@billmail.scconsult.com> wrote:
> What does running 'id -a' as the problem user say?
uid=1000(centos) gid=1000(centos) groups=1000(centos),4(adm),10(wheel),190(systemd-journal) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
> I still haven't been able to reproduce it.
I'm running a stock CentOS 7 configuration except that SA 3.4.2 is installed from the Fedora SRPM (using rpmbuild) since EPEL is lagging behind.
> If you add "-D all" to SPAMDOPTIONS in /etc/sysconfig/spamassassin and restart spamd, it will log an insane amount (i.e. don't leave it that way) including lines with "get_user_groups: added" that will show the group list being built. This may help provide insight to how $( is being mangked and how to avoid it.
Here are the relevant get_user_groups and other util lines leading up to the error. Note that the error occurs four times per message, twice (in a row) with a <GENxx> tag and twice (in a row) without. It runs pyzor and razor tests in between.
Apr 8 23:56:14 kismet spamd[24113]: get_user_groups: uid is 1000
Apr 8 23:56:14 kismet spamd[24113]: get_user_groups: added 4 (adm) to group list which is now: 1000 4
Apr 8 23:56:14 kismet spamd[24113]: get_user_groups: added 10 (wheel) to group list which is now: 1000 4 10
Apr 8 23:56:14 kismet spamd[24113]: get_user_groups: added 190 (systemd-journal) to group list which is now: 1000 4 10 190
Apr 8 23:56:14 kismet spamd[24113]: util: changing real primary gid from 0 4 10 190 to 1000 and supplemental groups to 4 10 190 to match effective uid 1000
Apr 8 23:56:14 kismet spamd[24113]: util: POSIX::setgid(1000) set errno to
Apr 8 23:56:14 kismet spamd[24113]: util: assignment $) = 1000 4 10 190 set errno to
Apr 8 23:56:14 kismet spamd[24113]: util: changing real uid from 0 to match effective uid 1000
Apr 8 23:56:14 kismet spamd[24113]: Use of uninitialized value $( in concatenation (.) or string at /usr/share/perl5/vendor_perl/Mail/SpamAssassin/Util.pm line 1595, <GEN11> line 29.
Apr 8 23:56:14 kismet spamd[24113]: Use of uninitialized value $) in concatenation (.) or string at /usr/share/perl5/vendor_perl/Mail/SpamAssassin/Util.pm line 1595, <GEN11> line 29.
Apr 8 23:56:14 kismet spamd[24113]: util: setuid: ruid=1000 euid=1000 rgid= 10 190 egid= 10 190
When I send a test email to a non-problem user (i.e., a user with only a single group), the lines look as follows:
Apr 9 00:01:22 kismet spamd[24195]: get_user_groups: uid is 22001
Apr 9 00:01:22 kismet spamd[24195]: util: changing real primary gid from 0 to 505 and supplemental groups to 505 to match effective uid 22001
Apr 9 00:01:22 kismet spamd[24195]: util: POSIX::setgid(505) set errno to
Apr 9 00:01:22 kismet spamd[24195]: util: assignment $) = 505 505 set errno to
Apr 9 00:01:22 kismet spamd[24195]: util: changing real uid from 0 to match effective uid 22001
Apr 9 00:01:22 kismet spamd[24195]: util: setuid: ruid=22001 euid=22001 rgid=505 egid=505
Does this help?
Thanks!
--- Amir
Re: uninitialized value $( in Util.pm line 1595
Posted by Bill Cole <sa...@billmail.scconsult.com>.
On 8 Apr 2019, at 17:41, Amir Caspi wrote:
> Kevin et al,
>
> I am getting errors in maillog relating to an uninitialized $( in
> Util.pm, line 1595:
> Mar 24 03:28:35 server spamd[27149]: Use of uninitialized value $( in
> concatenation (.) or string at
> /usr/share/perl5/vendor_perl/Mail/SpamAssassin/Util.pm line 1595.
Note that this line is just logging. Comment out that line and no
functionality is lost.
> I saw an existing bugzilla
> (https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7629) for this
> error occurring on lines 1510 and 1595... it looks like 1510 has been
> patched but 1595 is still happening. I posted in that BZ but it seems
> to not have much activity so I thought I'd probe the list to see if
> others might have an idea how to patch it...
>
> Following the discussion in that BZ, I'm guessing this is happening
> because the spamc user is a member of multiple groups:
> util: setuid: ruid=1000 euid=1000 rgid= 10 190 egid= 10 190
Proof that line 1595 works despite the warning. Also proof that "$(" is
not uninitialized. This is something broken in Perl.
Also, there's something weird there. See the extra spaces? This implies
that something has mangled "$(" (the space-delimited list of groups that
the user is a member of) to remove the first member, but the first
delimiter is still there.
> So when line 1595 is trying to find the gid, it's getting something
> formatted differently than what it wants.
In this case, it's getting something formatted dead wrong: counter to
the definition of that Perl-specified variable.
> When spamc is run for a user who is only a member of a single group,
> this error doesn't occur, so it seems clear that it's related to being
> part of multiple groups.
Or to the specific groups or the specific user or the build of Perl...
What does running 'id -a' as the problem user say?
> Line 1510 was solved by including a split statement... could that help
> here, too?
The original reporter of the bug said it did not. I expect that it would
not.
> Unfortunately I'm not sufficiently expert to figure out the proper
> syntax and where to include the split.
>
> (This is in CentOS 7, fully updated.)
I still haven't been able to reproduce it.
If you add "-D all" to SPAMDOPTIONS in /etc/sysconfig/spamassassin and
restart spamd, it will log an insane amount (i.e. don't leave it that
way) including lines with "get_user_groups: added" that will show the
group list being built. This may help provide insight to how $( is being
mangked and how to avoid it.