You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Basant Kukreja <Ba...@Sun.COM> on 2008/09/10 07:19:47 UTC
Re: svn commit: r691418 [2/2] - in /httpd/httpd/trunk: ./
docs/manual/mod/ modules/filters/
On Tue, Sep 09, 2008 at 10:10:42PM +0200, Ruediger Pluem wrote:
>
>
> On 09/03/2008 01:01 AM, niq@apache.org wrote:
>> Added: httpd/httpd/trunk/modules/filters/sed1.c
>> URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/filters/sed1.c?rev=691418&view=auto
>> ==============================================================================
>> --- httpd/httpd/trunk/modules/filters/sed1.c (added)
>> +++ httpd/httpd/trunk/modules/filters/sed1.c Tue Sep 2 16:01:47 2008
>> @@ -0,0 +1,957 @@
>> +/*
>> + * Copyright (c) 2005, 2008 Sun Microsystems, Inc. All Rights Reserved.
>> + * Use is subject to license terms.
>> + *
>> + * Copyright (c) 1984 AT&T
>> + * All Rights Reserved
>> + *
>> + * Licensed under the Apache License, Version 2.0 (the "License");
>> + * you may not use this file except in compliance with the License.
>> + * You may obtain a copy of the License at
>> + * http://www.apache.org/licenses/LICENSE-2.0. + * + * Unless
>> required by applicable law or agreed to in writing, software + *
>> distributed under the License is distributed on an "AS IS" BASIS, + *
>> WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or
>> implied. + * See the License for the specific language governing
>> permissions and
>> + * limitations under the License. + */
>> +
>> +#include "apr.h"
>> +#include "apr_lib.h"
>> +#include "libsed.h"
>> +#include "sed.h"
>> +#include "apr_strings.h"
>> +#include "regexp.h"
>> +
>> +char *trans[040] = {
>> + "\\01",
>> + "\\02",
>> + "\\03",
>> + "\\04",
>> + "\\05",
>> + "\\06",
>> + "\\07",
>> + "-<",
>> + "->",
>
> What are the above constants supposed to be. Opening the file in vi shows that they are special
> characters or better control characters. Looking with a hex editor these () seem to be \\08.
> Is this correct?
Sed has a "l" command. From the sed man page :
(2)l List the pattern space on the standard out-
put in an unambiguous form. Non-printing
characters are spelled in two digit ASCII
and long lines are folded.
>From the code :
p3 = trans[(unsigned char)*p1-1];
while ((*p2++ = *p3++) != 0)
if(p2 >= eval->lcomend) {
*p2 = '\\';
wline(eval, eval->genbuf, strlen(eval->genbuf));
p2 = eval->genbuf;
}
It looks to me that it is trying to print character from value 0 to 31 as
printable characters.
>> + "-<",
>> + "->",
It seems to me that it should be \\08 and \\09.
I will dig deeper to see if these can be simplified. It looks little weird that
there are binary characters in source file.
Regards,
Basant.
Re: svn commit: r691418 [2/2] -
in/httpd/httpd/trunk:./docs/manual/mod/ modules/filters/
Posted by Basant Kumar kukreja <Ba...@Sun.COM>.
On Thu, Sep 11, 2008 at 10:36:18PM +0200, Ruediger Pluem wrote:
>
>
> On 09/10/2008 11:40 PM, Basant Kumar kukreja wrote:
>> Patch is attached. I have tested the patch. It now prints \10 and \11 for
>> character values 0x8 and 0x9.
>
> Thanks for the patch.
> Committed as r694431 (http://svn.apache.org/viewvc?rev=694431&view=rev).
Thanks.
Basant.
>
> Regards
>
> Rüdiger
>
Re: svn commit: r691418 [2/2] - in/httpd/httpd/trunk:./docs/manual/mod/
modules/filters/
Posted by Ruediger Pluem <rp...@apache.org>.
On 09/10/2008 11:40 PM, Basant Kumar kukreja wrote:
> Patch is attached. I have tested the patch. It now prints \10 and \11 for
> character values 0x8 and 0x9.
Thanks for the patch.
Committed as r694431 (http://svn.apache.org/viewvc?rev=694431&view=rev).
Regards
Rüdiger
Re: svn commit: r691418 [2/2] -
in/httpd/httpd/trunk:./docs/manual/mod/ modules/filters/
Posted by Basant Kumar kukreja <Ba...@Sun.COM>.
Patch is attached. I have tested the patch. It now prints \10 and \11 for
character values 0x8 and 0x9.
Regards,
Basant.
On Wed, Sep 10, 2008 at 04:00:42PM +0200, "Plüm, Rüdiger, VF-Group" wrote:
>
>
> > -----Ursprüngliche Nachricht-----
> > Von: Basant.Kukreja@Sun.COM
> > Gesendet: Mittwoch, 10. September 2008 15:54
> > An: dev@httpd.apache.org
> > Betreff: Re: svn commit: r691418 [2/2] -
> > in/httpd/httpd/trunk:./docs/manual/mod/ modules/filters/
> >
>
> > > > Can you elaborate why you chose "<" and ">". I could not
> > think of any
> > > > reasons behind it.
> > >
> > > Because these characters are currently displayed by sed and
> > I saw no reason to change
> > > it.
> > >
> > > > sed's man page says :
> > > > (2)l List the pattern space on the standard out-
> > > > put in an unambiguous form. Non-printing
> > > > characters are spelled in two digit ASCII
> > > > and long lines are folded.
> > > >
> > > > So 0x8 and 0x9 char values, which I believe are non printable
> > > > characters,
> > > > should be printed into *two* digit ASCII. So \10 and \11
> > looks to me
> > > > as conforming to man page.
> > >
> > > As said displaying two digit ASCII's also makes sense and
> > conforms to the man
> > > page. So go for it.
> > > BTW: Shouldn't it be \08 and \09 for 0x8 and 0x9 instead of
> > \10 and \11 ?
> > If we notice, how other characters are printed then sed code
> > is using octal
> > numbers. After \17 it changes to \20, \21. Similarly After
> > \27 it changes to
> > \30, \31 etc. So based on the above pattern, \10 and \11
> > seems to be more
> > consistent to me.
>
> Thanks for pointing out. I missed that these were actual octal numbers.
> So yes, you are correct.
>
> Regards
>
> Rüdiger
>
Re: svn commit: r691418 [2/2] - in/httpd/httpd/trunk:./docs/manual/mod/ modules/filters/
Posted by "Plüm, Rüdiger, VF-Group" <ru...@vodafone.com>.
> -----Ursprüngliche Nachricht-----
> Von: Basant.Kukreja@Sun.COM
> Gesendet: Mittwoch, 10. September 2008 15:54
> An: dev@httpd.apache.org
> Betreff: Re: svn commit: r691418 [2/2] -
> in/httpd/httpd/trunk:./docs/manual/mod/ modules/filters/
>
> > > Can you elaborate why you chose "<" and ">". I could not
> think of any
> > > reasons behind it.
> >
> > Because these characters are currently displayed by sed and
> I saw no reason to change
> > it.
> >
> > > sed's man page says :
> > > (2)l List the pattern space on the standard out-
> > > put in an unambiguous form. Non-printing
> > > characters are spelled in two digit ASCII
> > > and long lines are folded.
> > >
> > > So 0x8 and 0x9 char values, which I believe are non printable
> > > characters,
> > > should be printed into *two* digit ASCII. So \10 and \11
> looks to me
> > > as conforming to man page.
> >
> > As said displaying two digit ASCII's also makes sense and
> conforms to the man
> > page. So go for it.
> > BTW: Shouldn't it be \08 and \09 for 0x8 and 0x9 instead of
> \10 and \11 ?
> If we notice, how other characters are printed then sed code
> is using octal
> numbers. After \17 it changes to \20, \21. Similarly After
> \27 it changes to
> \30, \31 etc. So based on the above pattern, \10 and \11
> seems to be more
> consistent to me.
Thanks for pointing out. I missed that these were actual octal numbers.
So yes, you are correct.
Regards
Rüdiger
Re: svn commit: r691418 [2/2] - in
/httpd/httpd/trunk:./docs/manual/mod/ modules/filters/
Posted by Basant Kukreja <Ba...@Sun.COM>.
On Wed, Sep 10, 2008 at 03:41:21PM +0200, "Plüm, Rüdiger, VF-Group" wrote:
>
>
> > -----Ursprüngliche Nachricht-----
> > Von: Basant.Kukreja@Sun.COM
> > Gesendet: Mittwoch, 10. September 2008 15:32
> > An: dev@httpd.apache.org
> > Betreff: Re: svn commit: r691418 [2/2] - in
> > /httpd/httpd/trunk:./docs/manual/mod/ modules/filters/
> >
> > On Wed, Sep 10, 2008 at 10:10:20AM +0200, "Plüm, Rüdiger,
> > VF-Group" wrote:
> > >
> > >
> > > > -----Ursprüngliche Nachricht-----
> > > > Von: Basant.Kukreja@Sun.COM
> > > > Gesendet: Mittwoch, 10. September 2008 08:56
> > > > An: dev@httpd.apache.org
> > > > Betreff: Re: svn commit: r691418 [2/2] - in
> > > > /httpd/httpd/trunk: ./docs/manual/mod/ modules/filters/
> > > >
> > >
> > > > I investigated further. I wrote a test file having binary
> > > > character from 0 to 31:
> > > > $ od -c out.txt
> > > > 0000000 \0 \n 001 \n 002 \n 003 \n 004 \n 005 \n 006
> > > > \n 007 \n
> > > > 0000020 \b \n \t \n \n \n 013 \n \f \n \r \n 016
> > > > \n 017 \n
> > > > ...
> > > >
> > > > And a small sed script :
> > > > $ cat one.sed
> > > > l
> > > > d
> > > >
> > > > Sed script just runs the "l" command for each line.
> > > > $ /usr/ucb/sed -f one.sed out.txt > out1.txt
> > > >
> > > > Here is the output of out1.txt
> > > > $ od -c out1.txt
> > > > 0000000 \n \ 0 1 \n \ 0 2 \n \ 0 3 \n
> > > > \ 0 4
> > > > 0000020 \n \ 0 5 \n \ 0 6 \n \ 0 7 \n
> > > > - \b <
> > > > 0000040 \n - \b > \n \n \n \ 1 3 \n \ 1
> > > > 4 \n \
> > > > 0000060 1 5 \n \ 1 6 \n \ 1 7 \n \ 2
> > > > 0 \n \
> > > > ...
> > > >
> > > > $ cat out1.txt
> > > > \01
> > > > \02
> > > > \03
> > > > \04
> > > > \05
> > > > \06
> > > > \07
> > > > <
> > > > >
> > > >
> > > >
> > > > \13
> > > > \14
> > > >
> > > > -------------------------------------------
> > > >
> > > > So for some strange reason :
> > > > 0x8 is converted to "-\b<" and
> > > > 0x9 is converted to "-\b>"
> > > >
> > > > That's what we see in "trans" variable.
> > > >
> > > > Do you think it could be a bug in original sed and should we
> > > > correct it?
> > >
> > > I guess it is a bug in original sed and it should be corrected.
> > > IMHO it should be suffient to replace
> > >
> > > -\b<
> > >
> > > and
> > >
> > > -\b<
> > >
> > > with
> > >
> > > <
> > >
> > > and
> > >
> > > >
> > Can you elaborate why you chose "<" and ">". I could not think of any
> > reasons behind it.
>
> Because these characters are currently displayed by sed and I saw no reason to change
> it.
>
> > sed's man page says :
> > (2)l List the pattern space on the standard out-
> > put in an unambiguous form. Non-printing
> > characters are spelled in two digit ASCII
> > and long lines are folded.
> >
> > So 0x8 and 0x9 char values, which I believe are non printable
> > characters,
> > should be printed into *two* digit ASCII. So \10 and \11 looks to me
> > as conforming to man page.
>
> As said displaying two digit ASCII's also makes sense and conforms to the man
> page. So go for it.
> BTW: Shouldn't it be \08 and \09 for 0x8 and 0x9 instead of \10 and \11 ?
If we notice, how other characters are printed then sed code is using octal
numbers. After \17 it changes to \20, \21. Similarly After \27 it changes to
\30, \31 etc. So based on the above pattern, \10 and \11 seems to be more
consistent to me.
For reference, output of the converted characters (0-31) by "l" command is
given below.
$ cat out1.txt
\01
\02
\03
\04
\05
\06
\07
<
>
\13
\14
\15
\16
\17
\20
\21
\22
\23
\24
\25
\26
\27
\30
\31
\32
\33
\34
\35
\36
\37
I will submit the patch today. Thanks for noticing this.
Regards,
Basant.
Re: svn commit: r691418 [2/2] - in /httpd/httpd/trunk:./docs/manual/mod/ modules/filters/
Posted by "Plüm, Rüdiger, VF-Group" <ru...@vodafone.com>.
> -----Ursprüngliche Nachricht-----
> Von: Basant.Kukreja@Sun.COM
> Gesendet: Mittwoch, 10. September 2008 15:32
> An: dev@httpd.apache.org
> Betreff: Re: svn commit: r691418 [2/2] - in
> /httpd/httpd/trunk:./docs/manual/mod/ modules/filters/
>
> On Wed, Sep 10, 2008 at 10:10:20AM +0200, "Plüm, Rüdiger,
> VF-Group" wrote:
> >
> >
> > > -----Ursprüngliche Nachricht-----
> > > Von: Basant.Kukreja@Sun.COM
> > > Gesendet: Mittwoch, 10. September 2008 08:56
> > > An: dev@httpd.apache.org
> > > Betreff: Re: svn commit: r691418 [2/2] - in
> > > /httpd/httpd/trunk: ./docs/manual/mod/ modules/filters/
> > >
> >
> > > I investigated further. I wrote a test file having binary
> > > character from 0 to 31:
> > > $ od -c out.txt
> > > 0000000 \0 \n 001 \n 002 \n 003 \n 004 \n 005 \n 006
> > > \n 007 \n
> > > 0000020 \b \n \t \n \n \n 013 \n \f \n \r \n 016
> > > \n 017 \n
> > > ...
> > >
> > > And a small sed script :
> > > $ cat one.sed
> > > l
> > > d
> > >
> > > Sed script just runs the "l" command for each line.
> > > $ /usr/ucb/sed -f one.sed out.txt > out1.txt
> > >
> > > Here is the output of out1.txt
> > > $ od -c out1.txt
> > > 0000000 \n \ 0 1 \n \ 0 2 \n \ 0 3 \n
> > > \ 0 4
> > > 0000020 \n \ 0 5 \n \ 0 6 \n \ 0 7 \n
> > > - \b <
> > > 0000040 \n - \b > \n \n \n \ 1 3 \n \ 1
> > > 4 \n \
> > > 0000060 1 5 \n \ 1 6 \n \ 1 7 \n \ 2
> > > 0 \n \
> > > ...
> > >
> > > $ cat out1.txt
> > > \01
> > > \02
> > > \03
> > > \04
> > > \05
> > > \06
> > > \07
> > > <
> > > >
> > >
> > >
> > > \13
> > > \14
> > >
> > > -------------------------------------------
> > >
> > > So for some strange reason :
> > > 0x8 is converted to "-\b<" and
> > > 0x9 is converted to "-\b>"
> > >
> > > That's what we see in "trans" variable.
> > >
> > > Do you think it could be a bug in original sed and should we
> > > correct it?
> >
> > I guess it is a bug in original sed and it should be corrected.
> > IMHO it should be suffient to replace
> >
> > -\b<
> >
> > and
> >
> > -\b<
> >
> > with
> >
> > <
> >
> > and
> >
> > >
> Can you elaborate why you chose "<" and ">". I could not think of any
> reasons behind it.
Because these characters are currently displayed by sed and I saw no reason to change
it.
> sed's man page says :
> (2)l List the pattern space on the standard out-
> put in an unambiguous form. Non-printing
> characters are spelled in two digit ASCII
> and long lines are folded.
>
> So 0x8 and 0x9 char values, which I believe are non printable
> characters,
> should be printed into *two* digit ASCII. So \10 and \11 looks to me
> as conforming to man page.
As said displaying two digit ASCII's also makes sense and conforms to the man
page. So go for it.
BTW: Shouldn't it be \08 and \09 for 0x8 and 0x9 instead of \10 and \11 ?
Regards
Rüdiger
Re: svn commit: r691418 [2/2] - in /httpd/httpd/trunk:
./docs/manual/mod/ modules/filters/
Posted by Basant Kukreja <Ba...@Sun.COM>.
On Wed, Sep 10, 2008 at 10:10:20AM +0200, "Plüm, Rüdiger, VF-Group" wrote:
>
>
> > -----Ursprüngliche Nachricht-----
> > Von: Basant.Kukreja@Sun.COM
> > Gesendet: Mittwoch, 10. September 2008 08:56
> > An: dev@httpd.apache.org
> > Betreff: Re: svn commit: r691418 [2/2] - in
> > /httpd/httpd/trunk: ./docs/manual/mod/ modules/filters/
> >
>
> > I investigated further. I wrote a test file having binary
> > character from 0 to 31:
> > $ od -c out.txt
> > 0000000 \0 \n 001 \n 002 \n 003 \n 004 \n 005 \n 006
> > \n 007 \n
> > 0000020 \b \n \t \n \n \n 013 \n \f \n \r \n 016
> > \n 017 \n
> > ...
> >
> > And a small sed script :
> > $ cat one.sed
> > l
> > d
> >
> > Sed script just runs the "l" command for each line.
> > $ /usr/ucb/sed -f one.sed out.txt > out1.txt
> >
> > Here is the output of out1.txt
> > $ od -c out1.txt
> > 0000000 \n \ 0 1 \n \ 0 2 \n \ 0 3 \n
> > \ 0 4
> > 0000020 \n \ 0 5 \n \ 0 6 \n \ 0 7 \n
> > - \b <
> > 0000040 \n - \b > \n \n \n \ 1 3 \n \ 1
> > 4 \n \
> > 0000060 1 5 \n \ 1 6 \n \ 1 7 \n \ 2
> > 0 \n \
> > ...
> >
> > $ cat out1.txt
> > \01
> > \02
> > \03
> > \04
> > \05
> > \06
> > \07
> > <
> > >
> >
> >
> > \13
> > \14
> >
> > -------------------------------------------
> >
> > So for some strange reason :
> > 0x8 is converted to "-\b<" and
> > 0x9 is converted to "-\b>"
> >
> > That's what we see in "trans" variable.
> >
> > Do you think it could be a bug in original sed and should we
> > correct it?
>
> I guess it is a bug in original sed and it should be corrected.
> IMHO it should be suffient to replace
>
> -\b<
>
> and
>
> -\b<
>
> with
>
> <
>
> and
>
> >
Can you elaborate why you chose "<" and ">". I could not think of any
reasons behind it.
sed's man page says :
(2)l List the pattern space on the standard out-
put in an unambiguous form. Non-printing
characters are spelled in two digit ASCII
and long lines are folded.
So 0x8 and 0x9 char values, which I believe are non printable characters,
should be printed into *two* digit ASCII. So \10 and \11 looks to me
as conforming to man page.
Regards,
Basant.
Re: svn commit: r691418 [2/2] - in /httpd/httpd/trunk: ./docs/manual/mod/ modules/filters/
Posted by "Plüm, Rüdiger, VF-Group" <ru...@vodafone.com>.
> -----Ursprüngliche Nachricht-----
> Von: Basant.Kukreja@Sun.COM
> Gesendet: Mittwoch, 10. September 2008 08:56
> An: dev@httpd.apache.org
> Betreff: Re: svn commit: r691418 [2/2] - in
> /httpd/httpd/trunk: ./docs/manual/mod/ modules/filters/
>
> I investigated further. I wrote a test file having binary
> character from 0 to 31:
> $ od -c out.txt
> 0000000 \0 \n 001 \n 002 \n 003 \n 004 \n 005 \n 006
> \n 007 \n
> 0000020 \b \n \t \n \n \n 013 \n \f \n \r \n 016
> \n 017 \n
> ...
>
> And a small sed script :
> $ cat one.sed
> l
> d
>
> Sed script just runs the "l" command for each line.
> $ /usr/ucb/sed -f one.sed out.txt > out1.txt
>
> Here is the output of out1.txt
> $ od -c out1.txt
> 0000000 \n \ 0 1 \n \ 0 2 \n \ 0 3 \n
> \ 0 4
> 0000020 \n \ 0 5 \n \ 0 6 \n \ 0 7 \n
> - \b <
> 0000040 \n - \b > \n \n \n \ 1 3 \n \ 1
> 4 \n \
> 0000060 1 5 \n \ 1 6 \n \ 1 7 \n \ 2
> 0 \n \
> ...
>
> $ cat out1.txt
> \01
> \02
> \03
> \04
> \05
> \06
> \07
> <
> >
>
>
> \13
> \14
>
> -------------------------------------------
>
> So for some strange reason :
> 0x8 is converted to "-\b<" and
> 0x9 is converted to "-\b>"
>
> That's what we see in "trans" variable.
>
> Do you think it could be a bug in original sed and should we
> correct it?
I guess it is a bug in original sed and it should be corrected.
IMHO it should be suffient to replace
-\b<
and
-\b<
with
<
and
>
>
> It should probably print "\10" and "\11".
This would be an option, but I wouldn't get this far. It is still fine with me
to print < and >.
>
> BTW /usr/bin/sed have the exactly the same behavior.
>
> It sound strange though that this was never caught in sed code.
Some issues last longer than other ones :-)
Regards
Rüdiger
Re: svn commit: r691418 [2/2] - in /httpd/httpd/trunk: ./
docs/manual/mod/ modules/filters/
Posted by Basant Kukreja <Ba...@Sun.COM>.
On Tue, Sep 09, 2008 at 10:19:47PM -0700, Basant Kukreja wrote:
> >
> > What are the above constants supposed to be. Opening the file in vi shows that they are special
> > characters or better control characters. Looking with a hex editor these () seem to be \\08.
> > Is this correct?
>
> Sed has a "l" command. From the sed man page :
> (2)l List the pattern space on the standard out-
> put in an unambiguous form. Non-printing
> characters are spelled in two digit ASCII
> and long lines are folded.
>
> From the code :
> p3 = trans[(unsigned char)*p1-1];
> while ((*p2++ = *p3++) != 0)
> if(p2 >= eval->lcomend) {
> *p2 = '\\';
> wline(eval, eval->genbuf, strlen(eval->genbuf));
> p2 = eval->genbuf;
> }
>
>
> It looks to me that it is trying to print character from value 0 to 31 as
> printable characters.
>
> >> + "-<",
> >> + "->",
> It seems to me that it should be \\08 and \\09.
>
> I will dig deeper to see if these can be simplified. It looks little weird that
> there are binary characters in source file.
>
> Regards,
> Basant.
>
I investigated further. I wrote a test file having binary character from 0 to 31:
$ od -c out.txt
0000000 \0 \n 001 \n 002 \n 003 \n 004 \n 005 \n 006 \n 007 \n
0000020 \b \n \t \n \n \n 013 \n \f \n \r \n 016 \n 017 \n
...
And a small sed script :
$ cat one.sed
l
d
Sed script just runs the "l" command for each line.
$ /usr/ucb/sed -f one.sed out.txt > out1.txt
Here is the output of out1.txt
$ od -c out1.txt
0000000 \n \ 0 1 \n \ 0 2 \n \ 0 3 \n \ 0 4
0000020 \n \ 0 5 \n \ 0 6 \n \ 0 7 \n - \b <
0000040 \n - \b > \n \n \n \ 1 3 \n \ 1 4 \n \
0000060 1 5 \n \ 1 6 \n \ 1 7 \n \ 2 0 \n \
...
$ cat out1.txt
\01
\02
\03
\04
\05
\06
\07
<
>
\13
\14
-------------------------------------------
So for some strange reason :
0x8 is converted to "-\b<" and
0x9 is converted to "-\b>"
That's what we see in "trans" variable.
Do you think it could be a bug in original sed and should we correct it?
It should probably print "\10" and "\11".
BTW /usr/bin/sed have the exactly the same behavior.
It sound strange though that this was never caught in sed code.
Regards,
Basant.
Note :
GNU sed (gsed) have a different behavior. GNU sed changes it to "\b$" and "\t$".
$ sed -f one.sed out.txt
\000$
\001$
\002$
\003$
\004$
\005$
\006$
\a$
\b$
\t$
$
$
\v$
\f$
\r$
\016$
\017$
\020$