You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@harmony.apache.org by Tim Ellison <t....@gmail.com> on 2006/04/07 13:10:30 UTC
keyword scanning (was: Re: ITC: Contribution of java.math and javax.crypto)
Thanks Alex,
We already have a keyword scanner[1] so we should ensure your dictionary
words are in there.
I'd rather add the scan to the automated build rather than rely upon the
committers to remember to run it (at least one of the committers is lazy
like that ;-) )
[1]
http://svn.apache.org/viewcvs.cgi/incubator/harmony/standard/tools/keywordscan
Regards,
Tim
Alex Orlov wrote:
> On 4/7/06, Chris Gray <ch...@kiffer.be> wrote:
>> On Thursday 06 April 2006 23:59, Tim Ellison wrote:
>>> You can run the Eclipse IDE on Classpath or Harmony(*) class libraries,
>>> both are sufficiently well advanced to run it.
>>>
>>> (*) you need to use the regex code from regex-beans-math which hasn't
>>> been merged into the build process yet
>> I knew eclipse ran on Classpath (e.g. gcj), wasn't sure about Harmony. That's
>> good to hear.
>>
>> IMO Harmony developers should be encouraged to use a Sun-free workstation
>> whenever possible, to avoid this kind of slip-up. There should also be
>> automatic detectors for references to sun.* or com.sun.* classes.
>
> Hi folks,
>
> BTW we use simple script to check if the sources contain the words we
> would like to avoid. You can find it attached. The usage is pretty
> straightforward:
>
> perl check_bad_words.pl <dictionary> <directory or file>
>
> Also attached is dictionary file that contains "Sun" and "GPL" as two
> sample words. You can turn on your imagination and enhance the
> dictionary significantly. :)
>
> May we propose the committers to run it before committing anyuthing to SVN?
>
> Thanks,
> Alex Orlov.
> Intel Middleware Products Division
>
>
>> Regards,
>>
>> Chris
>>
>> --
>> Chris Gray /k/ Embedded Java Solutions BE0503765045
>> Embedded & Mobile Java, OSGi http://www.k-embedded-java.com/
>> chris.gray@kiffer.be +32 3 216 0369
>>
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>>
>>
>> ------------------------------------------------------------------------
>>
>> #!/usr/bin/perl
>>
>> use File::Spec;
>> use strict;
>>
>> my $ext_src = "java|c|h|cpp|hpp|j|jfl|ccode";
>> my $ext_sup = "xml|txt|pl|sh|bat|cmd";
>> my $ext = "(" . $ext_src . "|" . $ext_sup . ")";
>> my $filename = "copyright|license|notice|readme";
>> my %bad_words = ();
>>
>> sub readDictionary {
>> my ($dic) = @_;
>> if (-f $dic) {
>> open (FD_DIC, "< $dic") or die "Can't open dictionary file: $dic\n";
>> while (<FD_DIC>) {
>> unless (/^#/ || /^\s+$/) {
>> my ($word, $area) = split(" ; ");
>> chomp $area;
>> my @areas = split(":", $area);
>> push @{$bad_words{$word}}, @areas;
>> }
>> }
>> close(FD_DIC);
>> } else {
>> die "Invalid dictionary file: $dic\n";
>> }
>> }
>>
>> sub readDir {
>> my ($file) = @_;
>> if (-d $file) {
>> if (opendir(FD_DIR, $file)) {
>> chdir ($file);
>> foreach my $item (readdir(FD_DIR)) {
>> if ($item ne '.' && $item ne '..') {
>> readDir($item);
>> }
>> }
>> chdir ("..");
>> closedir (FD_DIR);
>> } else {
>> print "Can't open dir $file\n";
>> }
>> } elsif (-f $file) {
>> if ($file =~ /\.$ext$/ || $file =~ /^$filename/i) {
>> &searchBadWords(&readFile($file), $file);
>> }
>> } else {
>> print "File or directory doesn't exist: $file\n";
>> }
>> }
>>
>> sub searchBadWords($$) {
>> my ($content, $file) = @_;
>> my ($prev, $after, $match, $area);
>> my $abs_fname = File::Spec->rel2abs($file);
>> foreach my $word (sort keys %bad_words) {
>> my $checkable = 0; # false
>> foreach $area (@{$bad_words{$word}}) {
>> if (($area eq "all") or (index($abs_fname, $area) ne -1)) {
>> $checkable = 1; # true
>> }
>> }
>> if ($checkable) {
>> if ($content =~ m/$word/i) {
>> $match = $&;
>> chomp($match);
>> $prev = $`;
>> $after = $';
>> print "File: $abs_fname\n";
>> print "Bad word: $match\n";
>> print "Line #" . &getLineNumber($prev) . "\n";
>> print "String: " . &getString($prev, $after, $match) . "\n";
>> print "------------------------------------\n";
>> }
>> }
>> }
>> if ($file =~ /^*\.($ext_src)/) {
>> if ($content =~ /[\s_]bug[\s_]*(#*)(\d+)/) {
>> $match = $&;
>> chomp($match);
>> $prev = $`;
>> $after = $';
>> print "File: $abs_fname\n";
>> print "Bug mentioning: $match\n";
>> print "Line #" . &getLineNumber($prev) . "\n";
>> print "String: " . &getString($prev, $after, $match) . "\n";
>> print "------------------------------------\n";
>> }
>> }
>> }
>>
>> sub readFile($) {
>> my($file_name) = @_;
>> my $content = "";
>> if (open(FH, "< " . $file_name)) {
>> while (<FH>) {
>> $content .= $_;
>> }
>> close FH;
>> } else {
>> printf STDERR "Can't open file $file_name\n";
>> }
>> $content;
>> }
>>
>> sub getString($$$) {
>> my($prev, $after, $match) = @_;
>> my $last_p = rindex($prev, "\n");
>> my $first_a = index($after, "\n");
>> substr($prev, $last_p + 1, length($prev) - $last_p) . $match . substr($after, 0, $first_a);
>> }
>>
>> sub getLineNumber($) {
>> my($string) = @_;
>> my $lnum = 1;
>> while ($string =~ m/\n/g) {
>> $lnum++;
>> }
>> $lnum;
>> }
>>
>> if (scalar(@ARGV) < 2) {
>> print "Usage: perl check_bad_words.pl <dictionary> <directory or file>";
>> exit;
>> }
>>
>> &readDictionary($ARGV[0]);
>> &readDir($ARGV[1]);
>>
>>
>> ------------------------------------------------------------------------
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
--
Tim Ellison (t.p.ellison@gmail.com)
IBM Java technology centre, UK.
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org
Re: keyword scanning (was: Re: ITC: Contribution of java.math and javax.crypto)
Posted by Alex Orlov <am...@gmail.com>.
:) Thanks for letting know this!
Thanks,
Alex Orlov.
Intel Middleware Products Division
On 4/7/06, Tim Ellison <t....@gmail.com> wrote:
> Thanks Alex,
>
> We already have a keyword scanner[1] so we should ensure your dictionary
> words are in there.
>
> I'd rather add the scan to the automated build rather than rely upon the
> committers to remember to run it (at least one of the committers is lazy
> like that ;-) )
>
> [1]
> http://svn.apache.org/viewcvs.cgi/incubator/harmony/standard/tools/keywordscan
>
> Regards,
> Tim
>
> Alex Orlov wrote:
> > On 4/7/06, Chris Gray <ch...@kiffer.be> wrote:
> >> On Thursday 06 April 2006 23:59, Tim Ellison wrote:
> >>> You can run the Eclipse IDE on Classpath or Harmony(*) class libraries,
> >>> both are sufficiently well advanced to run it.
> >>>
> >>> (*) you need to use the regex code from regex-beans-math which hasn't
> >>> been merged into the build process yet
> >> I knew eclipse ran on Classpath (e.g. gcj), wasn't sure about Harmony. That's
> >> good to hear.
> >>
> >> IMO Harmony developers should be encouraged to use a Sun-free workstation
> >> whenever possible, to avoid this kind of slip-up. There should also be
> >> automatic detectors for references to sun.* or com.sun.* classes.
> >
> > Hi folks,
> >
> > BTW we use simple script to check if the sources contain the words we
> > would like to avoid. You can find it attached. The usage is pretty
> > straightforward:
> >
> > perl check_bad_words.pl <dictionary> <directory or file>
> >
> > Also attached is dictionary file that contains "Sun" and "GPL" as two
> > sample words. You can turn on your imagination and enhance the
> > dictionary significantly. :)
> >
> > May we propose the committers to run it before committing anyuthing to SVN?
> >
> > Thanks,
> > Alex Orlov.
> > Intel Middleware Products Division
> >
> >
> >> Regards,
> >>
> >> Chris
> >>
> >> --
> >> Chris Gray /k/ Embedded Java Solutions BE0503765045
> >> Embedded & Mobile Java, OSGi http://www.k-embedded-java.com/
> >> chris.gray@kiffer.be +32 3 216 0369
> >>
> >>
> >> ---------------------------------------------------------------------
> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >>
> >>
> >>
> >> ------------------------------------------------------------------------
> >>
> >> #!/usr/bin/perl
> >>
> >> use File::Spec;
> >> use strict;
> >>
> >> my $ext_src = "java|c|h|cpp|hpp|j|jfl|ccode";
> >> my $ext_sup = "xml|txt|pl|sh|bat|cmd";
> >> my $ext = "(" . $ext_src . "|" . $ext_sup . ")";
> >> my $filename = "copyright|license|notice|readme";
> >> my %bad_words = ();
> >>
> >> sub readDictionary {
> >> my ($dic) = @_;
> >> if (-f $dic) {
> >> open (FD_DIC, "< $dic") or die "Can't open dictionary file: $dic\n";
> >> while (<FD_DIC>) {
> >> unless (/^#/ || /^\s+$/) {
> >> my ($word, $area) = split(" ; ");
> >> chomp $area;
> >> my @areas = split(":", $area);
> >> push @{$bad_words{$word}}, @areas;
> >> }
> >> }
> >> close(FD_DIC);
> >> } else {
> >> die "Invalid dictionary file: $dic\n";
> >> }
> >> }
> >>
> >> sub readDir {
> >> my ($file) = @_;
> >> if (-d $file) {
> >> if (opendir(FD_DIR, $file)) {
> >> chdir ($file);
> >> foreach my $item (readdir(FD_DIR)) {
> >> if ($item ne '.' && $item ne '..') {
> >> readDir($item);
> >> }
> >> }
> >> chdir ("..");
> >> closedir (FD_DIR);
> >> } else {
> >> print "Can't open dir $file\n";
> >> }
> >> } elsif (-f $file) {
> >> if ($file =~ /\.$ext$/ || $file =~ /^$filename/i) {
> >> &searchBadWords(&readFile($file), $file);
> >> }
> >> } else {
> >> print "File or directory doesn't exist: $file\n";
> >> }
> >> }
> >>
> >> sub searchBadWords($$) {
> >> my ($content, $file) = @_;
> >> my ($prev, $after, $match, $area);
> >> my $abs_fname = File::Spec->rel2abs($file);
> >> foreach my $word (sort keys %bad_words) {
> >> my $checkable = 0; # false
> >> foreach $area (@{$bad_words{$word}}) {
> >> if (($area eq "all") or (index($abs_fname, $area) ne -1)) {
> >> $checkable = 1; # true
> >> }
> >> }
> >> if ($checkable) {
> >> if ($content =~ m/$word/i) {
> >> $match = $&;
> >> chomp($match);
> >> $prev = $`;
> >> $after = $';
> >> print "File: $abs_fname\n";
> >> print "Bad word: $match\n";
> >> print "Line #" . &getLineNumber($prev) . "\n";
> >> print "String: " . &getString($prev, $after, $match) . "\n";
> >> print "------------------------------------\n";
> >> }
> >> }
> >> }
> >> if ($file =~ /^*\.($ext_src)/) {
> >> if ($content =~ /[\s_]bug[\s_]*(#*)(\d+)/) {
> >> $match = $&;
> >> chomp($match);
> >> $prev = $`;
> >> $after = $';
> >> print "File: $abs_fname\n";
> >> print "Bug mentioning: $match\n";
> >> print "Line #" . &getLineNumber($prev) . "\n";
> >> print "String: " . &getString($prev, $after, $match) . "\n";
> >> print "------------------------------------\n";
> >> }
> >> }
> >> }
> >>
> >> sub readFile($) {
> >> my($file_name) = @_;
> >> my $content = "";
> >> if (open(FH, "< " . $file_name)) {
> >> while (<FH>) {
> >> $content .= $_;
> >> }
> >> close FH;
> >> } else {
> >> printf STDERR "Can't open file $file_name\n";
> >> }
> >> $content;
> >> }
> >>
> >> sub getString($$$) {
> >> my($prev, $after, $match) = @_;
> >> my $last_p = rindex($prev, "\n");
> >> my $first_a = index($after, "\n");
> >> substr($prev, $last_p + 1, length($prev) - $last_p) . $match . substr($after, 0, $first_a);
> >> }
> >>
> >> sub getLineNumber($) {
> >> my($string) = @_;
> >> my $lnum = 1;
> >> while ($string =~ m/\n/g) {
> >> $lnum++;
> >> }
> >> $lnum;
> >> }
> >>
> >> if (scalar(@ARGV) < 2) {
> >> print "Usage: perl check_bad_words.pl <dictionary> <directory or file>";
> >> exit;
> >> }
> >>
> >> &readDictionary($ARGV[0]);
> >> &readDir($ARGV[1]);
> >>
> >>
> >> ------------------------------------------------------------------------
> >>
> >> ---------------------------------------------------------------------
> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
> >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> >> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
> --
>
> Tim Ellison (t.p.ellison@gmail.com)
> IBM Java technology centre, UK.
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org