You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@whimsical.apache.org by cu...@apache.org on 2019/05/06 14:04:53 UTC
[whimsy] branch master updated (6f207ce -> f12a00c)
This is an automated email from the ASF dual-hosted git repository.
curcuru pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/whimsy.git.
from 6f207ce Output sum of hits per app header
new d84c2cf Add scanning for ASF::SVN use
new adea285 Strip spaces and have caller provide path
new f12a00c Scan all cgi|rb for ASF::SVN use
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
tools/wwwdocs.rb | 59 +++++++++++++++++++++++++++++++++++++++++++-
www/members/repo-use.cgi | 64 ++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 122 insertions(+), 1 deletion(-)
create mode 100755 www/members/repo-use.cgi
[whimsy] 03/03: Scan all cgi|rb for ASF::SVN use
Posted by cu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
curcuru pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/whimsy.git
commit f12a00c82ee50a2e396ede8a28058af7af7cd8c7
Author: Shane Curcuru <as...@shanecurcuru.org>
AuthorDate: Mon May 6 10:04:44 2019 -0400
Scan all cgi|rb for ASF::SVN use
---
www/members/repo-use.cgi | 64 ++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 64 insertions(+)
diff --git a/www/members/repo-use.cgi b/www/members/repo-use.cgi
new file mode 100755
index 0000000..8dd792d
--- /dev/null
+++ b/www/members/repo-use.cgi
@@ -0,0 +1,64 @@
+#!/usr/bin/env ruby
+PAGETITLE = "Scripts that use ASF::SVN" # Wvisible:tools
+$LOAD_PATH.unshift '/srv/whimsy/lib'
+require 'whimsy/asf'
+require 'wunderbar'
+require 'wunderbar/bootstrap'
+require '../../tools/wwwdocs.rb'
+
+_html do
+ _body? do
+ _whimsy_body(
+ title: PAGETITLE,
+ subtitle: 'Scan all scripts for SVN access',
+ relatedtitle: 'More Useful Links',
+ related: {
+ '/members/log' => 'Full server error and access logs',
+ '/docs' => 'Whimsy code and API documentation',
+ '/status' => 'Whimsy production server status',
+ "https://github.com/apache/whimsy/blob/master/www#{ENV['SCRIPT_NAME']}" => 'See This Source Code'
+ },
+ helpblock: -> {
+ _p 'This scans the whimsy repo for uses of ASF::SVN, either public or private repos.'
+ }
+ ) do
+ priv, pub = read_repository(File.expand_path('../../../repository.yml', __FILE__))
+ priv = build_regexp(priv)
+ pub = build_regexp(pub)
+ scan = scan_dir_svn('../../', [priv, pub])
+ _whimsy_panel_table(title: 'Repo use by script') do
+ _table.table.table_hover do
+ _thead_ do
+ _tr do
+ _th 'Private repos used'
+ _th 'Public repos used'
+ end
+ scan.each do |file, (privlines, publines)|
+ _tbody do
+ _tr_ do
+ _td :colspan => '2' do
+ _code file
+ end
+ end
+ _tr do
+ _td do
+ privlines.each do |l|
+ _ l
+ _br
+ end
+ end
+ _td do
+ publines.each do |l|
+ _ l
+ _br
+ end
+ end
+ end
+ end
+ end
+ end
+ end
+ end
+ end
+ end
+end
\ No newline at end of file
[whimsy] 01/03: Add scanning for ASF::SVN use
Posted by cu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
curcuru pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/whimsy.git
commit d84c2cfc46f8b16aac4f1cbbed62688239b5da6d
Author: Shane Curcuru <as...@shanecurcuru.org>
AuthorDate: Mon May 6 09:21:35 2019 -0400
Add scanning for ASF::SVN use
---
tools/wwwdocs.rb | 60 +++++++++++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 59 insertions(+), 1 deletion(-)
diff --git a/tools/wwwdocs.rb b/tools/wwwdocs.rb
index 238b4cc..09b5169 100755
--- a/tools/wwwdocs.rb
+++ b/tools/wwwdocs.rb
@@ -1,5 +1,7 @@
#!/usr/bin/env ruby
-# Scan all /www scripts for WVisible PAGETITLE and categories
+# Utility function to scan various scripts
+# Docs: for WVisible PAGETITLE and categories in .cgi
+# Repos: for ASF::SVN access in .cgi|rb
$LOAD_PATH.unshift '/srv/whimsy/lib'
require 'whimsy/asf'
SCANDIR = "../www"
@@ -12,6 +14,9 @@ AUTHMAP = { # From whimsy-vm4.apache.org.yaml
'ASF Secretarial Team' => 'text-danger'
}
AUTHPUBLIC = 'glyphicon-eye-open'
+IS_PRIVATE = /\A(private|infra\/infrastructure)/
+ASFSVN = 'ASF::SVN'
+SCANDIRSVN = "../"
# Return [PAGETITLE, [cat,egories] ] after WVisible; or same as !Bogosity error
def scan_file(f)
@@ -75,3 +80,56 @@ def get_annotated_scan(dir)
auth = get_auth()
return annotate_scan(scan, auth)
end
+
+# Read repository.yml for all :svn dirs names to scan for
+# @return [['private1', 'privrepo2', ...], ['public1', 'pubrepo2', ...]
+def read_repository()
+ svn = YAML.load_file('../repository.yml')[:svn]
+ repos = [[], []]
+ svn.each do |repo, data|
+ data['url'] =~ IS_PRIVATE ? repos[0] << repo : repos[1] << repo
+ end
+ return repos
+end
+
+# Build a regex union from ASFSVN and an array
+# @return Regexp.union(r...)
+def build_regexp(list)
+ r = []
+ list.each do |itm|
+ r << "#{ASFSVN}\['#{itm}']"
+ end
+ return Regexp.union(r)
+end
+
+# Scan file for use of ASF::SVN (private or public)
+# @return [["x = ASF::SVN['Meetings'] # Whole line private repo", ...], [] ]
+def scan_file_svn(f, regexs)
+ repos = [[], []]
+ begin
+ File.open(f).each_line.map(&:chomp).each do |line|
+ if line =~ regexs[0] then
+ repos[0] << line
+ elsif line =~ regexs[1] then
+ repos[1] << line
+ end
+ end
+ return repos
+ rescue Exception => e
+ return [["#{ISERR}Bogosity! #{e.message[0..255]}", "\t#{e.backtrace.join("\n\t")}"],[]]
+ end
+end
+
+# Scan directory for use of ASF::SVN (private or public)
+# @return { file: [['private line'], []] }
+def scan_dir_svn(dir, regexs)
+ links = {}
+ Dir["#{dir}/**/*.{cgi,rb}".untaint].each do |f|
+ l = scan_file_svn(f.untaint, regexs)
+ if (l[0].length + l[1].length) > 0
+ links[f.sub(dir, '')] = l
+ end
+ end
+ return links
+end
+
[whimsy] 02/03: Strip spaces and have caller provide path
Posted by cu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
curcuru pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/whimsy.git
commit adea2853370a2b3f990e510222cb5522843d473d
Author: Shane Curcuru <as...@shanecurcuru.org>
AuthorDate: Mon May 6 10:04:29 2019 -0400
Strip spaces and have caller provide path
---
tools/wwwdocs.rb | 9 ++++-----
1 file changed, 4 insertions(+), 5 deletions(-)
diff --git a/tools/wwwdocs.rb b/tools/wwwdocs.rb
index 09b5169..79a3b09 100755
--- a/tools/wwwdocs.rb
+++ b/tools/wwwdocs.rb
@@ -83,8 +83,8 @@ end
# Read repository.yml for all :svn dirs names to scan for
# @return [['private1', 'privrepo2', ...], ['public1', 'pubrepo2', ...]
-def read_repository()
- svn = YAML.load_file('../repository.yml')[:svn]
+def read_repository(repofile)
+ svn = YAML.load_file(repofile)[:svn]
repos = [[], []]
svn.each do |repo, data|
data['url'] =~ IS_PRIVATE ? repos[0] << repo : repos[1] << repo
@@ -109,9 +109,9 @@ def scan_file_svn(f, regexs)
begin
File.open(f).each_line.map(&:chomp).each do |line|
if line =~ regexs[0] then
- repos[0] << line
+ repos[0] << line.strip
elsif line =~ regexs[1] then
- repos[1] << line
+ repos[1] << line.strip
end
end
return repos
@@ -132,4 +132,3 @@ def scan_dir_svn(dir, regexs)
end
return links
end
-