[whimsy] 01/02: recover if site doesn't exist

[ru...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

rubys pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/whimsy.git

commit 5849de67a464bee2726d2408cf1c1a925830c947
Author: Sam Ruby <ru...@intertwingly.net>
AuthorDate: Thu Sep 21 09:50:07 2017 -0400

    recover if site doesn't exist
---
 tools/site-scan.rb | 20 +++++++++++++++-----
 1 file changed, 15 insertions(+), 5 deletions(-)

diff --git a/tools/site-scan.rb b/tools/site-scan.rb
index 07fa979..8abf30c 100755
--- a/tools/site-scan.rb
+++ b/tools/site-scan.rb
@@ -25,14 +25,12 @@ end
 IMAGE_DIR = ASF::SVN.find('asf/infrastructure/site/trunk/content/img')
 
 def parse(id, site, name)
-  uri, response, status = $cache.get(site.to_s)
-  $stderr.puts "#{id} #{uri} #{status}"
-  doc = Nokogiri::HTML(response)
-
+  uri = URI.parse(site)
+    
   # default data
   data = {
     display_name: name,
-    uri: uri.to_s,
+    uri: site,
     events: nil,
     foundation: nil,
     license: nil,
@@ -43,6 +41,18 @@ def parse(id, site, name)
     image: nil,
   }
 
+  # check if site exists
+  begin
+    Socket.getaddrinfo(uri.host, uri.scheme)
+  rescue SocketError
+    return data
+  end
+
+  uri, response, status = $cache.get(site.to_s)
+  $stderr.puts "#{id} #{uri} #{status}"
+  doc = Nokogiri::HTML(response)
+  data[:uri] = uri.to_s
+
   # scan each link
   doc.css('a').each do |a|
 

-- 
To stop receiving notification emails like this one, please contact
"commits@whimsical.apache.org" <co...@whimsical.apache.org>.