[GitHub] [opennlp] jzonthemtn commented on a diff in pull request #473: OPENNLP-1428 - Enhance DownloadUtil to avoid the use of hard-coded model urls

[GitBox <gi...@apache.org>]

jzonthemtn commented on code in PR #473:
URL: https://github.com/apache/opennlp/pull/473#discussion_r1060051880


##########
opennlp-tools/src/main/java/opennlp/tools/util/DownloadUtil.java:
##########
@@ -174,4 +143,82 @@ public static <T extends BaseModel> T downloadModel(URL url, Class<T> type) thro
     }
   }
 
+  @Internal
+  static class DownloadParser {
+
+    private static final Pattern LINK_PATTERN = Pattern.compile("<a href=\\\"(.*?)\\\">(.*?)</a>", Pattern.CASE_INSENSITIVE | Pattern.DOTALL);
+    private final URL indexUrl;
+
+    DownloadParser(URL indexUrl) {
+      Objects.requireNonNull(indexUrl);
+      this.indexUrl = indexUrl;
+    }
+
+    Map<String, Map<ModelType, String>> getAvailableModels() {
+
+      final Matcher matcher = LINK_PATTERN.matcher(fetchPageIndex());
+
+      final List<String> links = new ArrayList<>();
+      while (matcher.find()) {
+        links.add(matcher.group(1));
+      }
+
+      return toMap(links);
+    }
+
+    private Map<String, Map<ModelType, String>> toMap(List<String> links) {
+
+      final Map<String, Map<ModelType, String>> result = new HashMap<>();
+
+      for (String link : links) {

Review Comment:
   That sounds reasonable. I wrote [OPENNLP-1433](https://issues.apache.org/jira/browse/OPENNLP-1433) for it and referenced this conversation.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@opennlp.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org