java.lang.StackOverflowError in HTMLMetaProcessor.getMetaTagsHelper

[Siddhartha Reddy <si...@grok.in>]

While parsing a certain page in Nutch, I am getting a
java.lang.StackOverflowError exception due to the recursion in
HTMLMetaProcessor.getMetaTagsHelper.

A copy of the offending page is available at
http://www.grok.in/tmp/f005.html When you look at the HTML source of that
page, it is clear why the StackOverflowError occurs.
HTMLMetaProcessor.getMetaTagsHelper uses recursion to go through the HTML
tree stopping when it encounters a "body" tag. But this page does not have a
body tag at all! Moreover this page does not end most of the HTML tags that
it opens, thus creating a very deep tree.

Such pages, though uncommon, exist in plenty on the Web. Ideally, Nutch
should not choke like this on encountering them. One option is to use
something like java.lang.LinkedList as a queue to traverse the tree without
using recursion. This is how I am currently avoiding the problem. If this
approach is acceptable, I can open a Jira issue and submit a patch.

Best,
Siddhartha