[jira] [Closed] (NUTCH-472) NullPointerException in ZipTextExtractor if no MIME type for zipped file

["Markus Jelsma (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/NUTCH-472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Markus Jelsma closed NUTCH-472.
-------------------------------

    Resolution: Won't Fix

> NullPointerException in ZipTextExtractor if no MIME type for zipped file
> ------------------------------------------------------------------------
>
>                 Key: NUTCH-472
>                 URL: https://issues.apache.org/jira/browse/NUTCH-472
>             Project: Nutch
>          Issue Type: Bug
>          Components: indexer
>    Affects Versions: 0.9.0
>         Environment: Any
>            Reporter: Antony Bowesman
>
> extractText throws a NPE in
>           String contentType = MIME.getMimeType(fname).getName();
> if the file in the zip has no configured mime type which breaks the parsing of the zip.
> Code should do:
>   public String extractText(InputStream input, String url, List outLinksList) throws IOException {
>     String resultText = "";
>     byte temp;
>     
>     ZipInputStream zin = new ZipInputStream(input);
>     
>     ZipEntry entry;
>     
>     while ((entry = zin.getNextEntry()) != null) {
>       
>       if (!entry.isDirectory()) {
>         int size = (int) entry.getSize();
>         byte[] b = new byte[size];
>         for(int x = 0; x < size; x++) {
>           int err = zin.read();
>           if(err != -1) {
>             b[x] = (byte)err;
>           }
>         }
>         String newurl = url + "/";
>         String fname = entry.getName();
>         newurl += fname;
>         URL aURL = new URL(newurl);
>         String base = aURL.toString();
>         int i = fname.lastIndexOf('.');
>         if (i != -1) {
>           // Trying to resolve the Mime-Type
>           MimeType mt = MIME.getMimeType(fname);
>           if (mt != null) {
>             String contentType = mt.getName();
>             try {
>               Metadata metadata = new Metadata();
>               metadata.set(Response.CONTENT_LENGTH, Long.toString(entry.getSize()));
>               metadata.set(Response.CONTENT_TYPE, contentType);
>               Content content = new Content(newurl, base, b, contentType, metadata, this.conf);
>               Parse parse = new ParseUtil(this.conf).parse(content);
>               ParseData theParseData = parse.getData();
>               Outlink[] theOutlinks = theParseData.getOutlinks();
>                 
>               for(int count = 0; count < theOutlinks.length; count++) {
>                 outLinksList.add(new Outlink(theOutlinks[count].getToUrl(), theOutlinks[count].getAnchor(), this.conf));
>               }
>               
>               resultText += entry.getName() + " " + parse.getText() + " ";
>             } catch (ParseException e) {
>               if (LOG.isInfoEnabled()) { 
>                LOG.info("fetch okay, but can't parse " + fname + ", reason: " + e.getMessage());
>               }
>             }
>           } else {
>               resultText += entry.getName();
>           }
>         }
>       }
>     }
>     
>     return resultText;
>   }

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira