You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "Sebastian Nagel (JIRA)" <ji...@apache.org> on 2018/03/05 09:35:00 UTC

[jira] [Commented] (NUTCH-2310) Protocol-Selenium does not support HTTPS protocol

    [ https://issues.apache.org/jira/browse/NUTCH-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16385853#comment-16385853 ] 

Sebastian Nagel commented on NUTCH-2310:
----------------------------------------

The [plugin.xml|https://github.com/apache/nutch/blob/master/src/plugin/protocol-selenium/plugin.xml] must also list https as supported protocol. That's done by adding:
{noformat}
      <implementation id="org.apache.nutch.protocol.http.Http"
                       class="org.apache.nutch.protocol.http.Http">
           <parameter name="protocolName" value="https"/>
      </implementation>
{noformat}
But it's likely that more points need to be done to support https.

> Protocol-Selenium does not support HTTPS protocol
> -------------------------------------------------
>
>                 Key: NUTCH-2310
>                 URL: https://issues.apache.org/jira/browse/NUTCH-2310
>             Project: Nutch
>          Issue Type: Bug
>          Components: protocol
>    Affects Versions: 1.12
>            Reporter: Joey Hong
>            Priority: Major
>              Labels: easyfix
>             Fix For: 1.15
>
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> The protocol-selenium and protocol-interactiveselenium plugins raise errors whenever there is a URL with the HTTPS protocol.
>  From the source code for those plugins, we can see that HTTP is the only scheme currently accepted, which makes Nutch unable to crawl HTTPS sites with JS using Selenium Webdrivers. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)