You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by Juan Felix <ja...@hotmail.com> on 2010/10/27 20:11:45 UTC

http authentication and multicore

Hi.

I'm using nutch 1.2 to crawl a site and after that I want to index to solr using solrIndexer command.

The problem is that the solr server needs a Digest authentication, Is there a way to authenticate from nutch?

Another problem is that I'm using multicore in solr, how do I tell solrIndexer that I'm using a multicore?

Thanks
Juan

RE: http authentication and multicore

Posted by Juan Felix <ja...@hotmail.com>.
It didn't work :(

I'm trying to change the SolrIndexer file to put the credentials but it's not working. Something like this:

      HttpClient adminClient = getHttpClient(solrUrl);
      SolrServer solr =  new CommonsHttpSolrServer(solrUrl, adminClient);
      solr.commit();

I added the method getHttpClient like this:

private HttpClient getHttpClient(String solrServerUrl){
        
        HttpClient adminClient = new HttpClient();

        try {
            HostConfiguration adminHost = adminClient.getHostConfiguration();
            adminHost.setHost(new URI(solrServerUrl, true));
            AuthScope adminScope = new AuthScope(adminHost.getHost(),adminHost.getPort(),null, null);
            adminClient.getState().setCredentials(adminScope,new UsernamePasswordCredentials(myUser, myPwd));
        } 
        catch (Exception e) {
            e.printStackTrace();
        } 
        return adminClient;
    }

> From: markus.jelsma@openindex.io
> To: user@nutch.apache.org
> Subject: Re: http authentication and multicore
> Date: Wed, 27 Oct 2010 20:21:59 +0200
> CC: jafsnano@hotmail.com
> 
> 
> > Hi.
> > 
> > I'm using nutch 1.2 to crawl a site and after that I want to index to solr
> > using solrIndexer command.
> > 
> > The problem is that the solr server needs a Digest authentication, Is there
> > a way to authenticate from nutch?
> 
> Never tried the authentication part, but i guess 
> http://user:pass@host:port/servlet/corename/ will not only do the 
> authentication but also send it to the proper core. At least, i'm sure sending 
> data to the right core works because that's what i do.
> 
> > 
> > Another problem is that I'm using multicore in solr, how do I tell
> > solrIndexer that I'm using a multicore?
> > 
> > Thanks
> > Juan

Re: http authentication and multicore

Posted by Markus Jelsma <ma...@openindex.io>.
> Hi.
> 
> I'm using nutch 1.2 to crawl a site and after that I want to index to solr
> using solrIndexer command.
> 
> The problem is that the solr server needs a Digest authentication, Is there
> a way to authenticate from nutch?

Never tried the authentication part, but i guess 
http://user:pass@host:port/servlet/corename/ will not only do the 
authentication but also send it to the proper core. At least, i'm sure sending 
data to the right core works because that's what i do.

> 
> Another problem is that I'm using multicore in solr, how do I tell
> solrIndexer that I'm using a multicore?
> 
> Thanks
> Juan