You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by Michael Boyar <bo...@gmail.com> on 2014/09/13 04:32:39 UTC
Nutch -> ElasticSearch Authentication
Is it possible to configure Nutch to authenticate to ElasticSearch (when
Nutch updates ES)?
I'd use any version combination, or any plugins and features just to make
the authentication work.
Thank you
Re: Nutch -> ElasticSearch Authentication
Posted by "Jake K. Dodd" <ja...@ontopic.io>.
Perfect, if you already have an authentication solution and know how to work it with the Java API, you're good to go.
Take a look at the open() method in ElasticIndexWriter, that's where the client gets built.
I'd recommend adding any authentication-related configuration options in the ElasticConstants class. Then, you can place authentication-related configuration in the elasticsearch.conf file so that you don't need to rebuild the project if your authentication credentials change.
Cheers
Jake K Dodd
> On Sep 13, 2014, at 9:47, Michael Boyar <bo...@gmail.com> wrote:
>
> My current and preferred ES authentication solution is jetty,
> https://github.com/sonian/elasticsearch-jetty
>
> I'll definitely look into modifying ES index writer plugin in Nutch.
>
> Thanks
>
>> On Sat, Sep 13, 2014 at 12:19 PM, Jake K. Dodd <ja...@ontopic.io> wrote:
>>
>> Unfortunately, elasticsearch does not support authentication/authorization
>> out of the box. This is a limitation of elasticsearch, and not Nutch.
>>
>> However, you have a few options, none of which I can personally recommend
>> because I haven't used them. One is the elasticsearch-http-basic plugin,
>> which you can find on GitHub. It seems to be pretty well maintained.
>> Another option is to put elasticsearch behind a reverse proxy--I know that
>> nginx is a popular solution here.
>>
>> However, before you explore any of those options too deeply, I suggest you
>> confirm that HTTP basic auth is supported in the elasticsearch Java API.
>> After you confirm this, and decide on an authentication system, you'll need
>> to modify the elastic index writer plugin in Nutch.
>>
>> Cheers
>>
>> Jake K Dodd
>>
>>> On Sep 13, 2014, at 4:30, Michael Boyar <bo...@gmail.com> wrote:
>>>
>>> HTTP basic auth would be fine. More generally, any auth would be fine.
>>>
>>> Thanks
>>>
>>>> On Fri, Sep 12, 2014 at 11:48 PM, Jake K. Dodd <ja...@ontopic.io> wrote:
>>>>
>>>> What type of authentication? HTTP basic auth?
>>>>
>>>> Jake K Dodd
>>>>
>>>>> On Sep 12, 2014, at 19:32, Michael Boyar <bo...@gmail.com> wrote:
>>>>>
>>>>> Is it possible to configure Nutch to authenticate to ElasticSearch
>> (when
>>>>> Nutch updates ES)?
>>>>>
>>>>> I'd use any version combination, or any plugins and features just to
>> make
>>>>> the authentication work.
>>>>>
>>>>> Thank you
>>>
>>>
>>>
>>> --
>>> Thanks,
>>>
>>> Mike
>
>
>
> --
> Thanks,
>
> Mike
Re: Nutch -> ElasticSearch Authentication
Posted by Michael Boyar <bo...@gmail.com>.
My current and preferred ES authentication solution is jetty,
https://github.com/sonian/elasticsearch-jetty
I'll definitely look into modifying ES index writer plugin in Nutch.
Thanks
On Sat, Sep 13, 2014 at 12:19 PM, Jake K. Dodd <ja...@ontopic.io> wrote:
> Unfortunately, elasticsearch does not support authentication/authorization
> out of the box. This is a limitation of elasticsearch, and not Nutch.
>
> However, you have a few options, none of which I can personally recommend
> because I haven't used them. One is the elasticsearch-http-basic plugin,
> which you can find on GitHub. It seems to be pretty well maintained.
> Another option is to put elasticsearch behind a reverse proxy--I know that
> nginx is a popular solution here.
>
> However, before you explore any of those options too deeply, I suggest you
> confirm that HTTP basic auth is supported in the elasticsearch Java API.
> After you confirm this, and decide on an authentication system, you'll need
> to modify the elastic index writer plugin in Nutch.
>
> Cheers
>
> Jake K Dodd
>
> > On Sep 13, 2014, at 4:30, Michael Boyar <bo...@gmail.com> wrote:
> >
> > HTTP basic auth would be fine. More generally, any auth would be fine.
> >
> > Thanks
> >
> >> On Fri, Sep 12, 2014 at 11:48 PM, Jake K. Dodd <ja...@ontopic.io> wrote:
> >>
> >> What type of authentication? HTTP basic auth?
> >>
> >> Jake K Dodd
> >>
> >>> On Sep 12, 2014, at 19:32, Michael Boyar <bo...@gmail.com> wrote:
> >>>
> >>> Is it possible to configure Nutch to authenticate to ElasticSearch
> (when
> >>> Nutch updates ES)?
> >>>
> >>> I'd use any version combination, or any plugins and features just to
> make
> >>> the authentication work.
> >>>
> >>> Thank you
> >
> >
> >
> > --
> > Thanks,
> >
> > Mike
>
--
Thanks,
Mike
Re: Nutch -> ElasticSearch Authentication
Posted by "Jake K. Dodd" <ja...@ontopic.io>.
Unfortunately, elasticsearch does not support authentication/authorization out of the box. This is a limitation of elasticsearch, and not Nutch.
However, you have a few options, none of which I can personally recommend because I haven't used them. One is the elasticsearch-http-basic plugin, which you can find on GitHub. It seems to be pretty well maintained. Another option is to put elasticsearch behind a reverse proxy--I know that nginx is a popular solution here.
However, before you explore any of those options too deeply, I suggest you confirm that HTTP basic auth is supported in the elasticsearch Java API. After you confirm this, and decide on an authentication system, you'll need to modify the elastic index writer plugin in Nutch.
Cheers
Jake K Dodd
> On Sep 13, 2014, at 4:30, Michael Boyar <bo...@gmail.com> wrote:
>
> HTTP basic auth would be fine. More generally, any auth would be fine.
>
> Thanks
>
>> On Fri, Sep 12, 2014 at 11:48 PM, Jake K. Dodd <ja...@ontopic.io> wrote:
>>
>> What type of authentication? HTTP basic auth?
>>
>> Jake K Dodd
>>
>>> On Sep 12, 2014, at 19:32, Michael Boyar <bo...@gmail.com> wrote:
>>>
>>> Is it possible to configure Nutch to authenticate to ElasticSearch (when
>>> Nutch updates ES)?
>>>
>>> I'd use any version combination, or any plugins and features just to make
>>> the authentication work.
>>>
>>> Thank you
>
>
>
> --
> Thanks,
>
> Mike
Re: Nutch -> ElasticSearch Authentication
Posted by Michael Boyar <bo...@gmail.com>.
HTTP basic auth would be fine. More generally, any auth would be fine.
Thanks
On Fri, Sep 12, 2014 at 11:48 PM, Jake K. Dodd <ja...@ontopic.io> wrote:
> What type of authentication? HTTP basic auth?
>
> Jake K Dodd
>
> > On Sep 12, 2014, at 19:32, Michael Boyar <bo...@gmail.com> wrote:
> >
> > Is it possible to configure Nutch to authenticate to ElasticSearch (when
> > Nutch updates ES)?
> >
> > I'd use any version combination, or any plugins and features just to make
> > the authentication work.
> >
> > Thank you
>
--
Thanks,
Mike
Re: Nutch -> ElasticSearch Authentication
Posted by "Jake K. Dodd" <ja...@ontopic.io>.
What type of authentication? HTTP basic auth?
Jake K Dodd
> On Sep 12, 2014, at 19:32, Michael Boyar <bo...@gmail.com> wrote:
>
> Is it possible to configure Nutch to authenticate to ElasticSearch (when
> Nutch updates ES)?
>
> I'd use any version combination, or any plugins and features just to make
> the authentication work.
>
> Thank you