You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by sw...@quandatics.com on 2021/10/25 01:21:05 UTC
Encrypt or Mask the password
Hi,
May I know if there is a way to encrypt or mask the password specified in
nutch-site.xml?
Your sincerely,
Shi Wei
Re: Encrypt or Mask the password
Posted by Sebastian Nagel <wa...@googlemail.com.INVALID>.
Hi Shi Wei,
I'm not aware of any work-arounds.
Best,
Sebastian
Am Mo., 25. Okt. 2021 um 11:59 Uhr schrieb <sw...@quandatics.com>:
> Hi Sebastian,
>
> In case you have missed out the previous email, is there any possible
> workaround to integrate Nutch with kerberized Solr cloud? e.g. via the
> HTTP Authentication Scheme
>
> Your sincerely,
> Shi Wei
>
> -----Original Message-----
> From: Sebastian Nagel <wa...@googlemail.com.INVALID>
> Sent: Monday, 25 October, 2021 5:31 PM
> To: user@nutch.apache.org
> Subject: Re: Encrypt or Mask the password
>
> Hi Shi Wei,
>
> for the nutch-site.xml it's possible to use Java properties and/or
> environment variables, see section "Variable expansion" in
>
>
> https://hadoop.apache.org/docs/r3.3.1/api/org/apache/hadoop/conf/Configuration.html
>
> In case you're asking about index-writers.xml - variable expansion
> (likely) does not work.
> Note: I didn't try it. But it's a scheme specific to Nutch and not a
> Hadoop configuration file and I cannot remember that any expansion
> mechanism is implemented when the index-writers.xml is read.
>
> Anyway, the better way would be to rely primarily on a credential
> provider, see
>
>
> https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/CredentialProviderAPI.html
>
>
> https://hadoop.apache.org/docs/stable/api/org/apache/hadoop/conf/Configuration.html#getPassword-java.lang.String-
>
> If you have time, please open a Jira issue (or multiple ones) to get a
> more safe way to hold credentials implemented.
>
> Thanks,
> Sebastian
>
> Am Mo., 25. Okt. 2021 um 03:21 Uhr schrieb <sw...@quandatics.com>:
>
> > Hi,
> >
> >
> >
> > May I know if there is a way to encrypt or mask the password specified
> > in nutch-site.xml?
> >
> >
> >
> > Your sincerely,
> >
> > Shi Wei
> >
> >
>
>
RE: Encrypt or Mask the password
Posted by sw...@quandatics.com.
Hi Sebastian,
In case you have missed out the previous email, is there any possible workaround to integrate Nutch with kerberized Solr cloud? e.g. via the HTTP Authentication Scheme
Your sincerely,
Shi Wei
-----Original Message-----
From: Sebastian Nagel <wa...@googlemail.com.INVALID>
Sent: Monday, 25 October, 2021 5:31 PM
To: user@nutch.apache.org
Subject: Re: Encrypt or Mask the password
Hi Shi Wei,
for the nutch-site.xml it's possible to use Java properties and/or environment variables, see section "Variable expansion" in
https://hadoop.apache.org/docs/r3.3.1/api/org/apache/hadoop/conf/Configuration.html
In case you're asking about index-writers.xml - variable expansion (likely) does not work.
Note: I didn't try it. But it's a scheme specific to Nutch and not a Hadoop configuration file and I cannot remember that any expansion mechanism is implemented when the index-writers.xml is read.
Anyway, the better way would be to rely primarily on a credential provider, see
https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/CredentialProviderAPI.html
https://hadoop.apache.org/docs/stable/api/org/apache/hadoop/conf/Configuration.html#getPassword-java.lang.String-
If you have time, please open a Jira issue (or multiple ones) to get a more safe way to hold credentials implemented.
Thanks,
Sebastian
Am Mo., 25. Okt. 2021 um 03:21 Uhr schrieb <sw...@quandatics.com>:
> Hi,
>
>
>
> May I know if there is a way to encrypt or mask the password specified
> in nutch-site.xml?
>
>
>
> Your sincerely,
>
> Shi Wei
>
>
RE: Encrypt or Mask the password
Posted by sw...@quandatics.com.
Hi Sebastian,
Thanks for your suggestions.
We have tried the variable expansion method on the index-writers.xml, it doesn’t work. Could you advise if there are any alternative ways to encrypt the password in the index-writers.xml file?
Best Regards,
Shi Wei
-----Original Message-----
From: Sebastian Nagel <wa...@googlemail.com.INVALID>
Sent: Monday, 25 October, 2021 5:31 PM
To: user@nutch.apache.org
Subject: Re: Encrypt or Mask the password
Hi Shi Wei,
for the nutch-site.xml it's possible to use Java properties and/or environment variables, see section "Variable expansion" in
https://hadoop.apache.org/docs/r3.3.1/api/org/apache/hadoop/conf/Configuration.html
In case you're asking about index-writers.xml - variable expansion (likely) does not work.
Note: I didn't try it. But it's a scheme specific to Nutch and not a Hadoop configuration file and I cannot remember that any expansion mechanism is implemented when the index-writers.xml is read.
Anyway, the better way would be to rely primarily on a credential provider, see
https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/CredentialProviderAPI.html
https://hadoop.apache.org/docs/stable/api/org/apache/hadoop/conf/Configuration.html#getPassword-java.lang.String-
If you have time, please open a Jira issue (or multiple ones) to get a more safe way to hold credentials implemented.
Thanks,
Sebastian
Am Mo., 25. Okt. 2021 um 03:21 Uhr schrieb <sw...@quandatics.com>:
> Hi,
>
>
>
> May I know if there is a way to encrypt or mask the password specified
> in nutch-site.xml?
>
>
>
> Your sincerely,
>
> Shi Wei
>
>
Re: Encrypt or Mask the password
Posted by Sebastian Nagel <wa...@googlemail.com.INVALID>.
Hi Shi Wei,
for the nutch-site.xml it's possible to use Java properties and/or
environment variables,
see section "Variable expansion" in
https://hadoop.apache.org/docs/r3.3.1/api/org/apache/hadoop/conf/Configuration.html
In case you're asking about index-writers.xml - variable expansion (likely)
does not work.
Note: I didn't try it. But it's a scheme specific to Nutch and not a Hadoop
configuration file
and I cannot remember that any expansion mechanism is implemented when the
index-writers.xml
is read.
Anyway, the better way would be to rely primarily on a credential provider,
see
https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/CredentialProviderAPI.html
https://hadoop.apache.org/docs/stable/api/org/apache/hadoop/conf/Configuration.html#getPassword-java.lang.String-
If you have time, please open a Jira issue (or multiple ones) to get a more
safe way to hold
credentials implemented.
Thanks,
Sebastian
Am Mo., 25. Okt. 2021 um 03:21 Uhr schrieb <sw...@quandatics.com>:
> Hi,
>
>
>
> May I know if there is a way to encrypt or mask the password specified in
> nutch-site.xml?
>
>
>
> Your sincerely,
>
> Shi Wei
>
>