You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Jamie Jackson <ja...@gmail.com> on 2016/11/02 18:21:56 UTC
Re: Problem with Password Decryption in Data Import Handler
I'm at a brick wall. Here's the latest status:
Here are some sample commands that I'm using:
*Create the encryptKeyFile and encrypted password:*
encrypter_password='this_is_my_encrypter_password'
plain_db_pw='Oakton153'
cd /var/docker/solr_stage2/credentials/
echo -n "${encrypter_password}" > encpwd.txt
echo -n "${plain_db_pwd}" > plaindbpwd.txt
openssl enc -aes-128-cbc -a -salt -in plaindbpwd.txt -k
"${encrypter_password}"
rm plaindbpwd.txt
That generated this as the password, by the way:
U2FsdGVkX19pBVTeZaSl43gFFAlrx+Th1zSg1GvlX9o=
*Configure DIH configuration:*
<dataConfig>
<dataSource
driver="org.mariadb.jdbc.Driver"
url="jdbc:mysql://local.mysite.com:3306/mysite"
user="root"
password="U2FsdGVkX19pBVTeZaSl43gFFAlrx+Th1zSg1GvlX9o="
encryptKeyFile="/opt/solr/credentials/encpwd.txt"
/>
...
By the way, /var/docker/solr_stage2/credentials/ is mapped to
/opt/solr/credentials/ in the docker container, so that's why the paths
*seem* different (but aren't, really).
*Authentication error when data import is run:*
Exception while processing: question document :
SolrInputDocument(fields:
[]):org.apache.solr.handler.dataimport.DataImportHandlerException:
Unable to execute query: select 'foo' as bar; Processing
Document # 1
at org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:69)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:323)
at org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:283)
at org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:52)
at org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:59)
at org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:73)
at org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:244)
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:475)
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:414)
at org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:329)
at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:232)
at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:416)
at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:480)
at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:461)
Caused by: java.sql.SQLInvalidAuthorizationSpecException: Could not
connect: Access denied for user 'root'@'ICZ2002912' (using password:
NO)
at org.mariadb.jdbc.internal.util.ExceptionMapper.get(ExceptionMapper.java:123)
at org.mariadb.jdbc.internal.util.ExceptionMapper.throwException(ExceptionMapper.java:71)
at org.mariadb.jdbc.Driver.connect(Driver.java:109)
at org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSource.java:192)
at org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSource.java:172)
at org.apache.solr.handler.dataimport.JdbcDataSource.getConnection(JdbcDataSource.java:503)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:313)
... 12 more
Caused by: org.mariadb.jdbc.internal.util.dao.QueryException: Could
not connect: Access denied for user 'root'@'ICZ2002912' (using
password: NO)
at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.authentication(AbstractConnectProtocol.java:524)
at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.handleConnectionPhases(AbstractConnectProtocol.java:472)
at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.connect(AbstractConnectProtocol.java:374)
at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.connectWithoutProxy(AbstractConnectProtocol.java:763)
at org.mariadb.jdbc.internal.util.Utils.retrieveProxy(Utils.java:469)
at org.mariadb.jdbc.Driver.connect(Driver.java:104)
... 16 more
On Thu, Oct 6, 2016 at 2:42 PM, Jamie Jackson <ja...@gmail.com> wrote:
> It happens to be ten characters.
>
> On Thu, Oct 6, 2016 at 12:44 PM, Alexandre Rafalovitch <arafalov@gmail.com
> > wrote:
>
>> How long is the encryption key (file content)? Because the code I am
>> looking at seems to expect it to be at most 100 characters.
>>
>> Regards,
>> Alex.
>> ----
>> Newsletter and resources for Solr beginners and intermediates:
>> http://www.solr-start.com/
>>
>>
>> On 6 October 2016 at 23:26, Kevin Risden <co...@gmail.com>
>> wrote:
>> > I haven't tried this but is it possible there is a new line at the end
>> in
>> > the file?
>> >
>> > If you did something like echo "" > file.txt then there would be a new
>> > line. Use echo -n "" > file.txt
>> >
>> > Also you should be able to check how many characters are in the file.
>> >
>> > Kevin Risden
>> >
>> > On Wed, Oct 5, 2016 at 5:00 PM, Jamie Jackson <ja...@gmail.com>
>> wrote:
>> >
>> >> Hi Folks,
>> >>
>> >> (Using Solr 5.5.3.)
>> >>
>> >> As far as I know, the only place where encrypted password use is
>> documented
>> >> is in
>> >> https://cwiki.apache.org/confluence/display/solr/
>> >> Uploading+Structured+Data+Store+Data+with+the+Data+Import+Handler,
>> >> under the "Configuring the DIH Configuration File", in a comment in the
>> >> sample XML file:
>> >>
>> >> <!--
>> >> Alternately the password can be encrypted as follows. This is the value
>> >> obtained as a result of the command
>> >> openssl enc -aes-128-cbc -a -salt -in pwd.txt
>> >> password="U2FsdGVkX18QMjY0yfCqlfBMvAB4d3XkwY96L7gfO2o="
>> >> WHen the password is encrypted, you must provide an extra attribute
>> >> encryptKeyFile="/location/of/encryptionkey"
>> >> This file should a text file with a single line containing the
>> >> encrypt/decrypt password
>> >> -->
>> >>
>> >> Anyway, I can encrypt just fine:
>> >>
>> >> $ openssl enc -aes-128-cbc -a -salt -in stgps.txt
>> >> enter aes-128-cbc encryption password:
>> >> Verifying - enter aes-128-cbc encryption password:
>> >> U2FsdGVkX1+VtVoQtmEREvB5qZjn3131+N4jRXmjyIY=
>> >>
>> >>
>> >> I can also decrypt just fine from the command line.
>> >>
>> >> However, if I use the encrypted password and encryptKeyFile in the
>> config
>> >> file, I end up with an error: "String length must be a multiple of
>> four."
>> >>
>> >> https://gist.github.com/jamiejackson/3852dacb03432328ea187d43ade5e4d9
>> >>
>> >> How do I get this working?
>> >>
>> >> Thanks,
>> >> Jamie
>> >>
>>
>
>
Re: Problem with Password Decryption in Data Import Handler
Posted by Jamie Jackson <ja...@gmail.com>.
You were right, Fuad. There was a flaw in my script (inconsistent naming of
the `plain_db_pwd` variable.
Thanks for figuring that out.
For posterity, here's the fixed script:
################################################################################
encrypt_key=your_encryption_key
plain_db_pwd=your_db_password
cred_dir=/your/credentials/directory
cd "${cred_dir}
echo -n "${encrypt_key}" > encrypt.key
echo -n "${plain_db_pwd}" | openssl enc -aes-128-cbc -a -salt -k
"${encrypt_key}"
#==============================================================================#
Then, in the DIH config:
encryptKeyFile="/your/credentials/directory/encrypt.key"
I have another, semi-related, issue that I'll bring up in another thread.
Thanks,
Jamie
On Wed, Nov 2, 2016 at 6:26 PM, Fuad Efendi <fu...@efendi.ca> wrote:
> Then I can only guess that in current configuration decrypted password is
> empty string.
>
> Try to manually replace some characters in encpwd.txt file to see if you
> get different errors; try to delete this file completely to see if you get
> different errors. Try to add new line in this file; try to change password
> in config file.
>
>
>
> On November 2, 2016 at 5:23:33 PM, Jamie Jackson (jamiejaxon@gmail.com)
> wrote:
>
> I should have mentioned that I verified connectivity with plain passwords:
>
> From the same machine that Solr's running on:
>
> solr@000650cbdd5e:/opt/solr$ mysql -uroot -pOakton153 -h local.mysite.com
> mysite -e "select 'foo' as bar;"
> +-----+
> | bar |
> +-----+
> | foo |
> +-----+
>
> Also, if I add the plain-text password to the config, it connects fine:
>
> <dataSource
> driver="org.mariadb.jdbc.Driver"
> url="jdbc:mysql://local.mysite.com:3306/mysite"
> user="root"
> password="Oakton153"
> />
>
>
> So that is why I claim to have a problem with encryptKeyFile, specifically,
> because I've eliminated general connectivity/authentication problems.
>
> Thanks,
> Jamie
>
> On Wed, Nov 2, 2016 at 4:58 PM, Fuad Efendi <fu...@efendi.ca> wrote:
>
> > In MySQL, this command will explicitly allow to connect from
> > remote ICZ2002912 host, check MySQL documentation:
> >
> > GRANT ALL ON mysite.* TO 'root’@'ICZ2002912' IDENTIFIED BY ‘Oakton123’;
> >
> >
> >
> > On November 2, 2016 at 4:41:48 PM, Fuad Efendi (fuad@efendi.ca) wrote:
> >
> > This is the root of the problem:
> > "Access denied for user 'root'@'ICZ2002912' (using password: NO) “
> >
> >
> > First of all, ensure that plain (non-encrypted) password settings work
> for
> > you.
> >
> > Check that you can connect using MySQL client from ICZ2002912 to your
> > MySQL & Co. instance
> >
> > I suspect you need to allow MySQL & Co. to accept connections
> > from ICZ2002912. Plus, check DNS resolution, etc.
> >
> >
> > Thanks,
> >
> >
> > --
> > Fuad Efendi
> > (416) 993-2060
> > http://www.tokenizer.ca
> > Recommender Systems
> >
> >
> > On November 2, 2016 at 2:37:08 PM, Jamie Jackson (jamiejaxon@gmail.com)
> > wrote:
> >
> > I'm at a brick wall. Here's the latest status:
> >
> > Here are some sample commands that I'm using:
> >
> > *Create the encryptKeyFile and encrypted password:*
> >
> >
> > encrypter_password='this_is_my_encrypter_password'
> > plain_db_pw='Oakton153'
> >
> > cd /var/docker/solr_stage2/credentials/
> > echo -n "${encrypter_password}" > encpwd.txt
> > echo -n "${plain_db_pwd}" > plaindbpwd.txt
> > openssl enc -aes-128-cbc -a -salt -in plaindbpwd.txt -k
> > "${encrypter_password}"
> >
> > rm plaindbpwd.txt
> >
> > That generated this as the password, by the way:
> >
> > U2FsdGVkX19pBVTeZaSl43gFFAlrx+Th1zSg1GvlX9o=
> >
> > *Configure DIH configuration:*
> >
> > <dataConfig>
> >
> > <dataSource
> > driver="org.mariadb.jdbc.Driver"
> > url="jdbc:mysql://local.mysite.com:3306/mysite"
> > user="root"
> > password="U2FsdGVkX19pBVTeZaSl43gFFAlrx+Th1zSg1GvlX9o="
> > encryptKeyFile="/opt/solr/credentials/encpwd.txt"
> > />
> > ...
> >
> >
> > By the way, /var/docker/solr_stage2/credentials/ is mapped to
> > /opt/solr/credentials/ in the docker container, so that's why the paths
> > *seem* different (but aren't, really).
> >
> >
> > *Authentication error when data import is run:*
> >
> > Exception while processing: question document :
> > SolrInputDocument(fields:
> > []):org.apache.solr.handler.dataimport.DataImportHandlerException:
> > Unable to execute query: select 'foo' as bar; Processing
> > Document # 1
> > at org.apache.solr.handler.dataimport.DataImportHandlerException.
> > wrapAndThrow(DataImportHandlerException.java:69)
> > at org.apache.solr.handler.dataimport.JdbcDataSource$
> > ResultSetIterator.<init>(JdbcDataSource.java:323)
> > at org.apache.solr.handler.dataimport.JdbcDataSource.
> > getData(JdbcDataSource.java:283)
> > at org.apache.solr.handler.dataimport.JdbcDataSource.
> > getData(JdbcDataSource.java:52)
> > at org.apache.solr.handler.dataimport.SqlEntityProcessor.
> > initQuery(SqlEntityProcessor.java:59)
> > at org.apache.solr.handler.dataimport.SqlEntityProcessor.
> > nextRow(SqlEntityProcessor.java:73)
> > at org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(
> > EntityProcessorWrapper.java:244)
> > at org.apache.solr.handler.dataimport.DocBuilder.
> > buildDocument(DocBuilder.java:475)
> > at org.apache.solr.handler.dataimport.DocBuilder.
> > buildDocument(DocBuilder.java:414)
> > at org.apache.solr.handler.dataimport.DocBuilder.
> > doFullDump(DocBuilder.java:329)
> > at org.apache.solr.handler.dataimport.DocBuilder.execute(
> > DocBuilder.java:232)
> > at org.apache.solr.handler.dataimport.DataImporter.
> > doFullImport(DataImporter.java:416)
> > at org.apache.solr.handler.dataimport.DataImporter.
> > runCmd(DataImporter.java:480)
> > at org.apache.solr.handler.dataimport.DataImporter$1.run(
> > DataImporter.java:461)
> > Caused by: java.sql.SQLInvalidAuthorizationSpecException: Could not
> > connect: Access denied for user 'root'@'ICZ2002912' (using password:
> > NO)
> > at org.mariadb.jdbc.internal.util.ExceptionMapper.get(
> > ExceptionMapper.java:123)
> > at org.mariadb.jdbc.internal.util.ExceptionMapper.throwException(
> > ExceptionMapper.java:71)
> > at org.mariadb.jdbc.Driver.connect(Driver.java:109)
> > at org.apache.solr.handler.dataimport.JdbcDataSource$1.
> > call(JdbcDataSource.java:192)
> > at org.apache.solr.handler.dataimport.JdbcDataSource$1.
> > call(JdbcDataSource.java:172)
> > at org.apache.solr.handler.dataimport.JdbcDataSource.
> > getConnection(JdbcDataSource.java:503)
> > at org.apache.solr.handler.dataimport.JdbcDataSource$
> > ResultSetIterator.<init>(JdbcDataSource.java:313)
> > ... 12 more
> > Caused by: org.mariadb.jdbc.internal.util.dao.QueryException: Could
> > not connect: Access denied for user 'root'@'ICZ2002912' (using
> > password: NO)
> > at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.
> > authentication(AbstractConnectProtocol.java:524)
> > at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.
> > handleConnectionPhases(AbstractConnectProtocol.java:472)
> > at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.connect(
> > AbstractConnectProtocol.java:374)
> > at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.
> > connectWithoutProxy(AbstractConnectProtocol.java:763)
> > at org.mariadb.jdbc.internal.util.Utils.retrieveProxy(Utils.java:469)
> > at org.mariadb.jdbc.Driver.connect(Driver.java:104)
> > ... 16 more
> >
> >
> >
> > On Thu, Oct 6, 2016 at 2:42 PM, Jamie Jackson <ja...@gmail.com>
> > wrote:
> >
> > > It happens to be ten characters.
> > >
> > > On Thu, Oct 6, 2016 at 12:44 PM, Alexandre Rafalovitch <
> > arafalov@gmail.com
> > > > wrote:
> > >
> > >> How long is the encryption key (file content)? Because the code I am
> > >> looking at seems to expect it to be at most 100 characters.
> > >>
> > >> Regards,
> > >> Alex.
> > >> ----
> > >> Newsletter and resources for Solr beginners and intermediates:
> > >> http://www.solr-start.com/
> > >>
> > >>
> > >> On 6 October 2016 at 23:26, Kevin Risden <co...@gmail.com>
> > >> wrote:
> > >> > I haven't tried this but is it possible there is a new line at the
> end
> > >> in
> > >> > the file?
> > >> >
> > >> > If you did something like echo "" > file.txt then there would be a
> new
> > >> > line. Use echo -n "" > file.txt
> > >> >
> > >> > Also you should be able to check how many characters are in the
> file.
> > >> >
> > >> > Kevin Risden
> > >> >
> > >> > On Wed, Oct 5, 2016 at 5:00 PM, Jamie Jackson <jamiejaxon@gmail.com
> >
> > >> wrote:
> > >> >
> > >> >> Hi Folks,
> > >> >>
> > >> >> (Using Solr 5.5.3.)
> > >> >>
> > >> >> As far as I know, the only place where encrypted password use is
> > >> documented
> > >> >> is in
> > >> >> https://cwiki.apache.org/confluence/display/solr/
> > >> >> Uploading+Structured+Data+Store+Data+with+the+Data+Import+Handler,
> > >> >> under the "Configuring the DIH Configuration File", in a comment in
> > the
> > >> >> sample XML file:
> > >> >>
> > >> >> <!--
> > >> >> Alternately the password can be encrypted as follows. This is the
> > value
> > >> >> obtained as a result of the command
> > >> >> openssl enc -aes-128-cbc -a -salt -in pwd.txt
> > >> >> password="U2FsdGVkX18QMjY0yfCqlfBMvAB4d3XkwY96L7gfO2o="
> > >> >> WHen the password is encrypted, you must provide an extra attribute
> > >> >> encryptKeyFile="/location/of/encryptionkey"
> > >> >> This file should a text file with a single line containing the
> > >> >> encrypt/decrypt password
> > >> >> -->
> > >> >>
> > >> >> Anyway, I can encrypt just fine:
> > >> >>
> > >> >> $ openssl enc -aes-128-cbc -a -salt -in stgps.txt
> > >> >> enter aes-128-cbc encryption password:
> > >> >> Verifying - enter aes-128-cbc encryption password:
> > >> >> U2FsdGVkX1+VtVoQtmEREvB5qZjn3131+N4jRXmjyIY=
> > >> >>
> > >> >>
> > >> >> I can also decrypt just fine from the command line.
> > >> >>
> > >> >> However, if I use the encrypted password and encryptKeyFile in the
> > >> config
> > >> >> file, I end up with an error: "String length must be a multiple of
> > >> four."
> > >> >>
> > >> >> https://gist.github.com/jamiejackson/
> 3852dacb03432328ea187d43ade5e4
> > d9
> > >> >>
> > >> >> How do I get this working?
> > >> >>
> > >> >> Thanks,
> > >> >> Jamie
> > >> >>
> > >>
> > >
> > >
> >
>
Re: Problem with Password Decryption in Data Import Handler
Posted by Fuad Efendi <fu...@efendi.ca>.
Then I can only guess that in current configuration decrypted password is empty string.
Try to manually replace some characters in encpwd.txt file to see if you get different errors; try to delete this file completely to see if you get different errors. Try to add new line in this file; try to change password in config file.
On November 2, 2016 at 5:23:33 PM, Jamie Jackson (jamiejaxon@gmail.com) wrote:
I should have mentioned that I verified connectivity with plain passwords:
From the same machine that Solr's running on:
solr@000650cbdd5e:/opt/solr$ mysql -uroot -pOakton153 -h local.mysite.com
mysite -e "select 'foo' as bar;"
+-----+
| bar |
+-----+
| foo |
+-----+
Also, if I add the plain-text password to the config, it connects fine:
<dataSource
driver="org.mariadb.jdbc.Driver"
url="jdbc:mysql://local.mysite.com:3306/mysite"
user="root"
password="Oakton153"
/>
So that is why I claim to have a problem with encryptKeyFile, specifically,
because I've eliminated general connectivity/authentication problems.
Thanks,
Jamie
On Wed, Nov 2, 2016 at 4:58 PM, Fuad Efendi <fu...@efendi.ca> wrote:
> In MySQL, this command will explicitly allow to connect from
> remote ICZ2002912 host, check MySQL documentation:
>
> GRANT ALL ON mysite.* TO 'root’@'ICZ2002912' IDENTIFIED BY ‘Oakton123’;
>
>
>
> On November 2, 2016 at 4:41:48 PM, Fuad Efendi (fuad@efendi.ca) wrote:
>
> This is the root of the problem:
> "Access denied for user 'root'@'ICZ2002912' (using password: NO) “
>
>
> First of all, ensure that plain (non-encrypted) password settings work for
> you.
>
> Check that you can connect using MySQL client from ICZ2002912 to your
> MySQL & Co. instance
>
> I suspect you need to allow MySQL & Co. to accept connections
> from ICZ2002912. Plus, check DNS resolution, etc.
>
>
> Thanks,
>
>
> --
> Fuad Efendi
> (416) 993-2060
> http://www.tokenizer.ca
> Recommender Systems
>
>
> On November 2, 2016 at 2:37:08 PM, Jamie Jackson (jamiejaxon@gmail.com)
> wrote:
>
> I'm at a brick wall. Here's the latest status:
>
> Here are some sample commands that I'm using:
>
> *Create the encryptKeyFile and encrypted password:*
>
>
> encrypter_password='this_is_my_encrypter_password'
> plain_db_pw='Oakton153'
>
> cd /var/docker/solr_stage2/credentials/
> echo -n "${encrypter_password}" > encpwd.txt
> echo -n "${plain_db_pwd}" > plaindbpwd.txt
> openssl enc -aes-128-cbc -a -salt -in plaindbpwd.txt -k
> "${encrypter_password}"
>
> rm plaindbpwd.txt
>
> That generated this as the password, by the way:
>
> U2FsdGVkX19pBVTeZaSl43gFFAlrx+Th1zSg1GvlX9o=
>
> *Configure DIH configuration:*
>
> <dataConfig>
>
> <dataSource
> driver="org.mariadb.jdbc.Driver"
> url="jdbc:mysql://local.mysite.com:3306/mysite"
> user="root"
> password="U2FsdGVkX19pBVTeZaSl43gFFAlrx+Th1zSg1GvlX9o="
> encryptKeyFile="/opt/solr/credentials/encpwd.txt"
> />
> ...
>
>
> By the way, /var/docker/solr_stage2/credentials/ is mapped to
> /opt/solr/credentials/ in the docker container, so that's why the paths
> *seem* different (but aren't, really).
>
>
> *Authentication error when data import is run:*
>
> Exception while processing: question document :
> SolrInputDocument(fields:
> []):org.apache.solr.handler.dataimport.DataImportHandlerException:
> Unable to execute query: select 'foo' as bar; Processing
> Document # 1
> at org.apache.solr.handler.dataimport.DataImportHandlerException.
> wrapAndThrow(DataImportHandlerException.java:69)
> at org.apache.solr.handler.dataimport.JdbcDataSource$
> ResultSetIterator.<init>(JdbcDataSource.java:323)
> at org.apache.solr.handler.dataimport.JdbcDataSource.
> getData(JdbcDataSource.java:283)
> at org.apache.solr.handler.dataimport.JdbcDataSource.
> getData(JdbcDataSource.java:52)
> at org.apache.solr.handler.dataimport.SqlEntityProcessor.
> initQuery(SqlEntityProcessor.java:59)
> at org.apache.solr.handler.dataimport.SqlEntityProcessor.
> nextRow(SqlEntityProcessor.java:73)
> at org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(
> EntityProcessorWrapper.java:244)
> at org.apache.solr.handler.dataimport.DocBuilder.
> buildDocument(DocBuilder.java:475)
> at org.apache.solr.handler.dataimport.DocBuilder.
> buildDocument(DocBuilder.java:414)
> at org.apache.solr.handler.dataimport.DocBuilder.
> doFullDump(DocBuilder.java:329)
> at org.apache.solr.handler.dataimport.DocBuilder.execute(
> DocBuilder.java:232)
> at org.apache.solr.handler.dataimport.DataImporter.
> doFullImport(DataImporter.java:416)
> at org.apache.solr.handler.dataimport.DataImporter.
> runCmd(DataImporter.java:480)
> at org.apache.solr.handler.dataimport.DataImporter$1.run(
> DataImporter.java:461)
> Caused by: java.sql.SQLInvalidAuthorizationSpecException: Could not
> connect: Access denied for user 'root'@'ICZ2002912' (using password:
> NO)
> at org.mariadb.jdbc.internal.util.ExceptionMapper.get(
> ExceptionMapper.java:123)
> at org.mariadb.jdbc.internal.util.ExceptionMapper.throwException(
> ExceptionMapper.java:71)
> at org.mariadb.jdbc.Driver.connect(Driver.java:109)
> at org.apache.solr.handler.dataimport.JdbcDataSource$1.
> call(JdbcDataSource.java:192)
> at org.apache.solr.handler.dataimport.JdbcDataSource$1.
> call(JdbcDataSource.java:172)
> at org.apache.solr.handler.dataimport.JdbcDataSource.
> getConnection(JdbcDataSource.java:503)
> at org.apache.solr.handler.dataimport.JdbcDataSource$
> ResultSetIterator.<init>(JdbcDataSource.java:313)
> ... 12 more
> Caused by: org.mariadb.jdbc.internal.util.dao.QueryException: Could
> not connect: Access denied for user 'root'@'ICZ2002912' (using
> password: NO)
> at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.
> authentication(AbstractConnectProtocol.java:524)
> at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.
> handleConnectionPhases(AbstractConnectProtocol.java:472)
> at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.connect(
> AbstractConnectProtocol.java:374)
> at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.
> connectWithoutProxy(AbstractConnectProtocol.java:763)
> at org.mariadb.jdbc.internal.util.Utils.retrieveProxy(Utils.java:469)
> at org.mariadb.jdbc.Driver.connect(Driver.java:104)
> ... 16 more
>
>
>
> On Thu, Oct 6, 2016 at 2:42 PM, Jamie Jackson <ja...@gmail.com>
> wrote:
>
> > It happens to be ten characters.
> >
> > On Thu, Oct 6, 2016 at 12:44 PM, Alexandre Rafalovitch <
> arafalov@gmail.com
> > > wrote:
> >
> >> How long is the encryption key (file content)? Because the code I am
> >> looking at seems to expect it to be at most 100 characters.
> >>
> >> Regards,
> >> Alex.
> >> ----
> >> Newsletter and resources for Solr beginners and intermediates:
> >> http://www.solr-start.com/
> >>
> >>
> >> On 6 October 2016 at 23:26, Kevin Risden <co...@gmail.com>
> >> wrote:
> >> > I haven't tried this but is it possible there is a new line at the end
> >> in
> >> > the file?
> >> >
> >> > If you did something like echo "" > file.txt then there would be a new
> >> > line. Use echo -n "" > file.txt
> >> >
> >> > Also you should be able to check how many characters are in the file.
> >> >
> >> > Kevin Risden
> >> >
> >> > On Wed, Oct 5, 2016 at 5:00 PM, Jamie Jackson <ja...@gmail.com>
> >> wrote:
> >> >
> >> >> Hi Folks,
> >> >>
> >> >> (Using Solr 5.5.3.)
> >> >>
> >> >> As far as I know, the only place where encrypted password use is
> >> documented
> >> >> is in
> >> >> https://cwiki.apache.org/confluence/display/solr/
> >> >> Uploading+Structured+Data+Store+Data+with+the+Data+Import+Handler,
> >> >> under the "Configuring the DIH Configuration File", in a comment in
> the
> >> >> sample XML file:
> >> >>
> >> >> <!--
> >> >> Alternately the password can be encrypted as follows. This is the
> value
> >> >> obtained as a result of the command
> >> >> openssl enc -aes-128-cbc -a -salt -in pwd.txt
> >> >> password="U2FsdGVkX18QMjY0yfCqlfBMvAB4d3XkwY96L7gfO2o="
> >> >> WHen the password is encrypted, you must provide an extra attribute
> >> >> encryptKeyFile="/location/of/encryptionkey"
> >> >> This file should a text file with a single line containing the
> >> >> encrypt/decrypt password
> >> >> -->
> >> >>
> >> >> Anyway, I can encrypt just fine:
> >> >>
> >> >> $ openssl enc -aes-128-cbc -a -salt -in stgps.txt
> >> >> enter aes-128-cbc encryption password:
> >> >> Verifying - enter aes-128-cbc encryption password:
> >> >> U2FsdGVkX1+VtVoQtmEREvB5qZjn3131+N4jRXmjyIY=
> >> >>
> >> >>
> >> >> I can also decrypt just fine from the command line.
> >> >>
> >> >> However, if I use the encrypted password and encryptKeyFile in the
> >> config
> >> >> file, I end up with an error: "String length must be a multiple of
> >> four."
> >> >>
> >> >> https://gist.github.com/jamiejackson/3852dacb03432328ea187d43ade5e4
> d9
> >> >>
> >> >> How do I get this working?
> >> >>
> >> >> Thanks,
> >> >> Jamie
> >> >>
> >>
> >
> >
>
Re: Problem with Password Decryption in Data Import Handler
Posted by Jamie Jackson <ja...@gmail.com>.
I should have mentioned that I verified connectivity with plain passwords:
From the same machine that Solr's running on:
solr@000650cbdd5e:/opt/solr$ mysql -uroot -pOakton153 -h local.mysite.com
mysite -e "select 'foo' as bar;"
+-----+
| bar |
+-----+
| foo |
+-----+
Also, if I add the plain-text password to the config, it connects fine:
<dataSource
driver="org.mariadb.jdbc.Driver"
url="jdbc:mysql://local.mysite.com:3306/mysite"
user="root"
password="Oakton153"
/>
So that is why I claim to have a problem with encryptKeyFile, specifically,
because I've eliminated general connectivity/authentication problems.
Thanks,
Jamie
On Wed, Nov 2, 2016 at 4:58 PM, Fuad Efendi <fu...@efendi.ca> wrote:
> In MySQL, this command will explicitly allow to connect from
> remote ICZ2002912 host, check MySQL documentation:
>
> GRANT ALL ON mysite.* TO 'root’@'ICZ2002912' IDENTIFIED BY ‘Oakton123’;
>
>
>
> On November 2, 2016 at 4:41:48 PM, Fuad Efendi (fuad@efendi.ca) wrote:
>
> This is the root of the problem:
> "Access denied for user 'root'@'ICZ2002912' (using password: NO) “
>
>
> First of all, ensure that plain (non-encrypted) password settings work for
> you.
>
> Check that you can connect using MySQL client from ICZ2002912 to your
> MySQL & Co. instance
>
> I suspect you need to allow MySQL & Co. to accept connections
> from ICZ2002912. Plus, check DNS resolution, etc.
>
>
> Thanks,
>
>
> --
> Fuad Efendi
> (416) 993-2060
> http://www.tokenizer.ca
> Recommender Systems
>
>
> On November 2, 2016 at 2:37:08 PM, Jamie Jackson (jamiejaxon@gmail.com)
> wrote:
>
> I'm at a brick wall. Here's the latest status:
>
> Here are some sample commands that I'm using:
>
> *Create the encryptKeyFile and encrypted password:*
>
>
> encrypter_password='this_is_my_encrypter_password'
> plain_db_pw='Oakton153'
>
> cd /var/docker/solr_stage2/credentials/
> echo -n "${encrypter_password}" > encpwd.txt
> echo -n "${plain_db_pwd}" > plaindbpwd.txt
> openssl enc -aes-128-cbc -a -salt -in plaindbpwd.txt -k
> "${encrypter_password}"
>
> rm plaindbpwd.txt
>
> That generated this as the password, by the way:
>
> U2FsdGVkX19pBVTeZaSl43gFFAlrx+Th1zSg1GvlX9o=
>
> *Configure DIH configuration:*
>
> <dataConfig>
>
> <dataSource
> driver="org.mariadb.jdbc.Driver"
> url="jdbc:mysql://local.mysite.com:3306/mysite"
> user="root"
> password="U2FsdGVkX19pBVTeZaSl43gFFAlrx+Th1zSg1GvlX9o="
> encryptKeyFile="/opt/solr/credentials/encpwd.txt"
> />
> ...
>
>
> By the way, /var/docker/solr_stage2/credentials/ is mapped to
> /opt/solr/credentials/ in the docker container, so that's why the paths
> *seem* different (but aren't, really).
>
>
> *Authentication error when data import is run:*
>
> Exception while processing: question document :
> SolrInputDocument(fields:
> []):org.apache.solr.handler.dataimport.DataImportHandlerException:
> Unable to execute query: select 'foo' as bar; Processing
> Document # 1
> at org.apache.solr.handler.dataimport.DataImportHandlerException.
> wrapAndThrow(DataImportHandlerException.java:69)
> at org.apache.solr.handler.dataimport.JdbcDataSource$
> ResultSetIterator.<init>(JdbcDataSource.java:323)
> at org.apache.solr.handler.dataimport.JdbcDataSource.
> getData(JdbcDataSource.java:283)
> at org.apache.solr.handler.dataimport.JdbcDataSource.
> getData(JdbcDataSource.java:52)
> at org.apache.solr.handler.dataimport.SqlEntityProcessor.
> initQuery(SqlEntityProcessor.java:59)
> at org.apache.solr.handler.dataimport.SqlEntityProcessor.
> nextRow(SqlEntityProcessor.java:73)
> at org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(
> EntityProcessorWrapper.java:244)
> at org.apache.solr.handler.dataimport.DocBuilder.
> buildDocument(DocBuilder.java:475)
> at org.apache.solr.handler.dataimport.DocBuilder.
> buildDocument(DocBuilder.java:414)
> at org.apache.solr.handler.dataimport.DocBuilder.
> doFullDump(DocBuilder.java:329)
> at org.apache.solr.handler.dataimport.DocBuilder.execute(
> DocBuilder.java:232)
> at org.apache.solr.handler.dataimport.DataImporter.
> doFullImport(DataImporter.java:416)
> at org.apache.solr.handler.dataimport.DataImporter.
> runCmd(DataImporter.java:480)
> at org.apache.solr.handler.dataimport.DataImporter$1.run(
> DataImporter.java:461)
> Caused by: java.sql.SQLInvalidAuthorizationSpecException: Could not
> connect: Access denied for user 'root'@'ICZ2002912' (using password:
> NO)
> at org.mariadb.jdbc.internal.util.ExceptionMapper.get(
> ExceptionMapper.java:123)
> at org.mariadb.jdbc.internal.util.ExceptionMapper.throwException(
> ExceptionMapper.java:71)
> at org.mariadb.jdbc.Driver.connect(Driver.java:109)
> at org.apache.solr.handler.dataimport.JdbcDataSource$1.
> call(JdbcDataSource.java:192)
> at org.apache.solr.handler.dataimport.JdbcDataSource$1.
> call(JdbcDataSource.java:172)
> at org.apache.solr.handler.dataimport.JdbcDataSource.
> getConnection(JdbcDataSource.java:503)
> at org.apache.solr.handler.dataimport.JdbcDataSource$
> ResultSetIterator.<init>(JdbcDataSource.java:313)
> ... 12 more
> Caused by: org.mariadb.jdbc.internal.util.dao.QueryException: Could
> not connect: Access denied for user 'root'@'ICZ2002912' (using
> password: NO)
> at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.
> authentication(AbstractConnectProtocol.java:524)
> at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.
> handleConnectionPhases(AbstractConnectProtocol.java:472)
> at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.connect(
> AbstractConnectProtocol.java:374)
> at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.
> connectWithoutProxy(AbstractConnectProtocol.java:763)
> at org.mariadb.jdbc.internal.util.Utils.retrieveProxy(Utils.java:469)
> at org.mariadb.jdbc.Driver.connect(Driver.java:104)
> ... 16 more
>
>
>
> On Thu, Oct 6, 2016 at 2:42 PM, Jamie Jackson <ja...@gmail.com>
> wrote:
>
> > It happens to be ten characters.
> >
> > On Thu, Oct 6, 2016 at 12:44 PM, Alexandre Rafalovitch <
> arafalov@gmail.com
> > > wrote:
> >
> >> How long is the encryption key (file content)? Because the code I am
> >> looking at seems to expect it to be at most 100 characters.
> >>
> >> Regards,
> >> Alex.
> >> ----
> >> Newsletter and resources for Solr beginners and intermediates:
> >> http://www.solr-start.com/
> >>
> >>
> >> On 6 October 2016 at 23:26, Kevin Risden <co...@gmail.com>
> >> wrote:
> >> > I haven't tried this but is it possible there is a new line at the end
> >> in
> >> > the file?
> >> >
> >> > If you did something like echo "" > file.txt then there would be a new
> >> > line. Use echo -n "" > file.txt
> >> >
> >> > Also you should be able to check how many characters are in the file.
> >> >
> >> > Kevin Risden
> >> >
> >> > On Wed, Oct 5, 2016 at 5:00 PM, Jamie Jackson <ja...@gmail.com>
> >> wrote:
> >> >
> >> >> Hi Folks,
> >> >>
> >> >> (Using Solr 5.5.3.)
> >> >>
> >> >> As far as I know, the only place where encrypted password use is
> >> documented
> >> >> is in
> >> >> https://cwiki.apache.org/confluence/display/solr/
> >> >> Uploading+Structured+Data+Store+Data+with+the+Data+Import+Handler,
> >> >> under the "Configuring the DIH Configuration File", in a comment in
> the
> >> >> sample XML file:
> >> >>
> >> >> <!--
> >> >> Alternately the password can be encrypted as follows. This is the
> value
> >> >> obtained as a result of the command
> >> >> openssl enc -aes-128-cbc -a -salt -in pwd.txt
> >> >> password="U2FsdGVkX18QMjY0yfCqlfBMvAB4d3XkwY96L7gfO2o="
> >> >> WHen the password is encrypted, you must provide an extra attribute
> >> >> encryptKeyFile="/location/of/encryptionkey"
> >> >> This file should a text file with a single line containing the
> >> >> encrypt/decrypt password
> >> >> -->
> >> >>
> >> >> Anyway, I can encrypt just fine:
> >> >>
> >> >> $ openssl enc -aes-128-cbc -a -salt -in stgps.txt
> >> >> enter aes-128-cbc encryption password:
> >> >> Verifying - enter aes-128-cbc encryption password:
> >> >> U2FsdGVkX1+VtVoQtmEREvB5qZjn3131+N4jRXmjyIY=
> >> >>
> >> >>
> >> >> I can also decrypt just fine from the command line.
> >> >>
> >> >> However, if I use the encrypted password and encryptKeyFile in the
> >> config
> >> >> file, I end up with an error: "String length must be a multiple of
> >> four."
> >> >>
> >> >> https://gist.github.com/jamiejackson/3852dacb03432328ea187d43ade5e4
> d9
> >> >>
> >> >> How do I get this working?
> >> >>
> >> >> Thanks,
> >> >> Jamie
> >> >>
> >>
> >
> >
>
Re: Problem with Password Decryption in Data Import Handler
Posted by Fuad Efendi <fu...@efendi.ca>.
In MySQL, this command will explicitly allow to connect from remote ICZ2002912 host, check MySQL documentation:
GRANT ALL ON mysite.* TO 'root’@'ICZ2002912' IDENTIFIED BY ‘Oakton123’;
On November 2, 2016 at 4:41:48 PM, Fuad Efendi (fuad@efendi.ca) wrote:
This is the root of the problem:
"Access denied for user 'root'@'ICZ2002912' (using password: NO) “
First of all, ensure that plain (non-encrypted) password settings work for you.
Check that you can connect using MySQL client from ICZ2002912 to your MySQL & Co. instance
I suspect you need to allow MySQL & Co. to accept connections from ICZ2002912. Plus, check DNS resolution, etc.
Thanks,
--
Fuad Efendi
(416) 993-2060
http://www.tokenizer.ca
Recommender Systems
On November 2, 2016 at 2:37:08 PM, Jamie Jackson (jamiejaxon@gmail.com) wrote:
I'm at a brick wall. Here's the latest status:
Here are some sample commands that I'm using:
*Create the encryptKeyFile and encrypted password:*
encrypter_password='this_is_my_encrypter_password'
plain_db_pw='Oakton153'
cd /var/docker/solr_stage2/credentials/
echo -n "${encrypter_password}" > encpwd.txt
echo -n "${plain_db_pwd}" > plaindbpwd.txt
openssl enc -aes-128-cbc -a -salt -in plaindbpwd.txt -k
"${encrypter_password}"
rm plaindbpwd.txt
That generated this as the password, by the way:
U2FsdGVkX19pBVTeZaSl43gFFAlrx+Th1zSg1GvlX9o=
*Configure DIH configuration:*
<dataConfig>
<dataSource
driver="org.mariadb.jdbc.Driver"
url="jdbc:mysql://local.mysite.com:3306/mysite"
user="root"
password="U2FsdGVkX19pBVTeZaSl43gFFAlrx+Th1zSg1GvlX9o="
encryptKeyFile="/opt/solr/credentials/encpwd.txt"
/>
...
By the way, /var/docker/solr_stage2/credentials/ is mapped to
/opt/solr/credentials/ in the docker container, so that's why the paths
*seem* different (but aren't, really).
*Authentication error when data import is run:*
Exception while processing: question document :
SolrInputDocument(fields:
[]):org.apache.solr.handler.dataimport.DataImportHandlerException:
Unable to execute query: select 'foo' as bar; Processing
Document # 1
at org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:69)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:323)
at org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:283)
at org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:52)
at org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:59)
at org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:73)
at org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:244)
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:475)
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:414)
at org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:329)
at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:232)
at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:416)
at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:480)
at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:461)
Caused by: java.sql.SQLInvalidAuthorizationSpecException: Could not
connect: Access denied for user 'root'@'ICZ2002912' (using password:
NO)
at org.mariadb.jdbc.internal.util.ExceptionMapper.get(ExceptionMapper.java:123)
at org.mariadb.jdbc.internal.util.ExceptionMapper.throwException(ExceptionMapper.java:71)
at org.mariadb.jdbc.Driver.connect(Driver.java:109)
at org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSource.java:192)
at org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSource.java:172)
at org.apache.solr.handler.dataimport.JdbcDataSource.getConnection(JdbcDataSource.java:503)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:313)
... 12 more
Caused by: org.mariadb.jdbc.internal.util.dao.QueryException: Could
not connect: Access denied for user 'root'@'ICZ2002912' (using
password: NO)
at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.authentication(AbstractConnectProtocol.java:524)
at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.handleConnectionPhases(AbstractConnectProtocol.java:472)
at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.connect(AbstractConnectProtocol.java:374)
at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.connectWithoutProxy(AbstractConnectProtocol.java:763)
at org.mariadb.jdbc.internal.util.Utils.retrieveProxy(Utils.java:469)
at org.mariadb.jdbc.Driver.connect(Driver.java:104)
... 16 more
On Thu, Oct 6, 2016 at 2:42 PM, Jamie Jackson <ja...@gmail.com> wrote:
> It happens to be ten characters.
>
> On Thu, Oct 6, 2016 at 12:44 PM, Alexandre Rafalovitch <arafalov@gmail.com
> > wrote:
>
>> How long is the encryption key (file content)? Because the code I am
>> looking at seems to expect it to be at most 100 characters.
>>
>> Regards,
>> Alex.
>> ----
>> Newsletter and resources for Solr beginners and intermediates:
>> http://www.solr-start.com/
>>
>>
>> On 6 October 2016 at 23:26, Kevin Risden <co...@gmail.com>
>> wrote:
>> > I haven't tried this but is it possible there is a new line at the end
>> in
>> > the file?
>> >
>> > If you did something like echo "" > file.txt then there would be a new
>> > line. Use echo -n "" > file.txt
>> >
>> > Also you should be able to check how many characters are in the file.
>> >
>> > Kevin Risden
>> >
>> > On Wed, Oct 5, 2016 at 5:00 PM, Jamie Jackson <ja...@gmail.com>
>> wrote:
>> >
>> >> Hi Folks,
>> >>
>> >> (Using Solr 5.5.3.)
>> >>
>> >> As far as I know, the only place where encrypted password use is
>> documented
>> >> is in
>> >> https://cwiki.apache.org/confluence/display/solr/
>> >> Uploading+Structured+Data+Store+Data+with+the+Data+Import+Handler,
>> >> under the "Configuring the DIH Configuration File", in a comment in the
>> >> sample XML file:
>> >>
>> >> <!--
>> >> Alternately the password can be encrypted as follows. This is the value
>> >> obtained as a result of the command
>> >> openssl enc -aes-128-cbc -a -salt -in pwd.txt
>> >> password="U2FsdGVkX18QMjY0yfCqlfBMvAB4d3XkwY96L7gfO2o="
>> >> WHen the password is encrypted, you must provide an extra attribute
>> >> encryptKeyFile="/location/of/encryptionkey"
>> >> This file should a text file with a single line containing the
>> >> encrypt/decrypt password
>> >> -->
>> >>
>> >> Anyway, I can encrypt just fine:
>> >>
>> >> $ openssl enc -aes-128-cbc -a -salt -in stgps.txt
>> >> enter aes-128-cbc encryption password:
>> >> Verifying - enter aes-128-cbc encryption password:
>> >> U2FsdGVkX1+VtVoQtmEREvB5qZjn3131+N4jRXmjyIY=
>> >>
>> >>
>> >> I can also decrypt just fine from the command line.
>> >>
>> >> However, if I use the encrypted password and encryptKeyFile in the
>> config
>> >> file, I end up with an error: "String length must be a multiple of
>> four."
>> >>
>> >> https://gist.github.com/jamiejackson/3852dacb03432328ea187d43ade5e4d9
>> >>
>> >> How do I get this working?
>> >>
>> >> Thanks,
>> >> Jamie
>> >>
>>
>
>
Re: Problem with Password Decryption in Data Import Handler
Posted by Fuad Efendi <fu...@efendi.ca>.
This is the root of the problem:
"Access denied for user 'root'@'ICZ2002912' (using password: NO) “
First of all, ensure that plain (non-encrypted) password settings work for you.
Check that you can connect using MySQL client from ICZ2002912 to your MySQL & Co. instance
I suspect you need to allow MySQL & Co. to accept connections from ICZ2002912. Plus, check DNS resolution, etc.
Thanks,
--
Fuad Efendi
(416) 993-2060
http://www.tokenizer.ca
Recommender Systems
On November 2, 2016 at 2:37:08 PM, Jamie Jackson (jamiejaxon@gmail.com) wrote:
I'm at a brick wall. Here's the latest status:
Here are some sample commands that I'm using:
*Create the encryptKeyFile and encrypted password:*
encrypter_password='this_is_my_encrypter_password'
plain_db_pw='Oakton153'
cd /var/docker/solr_stage2/credentials/
echo -n "${encrypter_password}" > encpwd.txt
echo -n "${plain_db_pwd}" > plaindbpwd.txt
openssl enc -aes-128-cbc -a -salt -in plaindbpwd.txt -k
"${encrypter_password}"
rm plaindbpwd.txt
That generated this as the password, by the way:
U2FsdGVkX19pBVTeZaSl43gFFAlrx+Th1zSg1GvlX9o=
*Configure DIH configuration:*
<dataConfig>
<dataSource
driver="org.mariadb.jdbc.Driver"
url="jdbc:mysql://local.mysite.com:3306/mysite"
user="root"
password="U2FsdGVkX19pBVTeZaSl43gFFAlrx+Th1zSg1GvlX9o="
encryptKeyFile="/opt/solr/credentials/encpwd.txt"
/>
...
By the way, /var/docker/solr_stage2/credentials/ is mapped to
/opt/solr/credentials/ in the docker container, so that's why the paths
*seem* different (but aren't, really).
*Authentication error when data import is run:*
Exception while processing: question document :
SolrInputDocument(fields:
[]):org.apache.solr.handler.dataimport.DataImportHandlerException:
Unable to execute query: select 'foo' as bar; Processing
Document # 1
at org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:69)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:323)
at org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:283)
at org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:52)
at org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:59)
at org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:73)
at org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:244)
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:475)
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:414)
at org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:329)
at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:232)
at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:416)
at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:480)
at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:461)
Caused by: java.sql.SQLInvalidAuthorizationSpecException: Could not
connect: Access denied for user 'root'@'ICZ2002912' (using password:
NO)
at org.mariadb.jdbc.internal.util.ExceptionMapper.get(ExceptionMapper.java:123)
at org.mariadb.jdbc.internal.util.ExceptionMapper.throwException(ExceptionMapper.java:71)
at org.mariadb.jdbc.Driver.connect(Driver.java:109)
at org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSource.java:192)
at org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSource.java:172)
at org.apache.solr.handler.dataimport.JdbcDataSource.getConnection(JdbcDataSource.java:503)
at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:313)
... 12 more
Caused by: org.mariadb.jdbc.internal.util.dao.QueryException: Could
not connect: Access denied for user 'root'@'ICZ2002912' (using
password: NO)
at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.authentication(AbstractConnectProtocol.java:524)
at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.handleConnectionPhases(AbstractConnectProtocol.java:472)
at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.connect(AbstractConnectProtocol.java:374)
at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.connectWithoutProxy(AbstractConnectProtocol.java:763)
at org.mariadb.jdbc.internal.util.Utils.retrieveProxy(Utils.java:469)
at org.mariadb.jdbc.Driver.connect(Driver.java:104)
... 16 more
On Thu, Oct 6, 2016 at 2:42 PM, Jamie Jackson <ja...@gmail.com> wrote:
> It happens to be ten characters.
>
> On Thu, Oct 6, 2016 at 12:44 PM, Alexandre Rafalovitch <arafalov@gmail.com
> > wrote:
>
>> How long is the encryption key (file content)? Because the code I am
>> looking at seems to expect it to be at most 100 characters.
>>
>> Regards,
>> Alex.
>> ----
>> Newsletter and resources for Solr beginners and intermediates:
>> http://www.solr-start.com/
>>
>>
>> On 6 October 2016 at 23:26, Kevin Risden <co...@gmail.com>
>> wrote:
>> > I haven't tried this but is it possible there is a new line at the end
>> in
>> > the file?
>> >
>> > If you did something like echo "" > file.txt then there would be a new
>> > line. Use echo -n "" > file.txt
>> >
>> > Also you should be able to check how many characters are in the file.
>> >
>> > Kevin Risden
>> >
>> > On Wed, Oct 5, 2016 at 5:00 PM, Jamie Jackson <ja...@gmail.com>
>> wrote:
>> >
>> >> Hi Folks,
>> >>
>> >> (Using Solr 5.5.3.)
>> >>
>> >> As far as I know, the only place where encrypted password use is
>> documented
>> >> is in
>> >> https://cwiki.apache.org/confluence/display/solr/
>> >> Uploading+Structured+Data+Store+Data+with+the+Data+Import+Handler,
>> >> under the "Configuring the DIH Configuration File", in a comment in the
>> >> sample XML file:
>> >>
>> >> <!--
>> >> Alternately the password can be encrypted as follows. This is the value
>> >> obtained as a result of the command
>> >> openssl enc -aes-128-cbc -a -salt -in pwd.txt
>> >> password="U2FsdGVkX18QMjY0yfCqlfBMvAB4d3XkwY96L7gfO2o="
>> >> WHen the password is encrypted, you must provide an extra attribute
>> >> encryptKeyFile="/location/of/encryptionkey"
>> >> This file should a text file with a single line containing the
>> >> encrypt/decrypt password
>> >> -->
>> >>
>> >> Anyway, I can encrypt just fine:
>> >>
>> >> $ openssl enc -aes-128-cbc -a -salt -in stgps.txt
>> >> enter aes-128-cbc encryption password:
>> >> Verifying - enter aes-128-cbc encryption password:
>> >> U2FsdGVkX1+VtVoQtmEREvB5qZjn3131+N4jRXmjyIY=
>> >>
>> >>
>> >> I can also decrypt just fine from the command line.
>> >>
>> >> However, if I use the encrypted password and encryptKeyFile in the
>> config
>> >> file, I end up with an error: "String length must be a multiple of
>> four."
>> >>
>> >> https://gist.github.com/jamiejackson/3852dacb03432328ea187d43ade5e4d9
>> >>
>> >> How do I get this working?
>> >>
>> >> Thanks,
>> >> Jamie
>> >>
>>
>
>
Re: Problem with Password Decryption in Data Import Handler
Posted by William Bell <bi...@gmail.com>.
OK it was
echo -n "${encrypt_key}" > encrypt.key
On Thu, Nov 3, 2016 at 12:20 PM, William Bell <bi...@gmail.com> wrote:
> I cannot get it to work either.
>
> Here are my steps. I took the key from the Patch in
> https://issues.apache.org/jira/secure/attachment/12730862/SOLR-4392.patch.
>
> echo U2FsdGVkX19Gz7q7/4jj3Wsin7801TlFbob1PBT2YEacbPE
> UARDiuV5zGSAwU4Sz7upXDEPIQPU48oY1fBWM6Q== > pass.enc
>
> openssl aes-128-cbc -d -a -salt -in pass.enc
>
> I typed: Password
>
> enter aes-128-cbc decryption password:
>
> SomeRandomEncryptedTextUsingAES128
>
> I cannot find a test case in the latest v5.5.3 code.? It seems like
> openssl command is wrong?
>
> So it worked for that. Not sure if the code changed, but after doing this
> I get in solr.log:
>
>
> 2016-11-03 12:06:20.139 INFO (Thread-127) [ x:autosuggestfull]
> o.a.s.u.p.LogUpdateProcessorFactory [autosuggestfull] webapp=/solr
> path=/dataimport params={debug=false&optimize=false&indent=true&commit=
> false&clean=false&wt=json&command=full-import&entity=spec&verbose=false}
> status=0 QTime=19{} 0 64
>
> 2016-11-03 12:06:20.140 ERROR (Thread-127) [ x:autosuggestfull]
> o.a.s.h.d.DataImporter Full Import failed:java.lang.RuntimeException:
> java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException:
> Error decoding password Processing Document # 1
>
> at org.apache.solr.handler.dataimport.DocBuilder.execute(
> DocBuilder.java:270)
>
> at org.apache.solr.handler.dataimport.DataImporter.
> doFullImport(DataImporter.java:416)
>
> at org.apache.solr.handler.dataimport.DataImporter.
> runCmd(DataImporter.java:480)
>
> at org.apache.solr.handler.dataimport.DataImporter$1.run(
> DataImporter.java:461)
>
> Caused by: java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException:
> Error decoding password Processing Document # 1
>
> at org.apache.solr.handler.dataimport.DocBuilder.
> buildDocument(DocBuilder.java:416)
>
> at org.apache.solr.handler.dataimport.DocBuilder.
> doFullDump(DocBuilder.java:329)
>
> at org.apache.solr.handler.dataimport.DocBuilder.execute(
> DocBuilder.java:232)
>
> ... 3 more
>
> Caused by: org.apache.solr.handler.dataimport.DataImportHandlerException:
> Error decoding password Processing Document # 1
>
> at org.apache.solr.handler.dataimport.JdbcDataSource.
> decryptPwd(JdbcDataSource.java:131)
>
> at org.apache.solr.handler.dataimport.JdbcDataSource.
> init(JdbcDataSource.java:74)
>
> at org.apache.solr.handler.dataimport.DataImporter.
> getDataSourceInstance(DataImporter.java:389)
>
> at org.apache.solr.handler.dataimport.ContextImpl.
> getDataSource(ContextImpl.java:100)
>
> at org.apache.solr.handler.dataimport.SqlEntityProcessor.
> init(SqlEntityProcessor.java:53)
>
> at org.apache.solr.handler.dataimport.EntityProcessorWrapper.init(
> EntityProcessorWrapper.java:75)
>
> at org.apache.solr.handler.dataimport.DocBuilder.
> buildDocument(DocBuilder.java:433)
>
> at org.apache.solr.handler.dataimport.DocBuilder.
> buildDocument(DocBuilder.java:414)
>
> ... 5 more
>
> Caused by: java.lang.IllegalStateException: Bad password, algorithm, mode
> or padding; no salt, wrong number of iterations or corrupted ciphertext.
>
> at org.apache.solr.util.CryptoKeys.decodeAES(CryptoKeys.java:249)
>
> at org.apache.solr.util.CryptoKeys.decodeAES(CryptoKeys.java:195)
>
> at org.apache.solr.handler.dataimport.JdbcDataSource.
> decryptPwd(JdbcDataSource.java:129)
>
> ... 12 more
>
> Caused by: javax.crypto.BadPaddingException: Given final block not
> properly padded
>
> at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:975)
>
> at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:833)
>
> at com.sun.crypto.provider.AESCipher.engineDoFinal(
> AESCipher.java:446)
>
> at javax.crypto.Cipher.doFinal(Cipher.java:2165)
>
> at org.apache.solr.util.CryptoKeys.decodeAES(CryptoKeys.java:245)
>
> ... 14 more
>
>
> 2016-11-03 12:06:20.140 INFO (Thread-127) [ x:autosuggestfull]
> o.a.s.u.DirectUpdateHandler2 start rollback{}
>
> 2016-11-03 12:06:20.140 INFO (Thread-127) [ x:autosuggestfull]
> o.a.s.u.DefaultSolrCoreState Rollback old IndexWriter...
> core=autosuggestfull
>
> 2016-11-03 12:06:20.154 INFO (Thread-127) [ x:autosuggestfull]
> o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
>
>
>
> On Wed, Nov 2, 2016 at 12:21 PM, Jamie Jackson <ja...@gmail.com>
> wrote:
>
>> I'm at a brick wall. Here's the latest status:
>>
>> Here are some sample commands that I'm using:
>>
>> *Create the encryptKeyFile and encrypted password:*
>>
>>
>> encrypter_password='this_is_my_encrypter_password'
>> plain_db_pw='Oakton153'
>>
>> cd /var/docker/solr_stage2/credentials/
>> echo -n "${encrypter_password}" > encpwd.txt
>> echo -n "${plain_db_pwd}" > plaindbpwd.txt
>> openssl enc -aes-128-cbc -a -salt -in plaindbpwd.txt -k
>> "${encrypter_password}"
>>
>> rm plaindbpwd.txt
>>
>> That generated this as the password, by the way:
>>
>> U2FsdGVkX19pBVTeZaSl43gFFAlrx+Th1zSg1GvlX9o=
>>
>> *Configure DIH configuration:*
>>
>> <dataConfig>
>>
>> <dataSource
>> driver="org.mariadb.jdbc.Driver"
>> url="jdbc:mysql://local.mysite.com:3306/mysite"
>> user="root"
>> password="U2FsdGVkX19pBVTeZaSl43gFFAlrx+Th1zSg1GvlX9o="
>> encryptKeyFile="/opt/solr/credentials/encpwd.txt"
>> />
>> ...
>>
>>
>> By the way, /var/docker/solr_stage2/credentials/ is mapped to
>> /opt/solr/credentials/ in the docker container, so that's why the paths
>> *seem* different (but aren't, really).
>>
>>
>> *Authentication error when data import is run:*
>>
>> Exception while processing: question document :
>> SolrInputDocument(fields:
>> []):org.apache.solr.handler.dataimport.DataImportHandlerException:
>> Unable to execute query: select 'foo' as bar; Processing
>> Document # 1
>> at org.apache.solr.handler.dataimport.DataImportHandlerExceptio
>> n.wrapAndThrow(DataImportHandlerException.java:69)
>> at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetI
>> terator.<init>(JdbcDataSource.java:323)
>> at org.apache.solr.handler.dataimport.JdbcDataSource.getData(
>> JdbcDataSource.java:283)
>> at org.apache.solr.handler.dataimport.JdbcDataSource.getData(
>> JdbcDataSource.java:52)
>> at org.apache.solr.handler.dataimport.SqlEntityProcessor.initQu
>> ery(SqlEntityProcessor.java:59)
>> at org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRo
>> w(SqlEntityProcessor.java:73)
>> at org.apache.solr.handler.dataimport.EntityProcessorWrapper.ne
>> xtRow(EntityProcessorWrapper.java:244)
>> at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(
>> DocBuilder.java:475)
>> at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(
>> DocBuilder.java:414)
>> at org.apache.solr.handler.dataimport.DocBuilder.doFullDump(
>> DocBuilder.java:329)
>> at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBui
>> lder.java:232)
>> at org.apache.solr.handler.dataimport.DataImporter.doFullImport
>> (DataImporter.java:416)
>> at org.apache.solr.handler.dataimport.DataImporter.runCmd(
>> DataImporter.java:480)
>> at org.apache.solr.handler.dataimport.DataImporter$1.run(DataIm
>> porter.java:461)
>> Caused by: java.sql.SQLInvalidAuthorizationSpecException: Could not
>> connect: Access denied for user 'root'@'ICZ2002912' (using password:
>> NO)
>> at org.mariadb.jdbc.internal.util.ExceptionMapper.get(Exception
>> Mapper.java:123)
>> at org.mariadb.jdbc.internal.util.ExceptionMapper.throwExceptio
>> n(ExceptionMapper.java:71)
>> at org.mariadb.jdbc.Driver.connect(Driver.java:109)
>> at org.apache.solr.handler.dataimport.JdbcDataSource$1.call(
>> JdbcDataSource.java:192)
>> at org.apache.solr.handler.dataimport.JdbcDataSource$1.call(
>> JdbcDataSource.java:172)
>> at org.apache.solr.handler.dataimport.JdbcDataSource.getConnect
>> ion(JdbcDataSource.java:503)
>> at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetI
>> terator.<init>(JdbcDataSource.java:313)
>> ... 12 more
>> Caused by: org.mariadb.jdbc.internal.util.dao.QueryException: Could
>> not connect: Access denied for user 'root'@'ICZ2002912' (using
>> password: NO)
>> at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.a
>> uthentication(AbstractConnectProtocol.java:524)
>> at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.h
>> andleConnectionPhases(AbstractConnectProtocol.java:472)
>> at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.c
>> onnect(AbstractConnectProtocol.java:374)
>> at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.c
>> onnectWithoutProxy(AbstractConnectProtocol.java:763)
>> at org.mariadb.jdbc.internal.util.Utils.retrieveProxy(Utils.
>> java:469)
>> at org.mariadb.jdbc.Driver.connect(Driver.java:104)
>> ... 16 more
>>
>>
>>
>> On Thu, Oct 6, 2016 at 2:42 PM, Jamie Jackson <ja...@gmail.com>
>> wrote:
>>
>> > It happens to be ten characters.
>> >
>> > On Thu, Oct 6, 2016 at 12:44 PM, Alexandre Rafalovitch <
>> arafalov@gmail.com
>> > > wrote:
>> >
>> >> How long is the encryption key (file content)? Because the code I am
>> >> looking at seems to expect it to be at most 100 characters.
>> >>
>> >> Regards,
>> >> Alex.
>> >> ----
>> >> Newsletter and resources for Solr beginners and intermediates:
>> >> http://www.solr-start.com/
>> >>
>> >>
>> >> On 6 October 2016 at 23:26, Kevin Risden <co...@gmail.com>
>> >> wrote:
>> >> > I haven't tried this but is it possible there is a new line at the
>> end
>> >> in
>> >> > the file?
>> >> >
>> >> > If you did something like echo "" > file.txt then there would be a
>> new
>> >> > line. Use echo -n "" > file.txt
>> >> >
>> >> > Also you should be able to check how many characters are in the file.
>> >> >
>> >> > Kevin Risden
>> >> >
>> >> > On Wed, Oct 5, 2016 at 5:00 PM, Jamie Jackson <ja...@gmail.com>
>> >> wrote:
>> >> >
>> >> >> Hi Folks,
>> >> >>
>> >> >> (Using Solr 5.5.3.)
>> >> >>
>> >> >> As far as I know, the only place where encrypted password use is
>> >> documented
>> >> >> is in
>> >> >> https://cwiki.apache.org/confluence/display/solr/
>> >> >> Uploading+Structured+Data+Store+Data+with+the+Data+Import+Handler,
>> >> >> under the "Configuring the DIH Configuration File", in a comment in
>> the
>> >> >> sample XML file:
>> >> >>
>> >> >> <!--
>> >> >> Alternately the password can be encrypted as follows. This is the
>> value
>> >> >> obtained as a result of the command
>> >> >> openssl enc -aes-128-cbc -a -salt -in pwd.txt
>> >> >> password="U2FsdGVkX18QMjY0yfCqlfBMvAB4d3XkwY96L7gfO2o="
>> >> >> WHen the password is encrypted, you must provide an extra attribute
>> >> >> encryptKeyFile="/location/of/encryptionkey"
>> >> >> This file should a text file with a single line containing the
>> >> >> encrypt/decrypt password
>> >> >> -->
>> >> >>
>> >> >> Anyway, I can encrypt just fine:
>> >> >>
>> >> >> $ openssl enc -aes-128-cbc -a -salt -in stgps.txt
>> >> >> enter aes-128-cbc encryption password:
>> >> >> Verifying - enter aes-128-cbc encryption password:
>> >> >> U2FsdGVkX1+VtVoQtmEREvB5qZjn3131+N4jRXmjyIY=
>> >> >>
>> >> >>
>> >> >> I can also decrypt just fine from the command line.
>> >> >>
>> >> >> However, if I use the encrypted password and encryptKeyFile in the
>> >> config
>> >> >> file, I end up with an error: "String length must be a multiple of
>> >> four."
>> >> >>
>> >> >> https://gist.github.com/jamiejackson/3852dacb03432328ea187d4
>> 3ade5e4d9
>> >> >>
>> >> >> How do I get this working?
>> >> >>
>> >> >> Thanks,
>> >> >> Jamie
>> >> >>
>> >>
>> >
>> >
>>
>
>
>
> --
> Bill Bell
> billnbell@gmail.com
> cell 720-256-8076
>
--
Bill Bell
billnbell@gmail.com
cell 720-256-8076
Re: Problem with Password Decryption in Data Import Handler
Posted by William Bell <bi...@gmail.com>.
I cannot get it to work either.
Here are my steps. I took the key from the Patch in
https://issues.apache.org/jira/secure/attachment/12730862/SOLR-4392.patch.
echo
U2FsdGVkX19Gz7q7/4jj3Wsin7801TlFbob1PBT2YEacbPEUARDiuV5zGSAwU4Sz7upXDEPIQPU48oY1fBWM6Q==
> pass.enc
openssl aes-128-cbc -d -a -salt -in pass.enc
I typed: Password
enter aes-128-cbc decryption password:
SomeRandomEncryptedTextUsingAES128
I cannot find a test case in the latest v5.5.3 code.? It seems like openssl
command is wrong?
So it worked for that. Not sure if the code changed, but after doing this I
get in solr.log:
2016-11-03 12:06:20.139 INFO (Thread-127) [ x:autosuggestfull]
o.a.s.u.p.LogUpdateProcessorFactory [autosuggestfull] webapp=/solr
path=/dataimport
params={debug=false&optimize=false&indent=true&commit=false&clean=false&wt=json&command=full-import&entity=spec&verbose=false}
status=0 QTime=19{} 0 64
2016-11-03 12:06:20.140 ERROR (Thread-127) [ x:autosuggestfull]
o.a.s.h.d.DataImporter Full Import failed:java.lang.RuntimeException:
java.lang.RuntimeException:
org.apache.solr.handler.dataimport.DataImportHandlerException: Error
decoding password Processing Document # 1
at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:270)
at
org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:416)
at
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:480)
at
org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:461)
Caused by: java.lang.RuntimeException:
org.apache.solr.handler.dataimport.DataImportHandlerException: Error
decoding password Processing Document # 1
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:416)
at
org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:329)
at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:232)
... 3 more
Caused by: org.apache.solr.handler.dataimport.DataImportHandlerException:
Error decoding password Processing Document # 1
at
org.apache.solr.handler.dataimport.JdbcDataSource.decryptPwd(JdbcDataSource.java:131)
at
org.apache.solr.handler.dataimport.JdbcDataSource.init(JdbcDataSource.java:74)
at
org.apache.solr.handler.dataimport.DataImporter.getDataSourceInstance(DataImporter.java:389)
at
org.apache.solr.handler.dataimport.ContextImpl.getDataSource(ContextImpl.java:100)
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.init(SqlEntityProcessor.java:53)
at
org.apache.solr.handler.dataimport.EntityProcessorWrapper.init(EntityProcessorWrapper.java:75)
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:433)
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:414)
... 5 more
Caused by: java.lang.IllegalStateException: Bad password, algorithm, mode
or padding; no salt, wrong number of iterations or corrupted ciphertext.
at org.apache.solr.util.CryptoKeys.decodeAES(CryptoKeys.java:249)
at org.apache.solr.util.CryptoKeys.decodeAES(CryptoKeys.java:195)
at
org.apache.solr.handler.dataimport.JdbcDataSource.decryptPwd(JdbcDataSource.java:129)
... 12 more
Caused by: javax.crypto.BadPaddingException: Given final block not properly
padded
at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:975)
at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:833)
at
com.sun.crypto.provider.AESCipher.engineDoFinal(AESCipher.java:446)
at javax.crypto.Cipher.doFinal(Cipher.java:2165)
at org.apache.solr.util.CryptoKeys.decodeAES(CryptoKeys.java:245)
... 14 more
2016-11-03 12:06:20.140 INFO (Thread-127) [ x:autosuggestfull]
o.a.s.u.DirectUpdateHandler2 start rollback{}
2016-11-03 12:06:20.140 INFO (Thread-127) [ x:autosuggestfull]
o.a.s.u.DefaultSolrCoreState Rollback old IndexWriter...
core=autosuggestfull
2016-11-03 12:06:20.154 INFO (Thread-127) [ x:autosuggestfull]
o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
On Wed, Nov 2, 2016 at 12:21 PM, Jamie Jackson <ja...@gmail.com> wrote:
> I'm at a brick wall. Here's the latest status:
>
> Here are some sample commands that I'm using:
>
> *Create the encryptKeyFile and encrypted password:*
>
>
> encrypter_password='this_is_my_encrypter_password'
> plain_db_pw='Oakton153'
>
> cd /var/docker/solr_stage2/credentials/
> echo -n "${encrypter_password}" > encpwd.txt
> echo -n "${plain_db_pwd}" > plaindbpwd.txt
> openssl enc -aes-128-cbc -a -salt -in plaindbpwd.txt -k
> "${encrypter_password}"
>
> rm plaindbpwd.txt
>
> That generated this as the password, by the way:
>
> U2FsdGVkX19pBVTeZaSl43gFFAlrx+Th1zSg1GvlX9o=
>
> *Configure DIH configuration:*
>
> <dataConfig>
>
> <dataSource
> driver="org.mariadb.jdbc.Driver"
> url="jdbc:mysql://local.mysite.com:3306/mysite"
> user="root"
> password="U2FsdGVkX19pBVTeZaSl43gFFAlrx+Th1zSg1GvlX9o="
> encryptKeyFile="/opt/solr/credentials/encpwd.txt"
> />
> ...
>
>
> By the way, /var/docker/solr_stage2/credentials/ is mapped to
> /opt/solr/credentials/ in the docker container, so that's why the paths
> *seem* different (but aren't, really).
>
>
> *Authentication error when data import is run:*
>
> Exception while processing: question document :
> SolrInputDocument(fields:
> []):org.apache.solr.handler.dataimport.DataImportHandlerException:
> Unable to execute query: select 'foo' as bar; Processing
> Document # 1
> at org.apache.solr.handler.dataimport.DataImportHandlerException.
> wrapAndThrow(DataImportHandlerException.java:69)
> at org.apache.solr.handler.dataimport.JdbcDataSource$
> ResultSetIterator.<init>(JdbcDataSource.java:323)
> at org.apache.solr.handler.dataimport.JdbcDataSource.
> getData(JdbcDataSource.java:283)
> at org.apache.solr.handler.dataimport.JdbcDataSource.
> getData(JdbcDataSource.java:52)
> at org.apache.solr.handler.dataimport.SqlEntityProcessor.
> initQuery(SqlEntityProcessor.java:59)
> at org.apache.solr.handler.dataimport.SqlEntityProcessor.
> nextRow(SqlEntityProcessor.java:73)
> at org.apache.solr.handler.dataimport.EntityProcessorWrapper.
> nextRow(EntityProcessorWrapper.java:244)
> at org.apache.solr.handler.dataimport.DocBuilder.
> buildDocument(DocBuilder.java:475)
> at org.apache.solr.handler.dataimport.DocBuilder.
> buildDocument(DocBuilder.java:414)
> at org.apache.solr.handler.dataimport.DocBuilder.
> doFullDump(DocBuilder.java:329)
> at org.apache.solr.handler.dataimport.DocBuilder.execute(
> DocBuilder.java:232)
> at org.apache.solr.handler.dataimport.DataImporter.
> doFullImport(DataImporter.java:416)
> at org.apache.solr.handler.dataimport.DataImporter.
> runCmd(DataImporter.java:480)
> at org.apache.solr.handler.dataimport.DataImporter$1.run(
> DataImporter.java:461)
> Caused by: java.sql.SQLInvalidAuthorizationSpecException: Could not
> connect: Access denied for user 'root'@'ICZ2002912' (using password:
> NO)
> at org.mariadb.jdbc.internal.util.ExceptionMapper.get(
> ExceptionMapper.java:123)
> at org.mariadb.jdbc.internal.util.ExceptionMapper.throwException(
> ExceptionMapper.java:71)
> at org.mariadb.jdbc.Driver.connect(Driver.java:109)
> at org.apache.solr.handler.dataimport.JdbcDataSource$1.
> call(JdbcDataSource.java:192)
> at org.apache.solr.handler.dataimport.JdbcDataSource$1.
> call(JdbcDataSource.java:172)
> at org.apache.solr.handler.dataimport.JdbcDataSource.
> getConnection(JdbcDataSource.java:503)
> at org.apache.solr.handler.dataimport.JdbcDataSource$
> ResultSetIterator.<init>(JdbcDataSource.java:313)
> ... 12 more
> Caused by: org.mariadb.jdbc.internal.util.dao.QueryException: Could
> not connect: Access denied for user 'root'@'ICZ2002912' (using
> password: NO)
> at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.
> authentication(AbstractConnectProtocol.java:524)
> at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.
> handleConnectionPhases(AbstractConnectProtocol.java:472)
> at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.
> connect(AbstractConnectProtocol.java:374)
> at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.
> connectWithoutProxy(AbstractConnectProtocol.java:763)
> at org.mariadb.jdbc.internal.util.Utils.retrieveProxy(
> Utils.java:469)
> at org.mariadb.jdbc.Driver.connect(Driver.java:104)
> ... 16 more
>
>
>
> On Thu, Oct 6, 2016 at 2:42 PM, Jamie Jackson <ja...@gmail.com>
> wrote:
>
> > It happens to be ten characters.
> >
> > On Thu, Oct 6, 2016 at 12:44 PM, Alexandre Rafalovitch <
> arafalov@gmail.com
> > > wrote:
> >
> >> How long is the encryption key (file content)? Because the code I am
> >> looking at seems to expect it to be at most 100 characters.
> >>
> >> Regards,
> >> Alex.
> >> ----
> >> Newsletter and resources for Solr beginners and intermediates:
> >> http://www.solr-start.com/
> >>
> >>
> >> On 6 October 2016 at 23:26, Kevin Risden <co...@gmail.com>
> >> wrote:
> >> > I haven't tried this but is it possible there is a new line at the end
> >> in
> >> > the file?
> >> >
> >> > If you did something like echo "" > file.txt then there would be a new
> >> > line. Use echo -n "" > file.txt
> >> >
> >> > Also you should be able to check how many characters are in the file.
> >> >
> >> > Kevin Risden
> >> >
> >> > On Wed, Oct 5, 2016 at 5:00 PM, Jamie Jackson <ja...@gmail.com>
> >> wrote:
> >> >
> >> >> Hi Folks,
> >> >>
> >> >> (Using Solr 5.5.3.)
> >> >>
> >> >> As far as I know, the only place where encrypted password use is
> >> documented
> >> >> is in
> >> >> https://cwiki.apache.org/confluence/display/solr/
> >> >> Uploading+Structured+Data+Store+Data+with+the+Data+Import+Handler,
> >> >> under the "Configuring the DIH Configuration File", in a comment in
> the
> >> >> sample XML file:
> >> >>
> >> >> <!--
> >> >> Alternately the password can be encrypted as follows. This is the
> value
> >> >> obtained as a result of the command
> >> >> openssl enc -aes-128-cbc -a -salt -in pwd.txt
> >> >> password="U2FsdGVkX18QMjY0yfCqlfBMvAB4d3XkwY96L7gfO2o="
> >> >> WHen the password is encrypted, you must provide an extra attribute
> >> >> encryptKeyFile="/location/of/encryptionkey"
> >> >> This file should a text file with a single line containing the
> >> >> encrypt/decrypt password
> >> >> -->
> >> >>
> >> >> Anyway, I can encrypt just fine:
> >> >>
> >> >> $ openssl enc -aes-128-cbc -a -salt -in stgps.txt
> >> >> enter aes-128-cbc encryption password:
> >> >> Verifying - enter aes-128-cbc encryption password:
> >> >> U2FsdGVkX1+VtVoQtmEREvB5qZjn3131+N4jRXmjyIY=
> >> >>
> >> >>
> >> >> I can also decrypt just fine from the command line.
> >> >>
> >> >> However, if I use the encrypted password and encryptKeyFile in the
> >> config
> >> >> file, I end up with an error: "String length must be a multiple of
> >> four."
> >> >>
> >> >> https://gist.github.com/jamiejackson/3852dacb03432328ea187d43ade5e4
> d9
> >> >>
> >> >> How do I get this working?
> >> >>
> >> >> Thanks,
> >> >> Jamie
> >> >>
> >>
> >
> >
>
--
Bill Bell
billnbell@gmail.com
cell 720-256-8076