You are viewing a plain text version of this content. The canonical link for it is here.

Posted to solr-user@lucene.apache.org by Maheedhar Kolla <ma...@gmail.com> on 2014/02/12 16:23:15 UTC

Importing database DIH

Hi ,


I need help with importing data, through DIH.  ( using solr-3.6.1, tomcat6 )

 I see the following error when I try to do a full-import from my
local MySQL table ( http:/s..../solr//dataimport?command=full-import"
).

<snip>
..
<str name="Total Requests made to DataSource">0</str>
<str name="Total Rows Fetched">0</str>
<str name="Total Documents Processed">0</str>
<str name="Total Documents Skipped">0</str>
<str name="">Indexing failed. Rolled back all changes.</str>
....
</snip>

I did search to find ways to solve this problem and did create the
file dataimport.properties , but no success.

Any help would be appreciated.


cheers,
Kolla

PS:  When I check the admin panel for statistics for the query
/dataimport , I see the following:

Status : IDLE
Documents Processed : 0
Requests made to DataSource : 0
Rows Fetched : 0
Documents Deleted : 0
Documents Skipped : 0
Total Documents Processed : 0
Total Requests made to DataSource : 0
Total Rows Fetched : 0
Total Documents Deleted : 0
Total Documents Skipped : 0
handlerStart : 1391612468278
requests : 5
errors : 0
timeouts : 0
totalTime : 28
avgTimePerRequest : 5.6


Also, Here is my dataconfig file.
<dataConfig>
 <dataSource type="JdbcDataSource" driver="com.mysql.jdbc.Driver"
                     url="jdbc:mysql://localhost:3306/DBNAME"
                        user="USER" password="PWD"/>
 <document>
     <entity name="docs" query="select * from TABLENAME">
        <field column="id" name="id"/>
        <field column="content" name="text"/>
        <field column="title" name="title"/>
     </entity>
 </document>





-- 
Cheers,
Kolla

Re: Importing database DIH

Posted by Maheedhar Kolla <ma...@gmail.com>.

Thanks for the comments/advice. I did mess with the drivers ( by
deliberately moving the libs) and it did fail as it is supposed to.

When I looked into catalina.out, I realized that the problem lies with
data directory being owned by root instead of tomcat6.  I changed it
so that tomcat6 can write to data directory and now I see a different
error ( after processing 100+ rows) .  But, am happy that the initial
problem is gone. I should be able to fix these minor ones.

Thanks for the advice again.

Cheers,
Kolla

PS: I think I was looking into the wrong logs before :/



On Wed, Feb 12, 2014 at 10:31 AM, Pisarev, Vitaliy
<vi...@hp.com> wrote:
> It can be anything from wrong credentials, to missing driver in the class path, to malformed connection string, etc..
>
> What does the Solr log say?
>
> -----Original Message-----
> From: Maheedhar Kolla [mailto:maheedhar.kolla@gmail.com]
> Sent: יום ד 12 פברואר 2014 17:23
> To: solr-user@lucene.apache.org
> Subject: Importing database DIH
>
> Hi ,
>
>
> I need help with importing data, through DIH.  ( using solr-3.6.1, tomcat6 )
>
>  I see the following error when I try to do a full-import from my local MySQL table ( http:/s..../solr//dataimport?command=full-import"
> ).
>
> <snip>
> ..
> <str name="Total Requests made to DataSource">0</str> <str name="Total Rows Fetched">0</str> <str name="Total Documents Processed">0</str> <str name="Total Documents Skipped">0</str> <str name="">Indexing failed. Rolled back all changes.</str> ....
> </snip>
>
> I did search to find ways to solve this problem and did create the file dataimport.properties , but no success.
>
> Any help would be appreciated.
>
>
> cheers,
> Kolla
>
> PS:  When I check the admin panel for statistics for the query /dataimport , I see the following:
>
> Status : IDLE
> Documents Processed : 0
> Requests made to DataSource : 0
> Rows Fetched : 0
> Documents Deleted : 0
> Documents Skipped : 0
> Total Documents Processed : 0
> Total Requests made to DataSource : 0
> Total Rows Fetched : 0
> Total Documents Deleted : 0
> Total Documents Skipped : 0
> handlerStart : 1391612468278
> requests : 5
> errors : 0
> timeouts : 0
> totalTime : 28
> avgTimePerRequest : 5.6
>
>
> Also, Here is my dataconfig file.
> <dataConfig>
>  <dataSource type="JdbcDataSource" driver="com.mysql.jdbc.Driver"
>                      url="jdbc:mysql://localhost:3306/DBNAME"
>                         user="USER" password="PWD"/>  <document>
>      <entity name="docs" query="select * from TABLENAME">
>         <field column="id" name="id"/>
>         <field column="content" name="text"/>
>         <field column="title" name="title"/>
>      </entity>
>  </document>
>
>
>
>
>
> --
> Cheers,
> Kolla



-- 
Cheers,
Kolla

RE: Importing database DIH

Posted by "Pisarev, Vitaliy" <vi...@hp.com>.

It can be anything from wrong credentials, to missing driver in the class path, to malformed connection string, etc..

What does the Solr log say? 

-----Original Message-----
From: Maheedhar Kolla [mailto:maheedhar.kolla@gmail.com] 
Sent: יום ד 12 פברואר 2014 17:23
To: solr-user@lucene.apache.org
Subject: Importing database DIH

Hi ,

I need help with importing data, through DIH.  ( using solr-3.6.1, tomcat6 )

 I see the following error when I try to do a full-import from my local MySQL table ( http:/s..../solr//dataimport?command=full-import"
).

<snip>
..
<str name="Total Requests made to DataSource">0</str> <str name="Total Rows Fetched">0</str> <str name="Total Documents Processed">0</str> <str name="Total Documents Skipped">0</str> <str name="">Indexing failed. Rolled back all changes.</str> ....
</snip>

I did search to find ways to solve this problem and did create the file dataimport.properties , but no success.

Any help would be appreciated.

cheers,
Kolla

PS:  When I check the admin panel for statistics for the query /dataimport , I see the following:

Status : IDLE
Documents Processed : 0
Requests made to DataSource : 0
Rows Fetched : 0
Documents Deleted : 0
Documents Skipped : 0
Total Documents Processed : 0
Total Requests made to DataSource : 0
Total Rows Fetched : 0
Total Documents Deleted : 0
Total Documents Skipped : 0
handlerStart : 1391612468278
requests : 5
errors : 0
timeouts : 0
totalTime : 28
avgTimePerRequest : 5.6

Also, Here is my dataconfig file.
<dataConfig>
 <dataSource type="JdbcDataSource" driver="com.mysql.jdbc.Driver"
                     url="jdbc:mysql://localhost:3306/DBNAME"
                        user="USER" password="PWD"/>  <document>
     <entity name="docs" query="select * from TABLENAME">
        <field column="id" name="id"/>
        <field column="content" name="text"/>
        <field column="title" name="title"/>
     </entity>
 </document>

--
Cheers,
Kolla

Re: Importing database DIH

Posted by Gora Mohanty <go...@mimirtech.com>.

On 12 February 2014 20:53, Maheedhar Kolla <ma...@gmail.com> wrote:
>
> Hi ,
>
>
> I need help with importing data, through DIH.  ( using solr-3.6.1, tomcat6 )
>
>  I see the following error when I try to do a full-import from my
> local MySQL table ( http:/s..../solr//dataimport?command=full-import"
> ).
>
> <snip>
> ..
> <str name="Total Requests made to DataSource">0</str>
> <str name="Total Rows Fetched">0</str>
> <str name="Total Documents Processed">0</str>
> <str name="Total Documents Skipped">0</str>
> <str name="">Indexing failed. Rolled back all changes.</str>
> ....
> </snip>
>
> I did search to find ways to solve this problem and did create the
> file dataimport.properties , but no success.
[...]

You do not have to create dataimport.properties. Look in the Tomcat logs
for more details on the error, and post the relevant sections here if you
cannot make sense of it. My guess would be that your database credentials
are incorrect, or that the SELECT is failing. Try logging into mysql from
an admin. tool with those credentials, and running the SELECT manually.

Regards,
Gora