You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by "Marcin (JIRA)" <ji...@apache.org> on 2010/03/24 13:20:27 UTC

[jira] Created: (SOLR-1842) DataImportHandler ODBC keeps lock on the source table while optimisatising is being run...

DataImportHandler ODBC keeps lock on the source table while optimisatising is being run...
------------------------------------------------------------------------------------------

                 Key: SOLR-1842
                 URL: https://issues.apache.org/jira/browse/SOLR-1842
             Project: Solr
          Issue Type: Bug
          Components: contrib - DataImportHandler
    Affects Versions: 1.5
            Reporter: Marcin


Hi Guys,

I don't know if its really a bug but I think its quite good place for it.

The problem is with dataImportHandler and DB queries.

For example:
Let's have a big table which keeps docs to being indexed, we are running query against it on a datimporthandler and query locks table which is quite obvius and desire behaviour from the SQL points of view but while optimisation is being done its should not allow to issue query because in that case table is being locked till optimisation process will finish which can take a time...

As a workaround you can use "select SQL_BUFFER_RESULT..." statment which will move everything into temp table and release all locks but still dataImportHandlerwill be waiting for optimisation to finish. Which means you will be able to insert new docs into main table at least.


cheers

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (SOLR-1842) DataImportHandler ODBC keeps lock on the source table while optimisatising is being run...

Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/SOLR-1842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12851945#action_12851945 ] 

Hoss Man commented on SOLR-1842:
--------------------------------

Yeah, i don't know anything about ODBC, but it seems odd that DIH wouldn't "commit" any transactions it opens to release the table locks.  (unless this is something to do with auto generated transactions in the ODBC Connector)

Marcin: it woul be helpful if you could provide a specific example of a DIH config in which you see this problem (the simpler the better) ... perhaps you are using some feature of DIH in a way that is unexpected and that's why the table locks are living longer then they should.

> DataImportHandler ODBC keeps lock on the source table while optimisatising is being run...
> ------------------------------------------------------------------------------------------
>
>                 Key: SOLR-1842
>                 URL: https://issues.apache.org/jira/browse/SOLR-1842
>             Project: Solr
>          Issue Type: Bug
>          Components: contrib - DataImportHandler
>    Affects Versions: 1.5
>            Reporter: Marcin
>
> Hi Guys,
> I don't know if its really a bug but I think its quite good place for it.
> The problem is with dataImportHandler and DB queries.
> For example:
> Let's have a big table which keeps docs to being indexed, we are running query against it on a datimporthandler and query locks table which is quite obvius and desire behaviour from the SQL points of view but while optimisation is being done its should not allow to issue query because in that case table is being locked till optimisation process will finish which can take a time...
> As a workaround you can use "select SQL_BUFFER_RESULT..." statment which will move everything into temp table and release all locks but still dataImportHandlerwill be waiting for optimisation to finish. Which means you will be able to insert new docs into main table at least.
> cheers

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (SOLR-1842) DataImportHandler ODBC keeps lock on the source table while optimisatising is being run...

Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/SOLR-1842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12850554#action_12850554 ] 

Noble Paul commented on SOLR-1842:
----------------------------------

DIH cannot do anything specific for one type of driver. I'm not sure what is the expected fix

> DataImportHandler ODBC keeps lock on the source table while optimisatising is being run...
> ------------------------------------------------------------------------------------------
>
>                 Key: SOLR-1842
>                 URL: https://issues.apache.org/jira/browse/SOLR-1842
>             Project: Solr
>          Issue Type: Bug
>          Components: contrib - DataImportHandler
>    Affects Versions: 1.5
>            Reporter: Marcin
>
> Hi Guys,
> I don't know if its really a bug but I think its quite good place for it.
> The problem is with dataImportHandler and DB queries.
> For example:
> Let's have a big table which keeps docs to being indexed, we are running query against it on a datimporthandler and query locks table which is quite obvius and desire behaviour from the SQL points of view but while optimisation is being done its should not allow to issue query because in that case table is being locked till optimisation process will finish which can take a time...
> As a workaround you can use "select SQL_BUFFER_RESULT..." statment which will move everything into temp table and release all locks but still dataImportHandlerwill be waiting for optimisation to finish. Which means you will be able to insert new docs into main table at least.
> cheers

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (SOLR-1842) DataImportHandler ODBC keeps lock on the source table while optimisatising is being run...

Posted by "Lance Norskog (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/SOLR-1842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12851722#action_12851722 ] 

Lance Norskog commented on SOLR-1842:
-------------------------------------

Could the DIH shut down all Datasources & EntityProcessors before calling commit? After all, they are finished at that point. 


> DataImportHandler ODBC keeps lock on the source table while optimisatising is being run...
> ------------------------------------------------------------------------------------------
>
>                 Key: SOLR-1842
>                 URL: https://issues.apache.org/jira/browse/SOLR-1842
>             Project: Solr
>          Issue Type: Bug
>          Components: contrib - DataImportHandler
>    Affects Versions: 1.5
>            Reporter: Marcin
>
> Hi Guys,
> I don't know if its really a bug but I think its quite good place for it.
> The problem is with dataImportHandler and DB queries.
> For example:
> Let's have a big table which keeps docs to being indexed, we are running query against it on a datimporthandler and query locks table which is quite obvius and desire behaviour from the SQL points of view but while optimisation is being done its should not allow to issue query because in that case table is being locked till optimisation process will finish which can take a time...
> As a workaround you can use "select SQL_BUFFER_RESULT..." statment which will move everything into temp table and release all locks but still dataImportHandlerwill be waiting for optimisation to finish. Which means you will be able to insert new docs into main table at least.
> cheers

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.