[jira] Commented: (CONNECTORS-117) Database-specific maintenance activities such as reindexing should have their frequency be under the control of the database driver

["Karl Wright (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CONNECTORS-117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12920489#action_12920489 ] 

Karl Wright commented on CONNECTORS-117:
----------------------------------------

There are two kinds of activities that matter: ANALYZE, and REINDEX, each on a respective table.  The tracking of the number of modifies, inserts, and deletes should remain the responsibility of the table manager itself, but a notification method for each activity should be implemented.  The database driver (only PostgreSQL, so far) will then need to keep track of per-table data statically, and make appropriate reindexing decisions.  We can also readily add VACUUM FULL maintenance code under this same scheme.

I actually recommend using shared data (as defined within ILockManager) for this purpose.  Cross-process statistics then can be tracked, and indexing requests can be coordinated.   Eventually this stuff will be in zookeeper, so performance will be good.  In the interim, we can commit changes to counts lazily (every 100 actions or so) to reduce the overhead.

Modification tracking data will not be lost or reset if the agents process is restarted in a multi-process system.  This is new.  In a single-process system, it WILL be lost upon restart, which is as it always has been.  FWIW.


> Database-specific maintenance activities such as reindexing should have their frequency be under the control of the database driver
> -----------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: CONNECTORS-117
>                 URL: https://issues.apache.org/jira/browse/CONNECTORS-117
>             Project: ManifoldCF
>          Issue Type: Improvement
>          Components: Framework core
>            Reporter: Karl Wright
>
> Not all databases will require maintenance activity at the same frequency, and different versions of the same database may also differ in this way.  Two changes should thus be made: (1) Move the database maintenance frequency to be under the control of the database implementation, and (2) Where appropriate, introduce properties.xml properties for each database where this is important.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.