[jira] [Commented] (TRAFODION-1576) Performance improvement and reducing offline interval for backup

["Suresh Subbiah (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TRAFODION-1576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14986245#comment-14986245 ] 

Suresh Subbiah commented on TRAFODION-1576:
-------------------------------------------

Offline interval :
Trafodion now restarted by backup script after snapshots are taken, instead of waiting till backup copy is complete.
Offline interval reduced from few hours to few minutes.
On a cluster with 6 TB of data & 273 tables, offline interval is now 5 minutes, including startup time.
Use –o option to get previous behavior.
Trafodion must still be stopped by the admin before using the backup script.  

Backup & restore of small tables faster :
Small tables no longer use MapReduce to export or import a snapshot. Hadoop FileUtil.copy is used instead. Time for each table goes down from 25 secs to 2 seconds
By default small table is anything less than 100 MB. Can be changed by using –l option. Setting –l to 0 goes back to previous behavior
Change is due to new Trafodion Java class org.trafodion.utility.backuprestore.TrafExportSnapshot which has a few lines different from HBase org.hadoop.hbase.snapshot.ExportSnapshot class

Backup & restore of large tables faster :
Number of mappers used to export and import a large snapshot is now determined by size of table. 
2.6 TB table copied in 44 minutes with 102 mappers, on a 6 node cluster
To use a fixed number of mappers for all tables set –m option to desired value. Default is 0.



> Performance improvement and reducing offline interval for backup
> ----------------------------------------------------------------
>
>                 Key: TRAFODION-1576
>                 URL: https://issues.apache.org/jira/browse/TRAFODION-1576
>             Project: Apache Trafodion
>          Issue Type: Improvement
>          Components: db-utility-backup-restore
>    Affects Versions: 1.1 (pre-incubation)
>            Reporter: Suresh Subbiah
>            Assignee: Suresh Subbiah
>             Fix For: 1.3-incubating
>
>
> 1) Trafodion has to be offline for the entire duration that backup script runs. This could be several hours.
> 2) Backing up a small table's snapshot takes several seconds. When there are hundred or more small tables this impacts overall backup time
> 3) Number of mappers used to backup or restore large table snapshots is not depenent on table size 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)