Glenn Fawcett's Weblog

Parallelizing Oracle backup with RMAN on CMT based servers

Wednesday May 14, 2008

A backup window is important to keep in check to ensure time for batch and on-line work. With Oracle RMAN there are several ways to keep backups flowing smoothly. This example shows how you can use multiple channels and parallelism to increase the throughput of backup to the maximum of your IO configuration.

Results

This graph shows scaling in MB/sec based on the # of channels in use. The term "channels" used by Oracle does not have any relation to actual "physical" channels. In Oracle RMAN terms, a channel is simply a "connection" to a database for which to backup data. Data files are assigned to "connections" in a round-robin fashion so as to utilize all connections as evenly as possible.



By configuring a parallelism of 20 with RMAN, I was able to increase throughput from 5->80 MB/sec. Single threaded performance was limited to 5MB/sec mainly due to the high CPU component that comes with using "COMPRESSED" backups. The way to maximize IO throughput with COMPRESSION is to simply add more streams.

RMAN commands to achieve parallelism

I used the following commands to create 20 backup "channels" for RMAN. Notice that they configured to use the same directory, just with different file formats. 

RMAN> configure channel 1 device type disk format
     '/o6s_data/GLENNF/d2/backup_db_c1%d_S_%s_P_%p_T_%t' MAXPIECESIZE 1024 M;
RMAN> configure channel 2 device type disk format
     '/o6s_data/GLENNF/d2/backup_db_c2%d_S_%s_P_%p_T_%t' MAXPIECESIZE 1024 M;
...
...
RMAN> configure channel 20 device type disk format
     '/o6s_data/GLENNF/d2/backup_db_c20%d_S_%s_P_%p_T_%t' MAXPIECESIZE 1024 M;
After creating these channels, you must tell RMAN how to connect to these channels: 

RMAN> configure channel 1 DEVICE TYPE DISK CONNECT '/as sysdba';
RMAN> configure channel 2 DEVICE TYPE DISK CONNECT '/as sysdba';
...
...
RMAN> configure channel 20 DEVICE TYPE DISK CONNECT '/as sysdba';
Next, you need to tell RMAN to use disk parallelism of 20: 

RMAN> CONFIGURE DEVICE TYPE DISK BACKUP TYPE 
      TO COMPRESSED BACKUPSET PARALLELISM 20;
Finally, let's issue the backup command: 

RMAN> BACKUP TABLESPACE GLENNF_RMAN;

Starting backup at 09-MAY-08
allocated channel: ORA_DISK_1
channel ORA_DISK_1: sid=966 devtype=DISK
allocated channel: ORA_DISK_2
channel ORA_DISK_2: sid=952 devtype=DISK
allocated channel: ORA_DISK_3
channel ORA_DISK_3: sid=940 devtype=DISK
allocated channel: ORA_DISK_4
channel ORA_DISK_4: sid=938 devtype=DISK
allocated channel: ORA_DISK_5
channel ORA_DISK_5: sid=939 devtype=DISK
allocated channel: ORA_DISK_6
channel ORA_DISK_6: sid=969 devtype=DISK
allocated channel: ORA_DISK_7
channel ORA_DISK_7: sid=961 devtype=DISK
allocated channel: ORA_DISK_8
channel ORA_DISK_8: sid=963 devtype=DISK
allocated channel: ORA_DISK_9
channel ORA_DISK_9: sid=953 devtype=DISK
allocated channel: ORA_DISK_10
channel ORA_DISK_10: sid=970 devtype=DISK
allocated channel: ORA_DISK_11
channel ORA_DISK_11: sid=920 devtype=DISK
allocated channel: ORA_DISK_12
channel ORA_DISK_12: sid=943 devtype=DISK
allocated channel: ORA_DISK_13
channel ORA_DISK_13: sid=968 devtype=DISK
allocated channel: ORA_DISK_14
channel ORA_DISK_14: sid=929 devtype=DISK
allocated channel: ORA_DISK_15
channel ORA_DISK_15: sid=960 devtype=DISK
allocated channel: ORA_DISK_16
channel ORA_DISK_16: sid=931 devtype=DISK
allocated channel: ORA_DISK_17
channel ORA_DISK_17: sid=927 devtype=DISK
allocated channel: ORA_DISK_18
channel ORA_DISK_18: sid=957 devtype=DISK
allocated channel: ORA_DISK_19
channel ORA_DISK_19: sid=958 devtype=DISK
allocated channel: ORA_DISK_20
channel ORA_DISK_20: sid=964 devtype=DISK
channel ORA_DISK_1: starting compressed full datafile backupset
channel ORA_DISK_1: specifying datafile(s) in backupset
input datafile fno=00068 name=/oracle/O6S/sapraw/glenn1
channel ORA_DISK_1: starting piece 1 at 09-MAY-08
channel ORA_DISK_2: starting compressed full datafile backupset
channel ORA_DISK_2: specifying datafile(s) in backupset
input datafile fno=00069 name=/oracle/O6S/sapraw/glenn2
channel ORA_DISK_2: starting piece 1 at 09-MAY-08
channel ORA_DISK_3: starting compressed full datafile backupset
channel ORA_DISK_3: specifying datafile(s) in backupset
input datafile fno=00070 name=/oracle/O6S/sapraw/glenn3
channel ORA_DISK_3: starting piece 1 at 09-MAY-08
channel ORA_DISK_4: starting compressed full datafile backupset
channel ORA_DISK_4: specifying datafile(s) in backupset
input datafile fno=00071 name=/oracle/O6S/sapraw/glenn4
channel ORA_DISK_4: starting piece 1 at 09-MAY-08
channel ORA_DISK_5: starting compressed full datafile backupset
channel ORA_DISK_5: specifying datafile(s) in backupset
input datafile fno=00072 name=/oracle/O6S/sapraw/glenn5
channel ORA_DISK_5: starting piece 1 at 09-MAY-08
channel ORA_DISK_6: starting compressed full datafile backupset
channel ORA_DISK_6: specifying datafile(s) in backupset
input datafile fno=00073 name=/oracle/O6S/sapraw/glenn6
channel ORA_DISK_6: starting piece 1 at 09-MAY-08
channel ORA_DISK_7: starting compressed full datafile backupset
channel ORA_DISK_7: specifying datafile(s) in backupset
input datafile fno=00074 name=/oracle/O6S/sapraw/glenn7
channel ORA_DISK_7: starting piece 1 at 09-MAY-08
channel ORA_DISK_8: starting compressed full datafile backupset
channel ORA_DISK_8: specifying datafile(s) in backupset
input datafile fno=00075 name=/oracle/O6S/sapraw/glenn8
channel ORA_DISK_8: starting piece 1 at 09-MAY-08
channel ORA_DISK_9: starting compressed full datafile backupset
channel ORA_DISK_9: specifying datafile(s) in backupset
input datafile fno=00076 name=/oracle/O6S/sapraw/glenn9
channel ORA_DISK_9: starting piece 1 at 09-MAY-08
channel ORA_DISK_10: starting compressed full datafile backupset
channel ORA_DISK_10: specifying datafile(s) in backupset
input datafile fno=00077 name=/oracle/O6S/sapraw/glenn10
channel ORA_DISK_10: starting piece 1 at 09-MAY-08
channel ORA_DISK_11: starting compressed full datafile backupset
channel ORA_DISK_11: specifying datafile(s) in backupset
input datafile fno=00078 name=/oracle/O6S/sapraw/glenn11
channel ORA_DISK_11: starting piece 1 at 09-MAY-08
channel ORA_DISK_12: starting compressed full datafile backupset
channel ORA_DISK_12: specifying datafile(s) in backupset
input datafile fno=00079 name=/oracle/O6S/sapraw/glenn12
channel ORA_DISK_12: starting piece 1 at 09-MAY-08
channel ORA_DISK_13: starting compressed full datafile backupset
channel ORA_DISK_13: specifying datafile(s) in backupset
input datafile fno=00080 name=/oracle/O6S/sapraw/glenn13
channel ORA_DISK_13: starting piece 1 at 09-MAY-08
channel ORA_DISK_14: starting compressed full datafile backupset
channel ORA_DISK_14: specifying datafile(s) in backupset
input datafile fno=00081 name=/oracle/O6S/sapraw/glenn14
channel ORA_DISK_14: starting piece 1 at 09-MAY-08
channel ORA_DISK_15: starting compressed full datafile backupset
channel ORA_DISK_15: specifying datafile(s) in backupset
input datafile fno=00082 name=/oracle/O6S/sapraw/glenn15
channel ORA_DISK_15: starting piece 1 at 09-MAY-08
channel ORA_DISK_16: starting compressed full datafile backupset
channel ORA_DISK_16: specifying datafile(s) in backupset
input datafile fno=00083 name=/oracle/O6S/sapraw/glenn16
channel ORA_DISK_16: starting piece 1 at 09-MAY-08
channel ORA_DISK_17: starting compressed full datafile backupset
channel ORA_DISK_17: specifying datafile(s) in backupset
input datafile fno=00084 name=/oracle/O6S/sapraw/glenn17
channel ORA_DISK_17: starting piece 1 at 09-MAY-08
channel ORA_DISK_18: starting compressed full datafile backupset
channel ORA_DISK_18: specifying datafile(s) in backupset
input datafile fno=00085 name=/oracle/O6S/sapraw/glenn18
channel ORA_DISK_18: starting piece 1 at 09-MAY-08
channel ORA_DISK_19: starting compressed full datafile backupset
channel ORA_DISK_19: specifying datafile(s) in backupset
input datafile fno=00086 name=/oracle/O6S/sapraw/glenn19
channel ORA_DISK_19: starting piece 1 at 09-MAY-08
channel ORA_DISK_20: starting compressed full datafile backupset
channel ORA_DISK_20: specifying datafile(s) in backupset
input datafile fno=00087 name=/oracle/O6S/sapraw/glenn20
channel ORA_DISK_20: starting piece 1 at 09-MAY-08
channel ORA_DISK_2: finished piece 1 at 09-MAY-08
piece handle=/o6s_data/GLENNF/d2/backup_db_c2O6S_S_81_P_1_T_654270132 tag=TAG20080509T134205 comment=NONE
channel ORA_DISK_2: backup set complete, elapsed time: 00:00:58
channel ORA_DISK_3: finished piece 1 at 09-MAY-08
piece handle=/o6s_data/GLENNF/d2/backup_db_c3O6S_S_82_P_1_T_654270132 tag=TAG20080509T134205 comment=NONE
channel ORA_DISK_3: backup set complete, elapsed time: 00:00:58
channel ORA_DISK_4: finished piece 1 at 09-MAY-08
piece handle=/o6s_data/GLENNF/d2/backup_db_c4O6S_S_83_P_1_T_654270132 tag=TAG20080509T134205 comment=NONE
channel ORA_DISK_4: backup set complete, elapsed time: 00:00:58
channel ORA_DISK_9: finished piece 1 at 09-MAY-08
piece handle=/o6s_data/GLENNF/d2/backup_db_c9O6S_S_88_P_1_T_654270133 tag=TAG20080509T134205 comment=NONE
channel ORA_DISK_9: backup set complete, elapsed time: 00:00:57
channel ORA_DISK_11: finished piece 1 at 09-MAY-08
piece handle=/o6s_data/GLENNF/d2/backup_db_c11O6S_S_90_P_1_T_654270133 tag=TAG20080509T134205 comment=NONE
channel ORA_DISK_11: backup set complete, elapsed time: 00:00:57
channel ORA_DISK_12: finished piece 1 at 09-MAY-08
piece handle=/o6s_data/GLENNF/d2/backup_db_c12O6S_S_91_P_1_T_654270133 tag=TAG20080509T134205 comment=NONE
channel ORA_DISK_12: backup set complete, elapsed time: 00:00:57
channel ORA_DISK_13: finished piece 1 at 09-MAY-08
piece handle=/o6s_data/GLENNF/d2/backup_db_c13O6S_S_92_P_1_T_654270133 tag=TAG20080509T134205 comment=NONE
channel ORA_DISK_13: backup set complete, elapsed time: 00:00:57
channel ORA_DISK_18: finished piece 1 at 09-MAY-08
piece handle=/o6s_data/GLENNF/d2/backup_db_c18O6S_S_97_P_1_T_654270134 tag=TAG20080509T134205 comment=NONE
channel ORA_DISK_18: backup set complete, elapsed time: 00:00:56
channel ORA_DISK_20: finished piece 1 at 09-MAY-08
piece handle=/o6s_data/GLENNF/d2/backup_db_c20O6S_S_99_P_1_T_654270135 tag=TAG20080509T134205 comment=NONE
channel ORA_DISK_20: backup set complete, elapsed time: 00:00:55
channel ORA_DISK_10: finished piece 1 at 09-MAY-08
piece handle=/o6s_data/GLENNF/d2/backup_db_c10O6S_S_89_P_1_T_654270133 tag=TAG20080509T134205 comment=NONE
channel ORA_DISK_10: backup set complete, elapsed time: 00:00:58
channel ORA_DISK_16: finished piece 1 at 09-MAY-08
piece handle=/o6s_data/GLENNF/d2/backup_db_c16O6S_S_95_P_1_T_654270134 tag=TAG20080509T134205 comment=NONE
channel ORA_DISK_16: backup set complete, elapsed time: 00:00:57
channel ORA_DISK_1: finished piece 1 at 09-MAY-08
piece handle=/o6s_data/GLENNF/d2/backup_db_c1O6S_S_80_P_1_T_654270132 tag=TAG20080509T134205 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:01:00
channel ORA_DISK_5: finished piece 1 at 09-MAY-08
piece handle=/o6s_data/GLENNF/d2/backup_db_c5O6S_S_84_P_1_T_654270132 tag=TAG20080509T134205 comment=NONE
channel ORA_DISK_5: backup set complete, elapsed time: 00:01:00
channel ORA_DISK_14: finished piece 1 at 09-MAY-08
piece handle=/o6s_data/GLENNF/d2/backup_db_c14O6S_S_93_P_1_T_654270134 tag=TAG20080509T134205 comment=NONE
channel ORA_DISK_14: backup set complete, elapsed time: 00:00:58
channel ORA_DISK_7: finished piece 1 at 09-MAY-08
piece handle=/o6s_data/GLENNF/d2/backup_db_c7O6S_S_86_P_1_T_654270132 tag=TAG20080509T134205 comment=NONE
channel ORA_DISK_7: backup set complete, elapsed time: 00:01:01
channel ORA_DISK_8: finished piece 1 at 09-MAY-08
piece handle=/o6s_data/GLENNF/d2/backup_db_c8O6S_S_87_P_1_T_654270132 tag=TAG20080509T134205 comment=NONE
channel ORA_DISK_8: backup set complete, elapsed time: 00:01:01
channel ORA_DISK_6: finished piece 1 at 09-MAY-08
piece handle=/o6s_data/GLENNF/d2/backup_db_c6O6S_S_85_P_1_T_654270132 tag=TAG20080509T134205 comment=NONE
channel ORA_DISK_6: backup set complete, elapsed time: 00:01:04
channel ORA_DISK_15: finished piece 1 at 09-MAY-08
piece handle=/o6s_data/GLENNF/d2/backup_db_c15O6S_S_94_P_1_T_654270134 tag=TAG20080509T134205 comment=NONE
channel ORA_DISK_15: backup set complete, elapsed time: 00:01:02
channel ORA_DISK_17: finished piece 1 at 09-MAY-08
piece handle=/o6s_data/GLENNF/d2/backup_db_c17O6S_S_96_P_1_T_654270134 tag=TAG20080509T134205 comment=NONE
channel ORA_DISK_17: backup set complete, elapsed time: 00:01:02
channel ORA_DISK_19: finished piece 1 at 09-MAY-08
piece handle=/o6s_data/GLENNF/d2/backup_db_c19O6S_S_98_P_1_T_654270135 tag=TAG20080509T134205 comment=NONE
channel ORA_DISK_19: backup set complete, elapsed time: 00:01:01
Finished backup at 09-MAY-08

Configuration

Comments:

We're running Oracle 10g on a T2000 with a Netapp for storage (using NFSv3) -- and backups are also to the same Netapp but a different volume on a different aggregate (raid group) and the network interface is not saturated, the Netapp isn't working hard (CPU) and disk IO is not saturated -- is parallelizing the RMAN backup going to increase the number of NFS threads used for IO and give us more throughput?

Posted by R.P. Aditya on May 15, 2008 at 06:26 AM PDT #

While I have no experience with Netapp in this environment, I see no reason why this would not work. I would go for it.

Posted by Glenn Fawcett on May 15, 2008 at 11:04 AM PDT #

Post a Comment:
  • HTML Syntax: NOT allowed