We recently migrated from Oracle 11gR2 to 19c for our Oracle Applications (12.2.10) & decided to continue with old RMAN plans. Run a shell script, call RMAN as “Oracle” user, copy current day backups to a remote server & finalizing the activities with email sent.

Our previous database compressed RMAN backups were a maximum of 142GB per day, that grew to 190-192GB (few datafiles were added) after the upgrade. This made us to reduce the retention period from 7 to 5 and next day, while going through the RMAN logs, I found the following error.

released channel: c1
released channel: c2
released channel: c3
released channel: c4
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================

RMAN-03009: failure of backup command on c4 channel at 02/10/2022 22:24:51
ORA-19571: archived log RECID 416 STAMP 1095631214 not found in control file

RMAN> 

Recovery Manager complete.

So, I went through the previous days logs and found this error started being logged from 9th of Feb and rest were all okay.

I checked database “control_file_record_keep_time” parameter and found the value set as “7” which was more than our current backup retention period.

So the next factor to check was the RMAN script that we were using over a period of 6-7 years without a single failure.

export PATH=$ORACLE_HOME/bin:$PATH
export LAST_RMANLOG_FILE="rman_dailyfull`date +%d%m%y`.log"
echo "Starting Full Daily Database Backup of oracle ebsr12 PROD database"
date
rman target / << EOF >/u03/RMAN/log/rman_dailyfull`date +%d%m%y`.log
run
{
ALLOCATE CHANNEL c1 DEVICE TYPE disk;
ALLOCATE CHANNEL c2 DEVICE TYPE disk;
ALLOCATE CHANNEL c3 DEVICE TYPE disk;
ALLOCATE CHANNEL c4 DEVICE TYPE disk;
crosscheck archivelog all;
crosscheck backup;
BACKUP AS COMPRESSED BACKUPSET DATABASE FORMAT '/u03/RMAN/DAILYBKP/rman_comp_%d_lvl0_%U' TAG "dailyfull_db_lvl0_bkp" INCLUDE CURRENT CONTROLFILE;
sql 'ALTER SYSTEM ARCHIVE LOG CURRENT';
BACKUP AS COMPRESSED BACKUPSET ARCHIVELOG ALL FORMAT '/u03/RMAN/DAILYBKP/archive_%d_lvl0_%U';
DELETE NOPROMPT archivelog all completed before 'sysdate-7';
backup current controlfile format '/u03/RMAN/DAILYBKP/bkpcontrol_file.ctl_%d_%T';
DELETE NOPROMPT OBSOLETE RECOVERY WINDOW OF 5 DAYS;
RELEASE CHANNEL c1;
RELEASE CHANNEL c2;
RELEASE CHANNEL c3;
RELEASE CHANNEL c4;
}
EOF

After a close reading, I found the typo. I was deleting the obsolete backups within a window of 5 days & trying to delete logs which were “7” days old.

DELETE NOPROMPT archivelog all completed before 'sysdate-5';

That’s all. No more RMAN-03009, ORA-19571, or archived log RECID <NNN> STAMP <NNNNNNNNNN> not found in control file errors.