You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-user@db.apache.org by istanbullu <yu...@gmail.com> on 2012/09/05 22:44:38 UTC

Rollforward recovery problem

Hi,

I have a script that does rollforward recovery, but recovery does not appear
to be working in a particular scenario. 

Let me first describe a test scenario that works:

1) I take a DB back-up by enabling log archive mode
2) I delete all files under seg0 directory of the original DB
3) Then, I run my rollforward recovery script, and I see the DB restored
from the backed-up DB created in step 1.  I run consistency check and I see
no issues in the restored DB.

Now, here is the failing scenario:

1) I take a DB back-up by enabling log archive mode
2) I delete a subset of files under seg0 directory of the original DB (say
c5*.dat)
3) Then, I run rollforward recovery script, and I DO NOT see the deleted
files restored from the backed-up DB created in step 1.  I run consistency
check and it fails.

Note that the only difference is that in successful scenario all files under
seg0 are deleted.

My scripts are along the lines of code snippets discussed in this
developerworks article:

http://www.ibm.com/developerworks/data/library/techarticle/dm-0502thalamati/index.html
http://www.ibm.com/developerworks/data/library/techarticle/dm-0502thalamati/index.html 

Any comments appreciated.

-- 
View this message in context: http://old.nabble.com/Rollforward-recovery-problem-tp34394515p34394515.html
Sent from the Apache Derby Users mailing list archive at Nabble.com.


Re: Rollforward recovery problem

Posted by Dag Wanvik <da...@oracle.com>.

On 06.09.2012 16:46, Katherine Marsden wrote:
> On 9/6/2012 11:25 AM, Dag Wanvik wrote:
>> Hi istanbullu,
>>
>> Derby is not designed to recover from deleting the database files 
>> under seg0.
>> If these files get deleted/damaged you need a full backup as produced 
>> from
>> calling SYSCS_UTIL.SYSCS_BACKUP_DATABASE and then restoring it up to 
>> the last transaction committed in the log files.
>> Trying to recover from the log files alone is not guaranteed to work.
>>
>> Cf.
>> http://db.apache.org/derby/docs/10.9/ref/rrefbackupdbproc.html (backup)
>> http://db.apache.org/derby/docs/10.9/ref/rrefrestorefrom.html (restore)
>> http://db.apache.org/derby/docs/10.9/adminguide/cadminrollforward.html (roll 
>> forward recover)
>>
> Dag, I responded before seeing your post. Shouldn't the roll forward 
> recovery automatically first restore the backup created with 
> SYSCS_UTIL.SYSCS_BACKUP_DATABASE_AND_ENABLE_LOG_ARCHIVE_MODE?
>
Yes I believe that's true. I'm not sure whether one needs to removed the 
old image first, but it sounds prudent to me.

Dag


> http://db.apache.org/derby/docs/10.9/adminguide/cadminrollforward.html
>

Re: Rollforward recovery problem

Posted by Katherine Marsden <km...@sbcglobal.net>.
On 9/6/2012 11:25 AM, Dag Wanvik wrote:
> Hi istanbullu,
>
> Derby is not designed to recover from deleting the database files 
> under seg0.
> If these files get deleted/damaged you need a full backup as produced 
> from
> calling SYSCS_UTIL.SYSCS_BACKUP_DATABASE and then restoring it up to 
> the last transaction committed in the log files.
> Trying to recover from the log files alone is not guaranteed to work.
>
> Cf.
> http://db.apache.org/derby/docs/10.9/ref/rrefbackupdbproc.html (backup)
> http://db.apache.org/derby/docs/10.9/ref/rrefrestorefrom.html (restore)
> http://db.apache.org/derby/docs/10.9/adminguide/cadminrollforward.html 
> (roll forward recover)
>
Dag, I responded before seeing your post. Shouldn't the roll forward 
recovery automatically first restore the backup created with 
SYSCS_UTIL.SYSCS_BACKUP_DATABASE_AND_ENABLE_LOG_ARCHIVE_MODE?

http://db.apache.org/derby/docs/10.9/adminguide/cadminrollforward.html


Re: Rollforward recovery problem

Posted by Dag Wanvik <da...@oracle.com>.
Hi istanbullu,

Derby is not designed to recover from deleting the database files under 
seg0.
If these files get deleted/damaged you need a full backup as produced from
calling SYSCS_UTIL.SYSCS_BACKUP_DATABASE and then restoring it up to the 
last transaction committed in the log files.
Trying to recover from the log files alone is not guaranteed to work.

Cf.
http://db.apache.org/derby/docs/10.9/ref/rrefbackupdbproc.html (backup)
http://db.apache.org/derby/docs/10.9/ref/rrefrestorefrom.html (restore)
http://db.apache.org/derby/docs/10.9/adminguide/cadminrollforward.html 
(roll forward recover)

Thanks,
Dag

On 05.09.2012 16:44, istanbullu wrote:
> Hi,
>
> I have a script that does rollforward recovery, but recovery does not appear
> to be working in a particular scenario.
>
> Let me first describe a test scenario that works:
>
> 1) I take a DB back-up by enabling log archive mode
> 2) I delete all files under seg0 directory of the original DB
> 3) Then, I run my rollforward recovery script, and I see the DB restored
> from the backed-up DB created in step 1.  I run consistency check and I see
> no issues in the restored DB.
>
> Now, here is the failing scenario:
>
> 1) I take a DB back-up by enabling log archive mode
> 2) I delete a subset of files under seg0 directory of the original DB (say
> c5*.dat)
> 3) Then, I run rollforward recovery script, and I DO NOT see the deleted
> files restored from the backed-up DB created in step 1.  I run consistency
> check and it fails.
>
> Note that the only difference is that in successful scenario all files under
> seg0 are deleted.
>
> My scripts are along the lines of code snippets discussed in this
> developerworks article:
>
> http://www.ibm.com/developerworks/data/library/techarticle/dm-0502thalamati/index.html
> http://www.ibm.com/developerworks/data/library/techarticle/dm-0502thalamati/index.html
>
> Any comments appreciated.
>

Re: Rollforward recovery problem

Posted by Katherine Marsden <km...@sbcglobal.net>.
On 9/6/2012 1:53 PM, istanbullu wrote:
> Hi Kathey,
>
> Thanks for looking into this.
>
> The sample DB I have is fairly large and contains confidential info, so I
> cannot share it.  I will see if I can reproduce the problem with a simpler
> scenario.  Also, I will try to reproduce it using ij.  What I currently have
> are Java programs.
A java reproduction would be great too, even better probably, we just 
need to try to recreate the behavior.

> In your example, did you see just one file under seg0?  Note that if I
> delete all files under seg0, I do not see an issue: DB is restored
> successfully.
No. A new database on trunk had 72 files in an empty database. Making 
the two tables in my repro attempt added two more.

> Derby network server was not shutdown.  I tried to do rollforward recovery
> over Network server.
So  was the database booted at the time you attempted the restore? It 
would be important to shut down the database before attempting a 
restore. I could imagine some pretty  strange things would happen if the 
database was restored whilst booted because the page cache would have 
one set of data and then another get restored underneath. Sometimes I 
find it helpful to try these things out on Windows as it will be more 
vocal about file permissions.

I will try my reproduction on 10.5 and  with Network Server.  Please 
keep us updated on your attempts to narrow this down.

Thanks

Kathey


Re: Rollforward recovery problem

Posted by istanbullu <yu...@gmail.com>.
Hi Kathey,

Thanks for looking into this.

The sample DB I have is fairly large and contains confidential info, so I
cannot share it.  I will see if I can reproduce the problem with a simpler
scenario.  Also, I will try to reproduce it using ij.  What I currently have
are Java programs.  

In your example, did you see just one file under seg0?  Note that if I
delete all files under seg0, I do not see an issue: DB is restored
successfully.

Derby network server was not shutdown.  I tried to do rollforward recovery
over Network server.

Derby version is 10.5.3.0.

Thanks.

Yuksel


kmarsden wrote:
> 
> On 9/5/2012 1:44 PM, istanbullu wrote:
> <snip>
>> Now, here is the failing scenario:
>>
>> 1) I take a DB back-up by enabling log archive mode
>> 2) I delete a subset of files under seg0 directory of the original DB
>> (say
>> c5*.dat)
>> 3) Then, I run rollforward recovery script, and I DO NOT see the deleted
>> files restored from the backed-up DB created in step 1.  I run
>> consistency
>> check and it fails.
>>
>> Note that the only difference is that in successful scenario all files
>> under
>> seg0 are deleted.
>>
>> My scripts are along the lines of code snippets discussed in this
>> developerworks article:
>>
>> http://www.ibm.com/developerworks/data/library/techarticle/dm-0502thalamati/index.html
>>
>> Any comments appreciated.
>>
> 
> Thank you so much for thinking about backup and recovery and pointing to 
> the article.  I didn't know this article existed and will bookmark it 
> and promote it!
> 
> I have always instructed people restoring after corruption to make a zip 
> archive of the database  for analysis and then  remove a corrupt (or any 
> existing database) before restoring to that location.  I often see 
> corruptions that occur from restoring one database on top of another 
> (usually with operating system commands.)
> 
> I see now that the documentation says restore should delete the 
> existing  database :
> 
> http://db.apache.org/derby/docs/10.8/adminguide/tadminhubbkup44.html
> 
> so what you are describing sounds like a bug.
> 
> I tried reproducing on 10.8.2.2 - (1181258)  and trunk as follows: ( I 
> am using cygwin  so pardon the shell commands on windows)
> 
> I created the database:
> 
>   $java org.apache.derby.tools.ij
> ij version 10.8
> ij> connect 'jdbc:derby:wombat;create=true';
> ij> create table t (i int);
> 0 rows inserted/updated/deleted
> ij> create table t2 (i int);
> 0 rows inserted/updated/deleted
> ij> exit;
> 
> Found a file under seg0:
> $ls -l wombat/seg0/c470.dat
> -rwxrwxrwx 1 kmarsden None 8192 Sep  6 13:08 wombat/seg0/c470.dat
> 
> 
> $ Backed up with log archive mode
>   $java org.apache.derby.tools.ij
> ij version 10.8
> ij> connect 'jdbc:derby:wombat';
> ij>  CALL 
> SYSCS_UTIL.SYSCS_BACKUP_DATABASE_AND_ENABLE_LOG_ARCHIVE_MODE('c:/cygwin/tmp/backuplogarc/09_06_2012',1);
> 0 rows inserted/updated/deleted
> ij> exit;
> 
> Deleted the file:
> $rm wombat/seg0/c470.dat
> $ls -l wombat/seg0/c470.dat
> ls: cannot access wombat/seg0/c470.dat: No such file or directory
> 
> Restored with roll forward recovery:
> ij>connect 
> 'jdbc:derby:wombat;rollForwardRecoveryFrom=c:/cygwin/tmp/backuplogarc/09_06_2012/wombat';
> ij> exit;
> 
> My file was restored:
> 
>   $ls -l wombat/seg0/c470.dat
> rwxrwx 1 kmarsden None 8192 Sep  6 13:13 wombat/seg0/c470.dat
> 
> Can you provide a reproduction where the files don't get restored?
> 
> Also could you include the full Derby version from the derby.log? Was 
> Derby or network server fully shutdown before attempting the restore?
> 
> Thanks
> 
> Kathey
> 
> 

-- 
View this message in context: http://old.nabble.com/Rollforward-recovery-problem-tp34394515p34399688.html
Sent from the Apache Derby Users mailing list archive at Nabble.com.


Re: Rollforward recovery problem

Posted by Katherine Marsden <km...@sbcglobal.net>.
On 9/5/2012 1:44 PM, istanbullu wrote:
<snip>
> Now, here is the failing scenario:
>
> 1) I take a DB back-up by enabling log archive mode
> 2) I delete a subset of files under seg0 directory of the original DB (say
> c5*.dat)
> 3) Then, I run rollforward recovery script, and I DO NOT see the deleted
> files restored from the backed-up DB created in step 1.  I run consistency
> check and it fails.
>
> Note that the only difference is that in successful scenario all files under
> seg0 are deleted.
>
> My scripts are along the lines of code snippets discussed in this
> developerworks article:
>
> http://www.ibm.com/developerworks/data/library/techarticle/dm-0502thalamati/index.html
>
> Any comments appreciated.
>

Thank you so much for thinking about backup and recovery and pointing to 
the article.  I didn't know this article existed and will bookmark it 
and promote it!

I have always instructed people restoring after corruption to make a zip 
archive of the database  for analysis and then  remove a corrupt (or any 
existing database) before restoring to that location.  I often see 
corruptions that occur from restoring one database on top of another 
(usually with operating system commands.)

I see now that the documentation says restore should delete the 
existing  database :

http://db.apache.org/derby/docs/10.8/adminguide/tadminhubbkup44.html

so what you are describing sounds like a bug.

I tried reproducing on 10.8.2.2 - (1181258)  and trunk as follows: ( I 
am using cygwin  so pardon the shell commands on windows)

I created the database:

  $java org.apache.derby.tools.ij
ij version 10.8
ij> connect 'jdbc:derby:wombat;create=true';
ij> create table t (i int);
0 rows inserted/updated/deleted
ij> create table t2 (i int);
0 rows inserted/updated/deleted
ij> exit;

Found a file under seg0:
$ls -l wombat/seg0/c470.dat
-rwxrwxrwx 1 kmarsden None 8192 Sep  6 13:08 wombat/seg0/c470.dat


$ Backed up with log archive mode
  $java org.apache.derby.tools.ij
ij version 10.8
ij> connect 'jdbc:derby:wombat';
ij>  CALL 
SYSCS_UTIL.SYSCS_BACKUP_DATABASE_AND_ENABLE_LOG_ARCHIVE_MODE('c:/cygwin/tmp/backuplogarc/09_06_2012',1);
0 rows inserted/updated/deleted
ij> exit;

Deleted the file:
$rm wombat/seg0/c470.dat
$ls -l wombat/seg0/c470.dat
ls: cannot access wombat/seg0/c470.dat: No such file or directory

Restored with roll forward recovery:
ij>connect 
'jdbc:derby:wombat;rollForwardRecoveryFrom=c:/cygwin/tmp/backuplogarc/09_06_2012/wombat';
ij> exit;

My file was restored:

  $ls -l wombat/seg0/c470.dat
rwxrwx 1 kmarsden None 8192 Sep  6 13:13 wombat/seg0/c470.dat

Can you provide a reproduction where the files don't get restored?

Also could you include the full Derby version from the derby.log? Was 
Derby or network server fully shutdown before attempting the restore?

Thanks

Kathey