You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Daniel Watzinger <da...@gmail.com> on 2019/07/27 21:00:46 UTC
svnadmin fails to create empty repository on CSV storage (Windows
Cluster Shared Volume)
svnadmin reproducibly fails to initialize an empty repository on a cluster
shared volume in a Windows Failover Cluster environment on Windows Server
2019. The initial call seems to be successful.
svnadmin create foobar
A subsequent
svnadmin verify foobar
fails with
* Verifying metadata at revision 0 ...
* Error verifying repository metadata.
svnadmin: E160004: P2L offset 6b must be larger than L2P offset 6b in r0 footer
Re: svnadmin fails to create empty repository on CSV storage (Windows Cluster Shared Volume)
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Daniel Watzinger wrote on Mon, 12 Aug 2019 10:56 +00:00:
> Please don't ask why we are using the Windows platform in the first
> place ;-)
We weren't planning to ask. Subversion supports any platform that
supports C89, Python, and APR and has a maintained build script.
I'm not familiar with your storage solution so I'll defer to others to
comment on that.
Good luck,
Daniel
Re: svnadmin fails to create empty repository on CSV storage (Windows
Cluster Shared Volume)
Posted by Daniel Watzinger <da...@gmail.com>.
Please let me know if you need any more input on this matter :-)
PS: To give a little bit of background context, we encountered this issue
while trying to set up a HA subversion cluster (svnserve) using Windows
Failover Clustering features behind a nginx reverse proxy load balancer.
The recommended and performant way to share storage between cluster nodes
is to use a CSV volume. Shared storage may of course be implemented, albeit
less performant, by using simple file shares. Please don't ask why we are
using the Windows platform in the first place ;-)
Have a nice day.
Daniel
On Tue, Jul 30, 2019 at 11:13 PM Daniel Watzinger <
daniel.watzinger@gmail.com> wrote:
> The issue at hand may be solved by changing how svn code calls APR.
> However, I strongly suspect that there is more code that relies on data and
> metadata synchronization. I cannot comment on the latter due to
> insufficient knowledge of the APR library and its guarantees on IO
> operations.
>
> On Tue, Jul 30, 2019 at 11:04 PM Daniel Shahaf <d....@daniel.shahaf.name>
> wrote:
>
>> Daniel Watzinger wrote on Tue, 30 Jul 2019 20:44 +00:00:
>> > I took the liberty to dive into the relevant source code. The issue may
>> > be pinned down to file metadata not being flushed to disk by the
>> > underlying APR library.
>>
>> Do you mean that svn needs to change how it calls APR, or that APR needs
>> to change how it calls the kernel? In the latter case, I'm afraid I'll
>> have to forward you again, to <http://apr.apache.org/mailing-lists.html>.
>>
>>
Re: svnadmin fails to create empty repository on CSV storage (Windows
Cluster Shared Volume)
Posted by Daniel Watzinger <da...@gmail.com>.
The issue at hand may be solved by changing how svn code calls APR.
However, I strongly suspect that there is more code that relies on data and
metadata synchronization. I cannot comment on the latter due to
insufficient knowledge of the APR library and its guarantees on IO
operations.
On Tue, Jul 30, 2019 at 11:04 PM Daniel Shahaf <d....@daniel.shahaf.name>
wrote:
> Daniel Watzinger wrote on Tue, 30 Jul 2019 20:44 +00:00:
> > I took the liberty to dive into the relevant source code. The issue may
> > be pinned down to file metadata not being flushed to disk by the
> > underlying APR library.
>
> Do you mean that svn needs to change how it calls APR, or that APR needs
> to change how it calls the kernel? In the latter case, I'm afraid I'll
> have to forward you again, to <http://apr.apache.org/mailing-lists.html>.
>
>
Re: svnadmin fails to create empty repository on CSV storage (Windows Cluster Shared Volume)
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Daniel Watzinger wrote on Tue, 30 Jul 2019 20:44 +00:00:
> I took the liberty to dive into the relevant source code. The issue may
> be pinned down to file metadata not being flushed to disk by the
> underlying APR library.
Do you mean that svn needs to change how it calls APR, or that APR needs
to change how it calls the kernel? In the latter case, I'm afraid I'll
have to forward you again, to <http://apr.apache.org/mailing-lists.html>.
Re: svnadmin fails to create empty repository on CSV storage (Windows
Cluster Shared Volume)
Posted by Daniel Watzinger <da...@gmail.com>.
Good evening Daniel :-)
I somehow miserably failed to include a cross-reference to the JIRA issue I
have created.
- https://issues.apache.org/jira/browse/SVN-4819
I took the liberty to dive into the relevant source code. The issue may be
pinned down to file metadata not being flushed to disk by the underlying
APR library. This is a non-issue on many (Windows) filesystems like
NTFS/FAT as a single file handles metadata (like filesize) is synchronized
by the corresponding filesystem drivers, albeit undocumented. However, the
implementation of CSV seems to differ (due to the need to synchronize file
metadata across cluster nodes).
On Tue, Jul 30, 2019 at 9:45 PM Daniel Shahaf <d....@daniel.shahaf.name>
wrote:
> Good morning Daniel,
>
> I don't have the brainwidth to investigate this, but two quick questions:
>
> - Does it happen on a local disk?
> - Can you post the file foobar/db/revs/0/0 as an attachment?
>
> FWIW, 6b is the correct value of the P2L offset.
>
> Cheers,
>
> Daniel
>
> Daniel Watzinger wrote on Sun, 28 Jul 2019 07:19 +00:00:
> > svnadmin reproducibly fails to initialize an empty repository on a
> > cluster shared volume in a Windows Failover Cluster environment on
> > Windows Server 2019. The initial call seems to be successful.
> >
> > svnadmin create foobar
> > A subsequent
> >
> > svnadmin verify foobar
> > fails with
> >
> > * Verifying metadata at revision 0 ...
> > * Error verifying repository metadata.
> > svnadmin: E160004: P2L offset 6b must be larger than L2P offset 6b in r0
> footer
>
Re: svnadmin fails to create empty repository on CSV storage (Windows Cluster Shared Volume)
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Good morning Daniel,
I don't have the brainwidth to investigate this, but two quick questions:
- Does it happen on a local disk?
- Can you post the file foobar/db/revs/0/0 as an attachment?
FWIW, 6b is the correct value of the P2L offset.
Cheers,
Daniel
Daniel Watzinger wrote on Sun, 28 Jul 2019 07:19 +00:00:
> svnadmin reproducibly fails to initialize an empty repository on a
> cluster shared volume in a Windows Failover Cluster environment on
> Windows Server 2019. The initial call seems to be successful.
>
> svnadmin create foobar
> A subsequent
>
> svnadmin verify foobar
> fails with
>
> * Verifying metadata at revision 0 ...
> * Error verifying repository metadata.
> svnadmin: E160004: P2L offset 6b must be larger than L2P offset 6b in r0 footer