You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by "Ph. Marek" <ph...@bmlv.gv.at> on 2007/07/02 11:58:43 UTC
svntar, anybody?
Hello everybody,
I'd like to pipe parts of a repository over some channel to another box.
I found "svntar" mentioned here:
http://svn.haxx.se/users/archive-2006-03/1310.shtml
But that thread ends with a PHP solution that exports into a temporary
directory, which is not feasible for my needs.
Does someone know if such a thing does already exist?
Regards,
Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svntar, anybody?
Posted by Erik Huelsmann <eh...@gmail.com>.
Could this discussion be moved to users@ please, as it's obviously not
about developing Subversion itself anymore?
Thanks in advance!
bye,
Erik.
On 7/4/07, Ph. Marek <ph...@bmlv.gv.at> wrote:
> On Mittwoch, 4. Juli 2007, Talden wrote:
> > > > A
> > > > tar based approach will seek the length of the tar to evaluate
> > > > differences to sync.
> > >
> > > It won't seek, it will just push that to the clients to apply.
> > > And linear reading is on the >100MByte/sec range, eg. for a cheap stripe
> > > set of two harddisks.
> >
> > Assuming of course that only one client pulls at a time. If not then
> > you depending upon disk cache to avoid seeking. A large RAM
> > investment might be coming your way.
> The difference is still between having a few (self-synchronizing) points that
> are read in a single file vs. having many files being read randomized.
>
> See here:
>
> Tar-file ===================================================================
> ^ ^
> Client1 |
> Client2
>
>
> These two clients will synchronize, until they both send identical data -
> because Client2 will have to fetch from disk, while Client1 can use already
> cached data and is thus faster.
>
>
> Tar-file ===================================================================
> ^ ^ ^ ^
> Client1 | Client3&4 Client5-8
> Client2
>
> After a while there'll be (at most) a few points in the file being read; using
> the anticipatory io-scheduler and/or read-ahead it's easy to only seek, say,
> 3 or 4 times a second while keeping the bandwidth at maximum speed.
>
>
> Compare that with 8 clients reading physically discontinuos regions from the
> harddisk, with additional CPU load ....
> Sure, that data *should* be cached after the first reading ... but there's a
> lot more to cache: indizes from the repository, inodes for the revision
> files, and just the seeking around for "Client3&4" above takes most of the
> harddisk bandwidth.
>
>
> Regards,
>
> Phil
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svntar, anybody?
Posted by "Ph. Marek" <ph...@bmlv.gv.at>.
On Mittwoch, 4. Juli 2007, Talden wrote:
> > > A
> > > tar based approach will seek the length of the tar to evaluate
> > > differences to sync.
> >
> > It won't seek, it will just push that to the clients to apply.
> > And linear reading is on the >100MByte/sec range, eg. for a cheap stripe
> > set of two harddisks.
>
> Assuming of course that only one client pulls at a time. If not then
> you depending upon disk cache to avoid seeking. A large RAM
> investment might be coming your way.
The difference is still between having a few (self-synchronizing) points that
are read in a single file vs. having many files being read randomized.
See here:
Tar-file ===================================================================
^ ^
Client1 |
Client2
These two clients will synchronize, until they both send identical data -
because Client2 will have to fetch from disk, while Client1 can use already
cached data and is thus faster.
Tar-file ===================================================================
^ ^ ^ ^
Client1 | Client3&4 Client5-8
Client2
After a while there'll be (at most) a few points in the file being read; using
the anticipatory io-scheduler and/or read-ahead it's easy to only seek, say,
3 or 4 times a second while keeping the bandwidth at maximum speed.
Compare that with 8 clients reading physically discontinuos regions from the
harddisk, with additional CPU load ....
Sure, that data *should* be cached after the first reading ... but there's a
lot more to cache: indizes from the repository, inodes for the revision
files, and just the seeking around for "Client3&4" above takes most of the
harddisk bandwidth.
Regards,
Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svntar, anybody?
Posted by Talden <ta...@gmail.com>.
> > - Meaning that every day it's a new set of clients that still have no
> > existing install? if not then this only applies to the first day.
> This is for installation of machines ... a typical linux install has a few
> hundred thousand files, and a few GB of space used.
Thanks, this clarifies the issue quite considerably. None of the
suggested working copy or repository cloning options are going to
perform well.
On either the server machine (or one on a very fast link to the server
machine) use the suggested svntar approach on a daily created tag.
Since you'll have a daily tag you'll not need to retain copies of past
tars since they can be regenerated on demand.
> An update yes; a fresh checkout/export would need to read many revisions --
> perhaps *all* of them, if old data is needed to reconstruct the full-text.
> And this is CPU intensive.
That revision reading will still occur with the svntar approach.
There's no way to avoid the computation during such an activity. You
could very well hit all of the revisions unfortunately since, though
skip-diffs might allow shallow reads in very specific file requests,
broad 'get all files at the HEAD' will likely hit a very wide
distribution of the revisions.
Still, at least it occurs only once for the tar generation.
> > A
> > tar based approach will seek the length of the tar to evaluate
> > differences to sync.
> It won't seek, it will just push that to the clients to apply.
> And linear reading is on the >100MByte/sec range, eg. for a cheap stripe set
> of two harddisks.
Assuming of course that only one client pulls at a time. If not then
you depending upon disk cache to avoid seeking. A large RAM
investment might be coming your way.
--
Talden
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svntar, anybody?
Posted by "Ph. Marek" <ph...@bmlv.gv.at>.
On Mittwoch, 4. Juli 2007, Talden wrote:
> > I want to push some (big) amount of files as fast as possible to a number
> > of clients, which have no prior version available as they're freshly
> > installed.
> - Meaning that every day it's a new set of clients that still have no
> existing install? if not then this only applies to the first day.
This is for installation of machines ... a typical linux install has a few
hundred thousand files, and a few GB of space used.
> - Big AMOUNT of files or BIG files? And define big... 100,000 files
> of 150kb or 100 files of 150mb.
Both :-) Some files are big, most are small ...
> Also, clearly, working copies or repo clones are out of the question
> if they don't have the space to keep a duplicate copy. Working copies
> have pristine copies and the repo clone option is, by definition, a
> duplicate since you need to extract content from the clone.
Once again, the clients should get a snapshot.
Having ".svn" is not possible (space wasted, perhaps a security problem), and
a complete repository even more so.
> You want to avoid seeking and I assume this is at the server end.
Right.
> A
> working copy update would only be seeking the last few revisions.
An update yes; a fresh checkout/export would need to read many revisions --
perhaps *all* of them, if old data is needed to reconstruct the full-text.
And this is CPU intensive.
> A
> tar based approach will seek the length of the tar to evaluate
> differences to sync.
It won't seek, it will just push that to the clients to apply.
And linear reading is on the >100MByte/sec range, eg. for a cheap stripe set
of two harddisks.
Please note that the tar file is just a kind of cache, nothing more!
Regards,
Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svntar, anybody?
Posted by Talden <ta...@gmail.com>.
> I want to push some (big) amount of files as fast as possible to a number of
> clients, which have no prior version available as they're freshly installed.
- Meaning that every day it's a new set of clients that still have no
existing install? if not then this only applies to the first day.
- Big AMOUNT of files or BIG files? And define big... 100,000 files
of 150kb or 100 files of 150mb.
Also, clearly, working copies or repo clones are out of the question
if they don't have the space to keep a duplicate copy. Working copies
have pristine copies and the repo clone option is, by definition, a
duplicate since you need to extract content from the clone.
You want to avoid seeking and I assume this is at the server end. A
working copy update would only be seeking the last few revisions. A
tar based approach will seek the length of the tar to evaluate
differences to sync.
--
Talden
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svntar, anybody?
Posted by "Ph. Marek" <ph...@bmlv.gv.at>.
On Dienstag, 3. Juli 2007, Ben Collins-Sussman wrote:
> I have to admit, I'm lost at this point. Reading through this entire
> thread, it seems like the list of requirments keeps changing. Phil,
> can you be *really specific* and list *all* the requirements in one
> neat list?
Ok, I start afresh.
I want to push some (big) amount of files as fast as possible to a number of
clients, which have no prior version available as they're freshly installed.
This happens in parallel and has a change granularity of a day.
Machines do not have identical hardware.
I'd like to avoid harddisk seeks, so a single large stream is prefered, CPU
usage (on the central machine) should be kept as low as possible.
My conclusions (feel free to skip them, if you think they're invalid or
misleading, or have better solutions):
- Doing exports/checkouts directly from the repository used much time (CPU for
decompression, seeks for revision files, etc.)
- Doing a rsync from a (daily) exported/updated directory that has some
hundred thousand files has some CPU usage, too. And seeking ...
- Simply copying a partition pushes a big amount of zeroes; and the partition
sizes on the clients can change (as they've got varying harddisks), so
they'd had to copy some standard image and resize it afterwards.
- Using partimage or similar has the same problem.
- Of course it's possible to export/update into a temporary directory, and get
a tar from there. But if the middle step can be avoided, it's doesn't need
the additional space; and it's faster, so the image can get regenerated more
often.
Any better ideas?
Regards,
Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svntar, anybody?
Posted by Ben Collins-Sussman <su...@red-bean.com>.
I have to admit, I'm lost at this point. Reading through this entire
thread, it seems like the list of requirments keeps changing. Phil,
can you be *really specific* and list *all* the requirements in one
neat list?
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svntar, anybody?
Posted by "Ph. Marek" <ph...@bmlv.gv.at>.
On Dienstag, 3. Juli 2007, Talden wrote:
> If they're 'pulling' a latest snapshot on a granularity of a day then is
> there any reason they can't have a working copy after the first day?
Yes, they're different machines every day.
> This
> means you just need to pull a working copy yourself, tar and compress it
> and get them to unpack it, from then on they can simply update for the days
> changes.
>
> A required 2GB per client per day (400GB+) should be avoided if possible.
This is the set-up for the machines. A working minimum install is 2GB, gets up
to 10-15 GB (depending on software, sometimes higher [with lots of
datafiles]), and your assumed 200 machines a day might be too small :-)
Regards,
Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svntar, anybody?
Posted by Talden <ta...@gmail.com>.
If they're 'pulling' a latest snapshot on a granularity of a day then is
there any reason they can't have a working copy after the first day? This
means you just need to pull a working copy yourself, tar and compress it and
get them to unpack it, from then on they can simply update for the days
changes.
A required 2GB per client per day (400GB+) should be avoided if possible.
--
Talden
On 7/3/07, Ph. Marek <ph...@bmlv.gv.at> wrote:
>
> Hello Ben!
>
> On Dienstag, 3. Juli 2007, Ben Collins-Sussman wrote:
> > On 7/3/07, Ph. Marek <ph...@bmlv.gv.at> wrote:
> > > I'd like to push the contents of some repository (say, 2GB) to many
> > > (order of several hundred) machines at once, with a granularity of,
> say,
> > > a day.
> > >
> > > Believing (but not having benchmarked) that restoring file data from a
> > > compressed repository is *much* more CPU-using than simply streaming
> from
> > > a RAID array, I thought that the easy way would be to generate a tar
> > > file, and let the clients fetch it.
> >
> > If the repository itself is accessible, you can just 'svnadmin dump'
> > and let clients fetch that. A dumpfile is extremely similar to a
> > tarfile.
> >
> > If you're already planning on making several hundred machines "pull"
> > repository data, why not have each of them just use svnsync to
> > replicate (pull) the repository to their local disks? What's great
> > about that is that only the most recent commits will be replicated
> > each day. Clients can then run 'svn export' from their local
> > repositories.
> The clients need just the current snapshot, not the history.
> And this is for the first setup, so the clients have no "old" version.
>
> > Or, if you want to avoid the CPU work of actually interpreting
> > (decompressing) the repository data,
> Yes, that's what I want!
>
> > just have each client machine use
> > rsync to do daily pulls instead.
> That's not what I want, because
> -) they have no old version, so
> -) rsync would have to push *all* files, anyway, and
> -) as mentioned, a nice big tar file can be streamed very easily ...
> With a directory structure of some hundred thousand files you can
> get lost in seeks.
>
>
> Thank you, anyway.
>
>
> Regards,
>
> Phil
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
>
>
Re: svntar, anybody?
Posted by "Ph. Marek" <ph...@bmlv.gv.at>.
Hello John!
On Dienstag, 3. Juli 2007, John Peacock wrote:
...
> However, every single method of generating an export/dump/whatever from
> scratch every time will take much CPU on the server.
I don't know how much that matters, if I'm going to do a compressed tar
anyway.
Doing a bzip2 (or even only gzip) takes more time than getting the data out of
the repository, no?
> If you can have a
> single working copy on the server that is always in sync with the trunk
> (or whatever branch you are distributing), then you could either tar or
> otherwise stream that to as many computers as you want daily.
Yes, it *may* be worthwile that some directory is updated, and a tar is taken
from there.
> If, on
> the other hand, the image changes radically from day to day, there is no
> other current solution I am aware of that would work.
Hmmm, the change rate? Good question. Normally small, but if eg. openoffice is
replaced, thats 100MB in a single change...
> That being said, I could see using the bindings to read the repository
> directly and output something like a tar stream, but that would require
> two expertise:
>
> 1) using the bindings
> 2) knowing the tar file format to generate on the fly
>
> Essentially, it would be a matter of writing a recursive cat that
> embedded the files into a tar stream on the fly...
Later in this thread a svntar is mentioned; it seems to work.
Thank you for your thoughts!
Regards,
Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svntar, anybody?
Posted by John Peacock <jp...@rowman.com>.
Ph. Marek wrote:
> I just want a nice, easy, streamlined image of some directory structure ...
> Generated once or a few times a day, sent out without CPU load ...
There are several ways to generate snapshots; I'm not sure what script
the Subversion project recommends, but I've taken over maintenance of
SVN::Notify::Snapshot
and I could be persuaded to make alterations if that would help you (it
is currently biased towards generating a tarball every time something
changes). It also does it by exporting and tarring the export; I may
consider using something like Archive::TarGzip to streamline that somewhat.
However, every single method of generating an export/dump/whatever from
scratch every time will take much CPU on the server. If you can have a
single working copy on the server that is always in sync with the trunk
(or whatever branch you are distributing), then you could either tar or
otherwise stream that to as many computers as you want daily. If, on
the other hand, the image changes radically from day to day, there is no
other current solution I am aware of that would work.
That being said, I could see using the bindings to read the repository
directly and output something like a tar stream, but that would require
two expertise:
1) using the bindings
2) knowing the tar file format to generate on the fly
Essentially, it would be a matter of writing a recursive cat that
embedded the files into a tar stream on the fly...
HTH
John
--
John Peacock
Director of Information Research and Technology
Rowman & Littlefield Publishing Group
4501 Forbes Boulevard
Suite H
Lanham, MD 20706
301-459-3366 x.5010
fax 301-429-5748
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svntar, anybody?
Posted by "Ph. Marek" <ph...@bmlv.gv.at>.
Hello Miha!
On Dienstag, 3. Juli 2007, Miha Vitorovic wrote:
> "Ph. Marek" <ph...@bmlv.gv.at> wrote on 03.07.2007 13:22:28:
> > > If you're already planning on making several hundred machines "pull"
> > > repository data, why not have each of them just use svnsync to
> > > replicate (pull) the repository to their local disks? What's great
> > > about that is that only the most recent commits will be replicated
> > > each day. Clients can then run 'svn export' from their local
> > > repositories.
> >
> > The clients need just the current snapshot, not the history.
> > And this is for the first setup, so the clients have no "old" version.
>
> Of course you can 'svnadmin dump' starting with the last revision. This
> gets you exactly what you want (minus the CPU load).
But a "svnadmin dump" has to read other revision files, too (for files not
changed in the last revision), so the harddisk would be seeking again ...
I just want a nice, easy, streamlined image of some directory structure ...
Generated once or a few times a day, sent out without CPU load ...
With a generated .tar.gz the clients need to decompress, but otherwise have
*maximum* throughput.
And if I do just a .tar, I could transmit ~110MB/sec over GBit ... (with RAID
behind, of course).
Regards,
Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svntar, anybody?
Posted by Miha Vitorovic <mv...@nil.si>.
"Ph. Marek" <ph...@bmlv.gv.at> wrote on 03.07.2007 13:22:28:
> > If you're already planning on making several hundred machines "pull"
> > repository data, why not have each of them just use svnsync to
> > replicate (pull) the repository to their local disks? What's great
> > about that is that only the most recent commits will be replicated
> > each day. Clients can then run 'svn export' from their local
> > repositories.
> The clients need just the current snapshot, not the history.
> And this is for the first setup, so the clients have no "old" version.
Of course you can 'svnadmin dump' starting with the last revision. This
gets you exactly what you want (minus the CPU load).
Br,
---
Miha Vitorovic
Inženir v tehničnem področju
Customer Support Engineer
NIL Data Communications, Tivolska cesta 48, 1000 Ljubljana, Slovenia
Phone +386 1 4746 500 Fax +386 1 4746 501 http://www.NIL.si
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svntar, anybody?
Posted by "Ph. Marek" <ph...@bmlv.gv.at>.
Hello Ben!
On Dienstag, 3. Juli 2007, Ben Collins-Sussman wrote:
> On 7/3/07, Ph. Marek <ph...@bmlv.gv.at> wrote:
> > I'd like to push the contents of some repository (say, 2GB) to many
> > (order of several hundred) machines at once, with a granularity of, say,
> > a day.
> >
> > Believing (but not having benchmarked) that restoring file data from a
> > compressed repository is *much* more CPU-using than simply streaming from
> > a RAID array, I thought that the easy way would be to generate a tar
> > file, and let the clients fetch it.
>
> If the repository itself is accessible, you can just 'svnadmin dump'
> and let clients fetch that. A dumpfile is extremely similar to a
> tarfile.
>
> If you're already planning on making several hundred machines "pull"
> repository data, why not have each of them just use svnsync to
> replicate (pull) the repository to their local disks? What's great
> about that is that only the most recent commits will be replicated
> each day. Clients can then run 'svn export' from their local
> repositories.
The clients need just the current snapshot, not the history.
And this is for the first setup, so the clients have no "old" version.
> Or, if you want to avoid the CPU work of actually interpreting
> (decompressing) the repository data,
Yes, that's what I want!
> just have each client machine use
> rsync to do daily pulls instead.
That's not what I want, because
-) they have no old version, so
-) rsync would have to push *all* files, anyway, and
-) as mentioned, a nice big tar file can be streamed very easily ...
With a directory structure of some hundred thousand files you can
get lost in seeks.
Thank you, anyway.
Regards,
Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svntar, anybody?
Posted by Ben Collins-Sussman <su...@red-bean.com>.
On 7/3/07, Ph. Marek <ph...@bmlv.gv.at> wrote:
> I'd like to push the contents of some repository (say, 2GB) to many (order of
> several hundred) machines at once, with a granularity of, say, a day.
>
> Believing (but not having benchmarked) that restoring file data from a
> compressed repository is *much* more CPU-using than simply streaming from a
> RAID array, I thought that the easy way would be to generate a tar file, and
> let the clients fetch it.
If the repository itself is accessible, you can just 'svnadmin dump'
and let clients fetch that. A dumpfile is extremely similar to a
tarfile.
If you're already planning on making several hundred machines "pull"
repository data, why not have each of them just use svnsync to
replicate (pull) the repository to their local disks? What's great
about that is that only the most recent commits will be replicated
each day. Clients can then run 'svn export' from their local
repositories.
Or, if you want to avoid the CPU work of actually interpreting
(decompressing) the repository data, just have each client machine use
rsync to do daily pulls instead.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svntar, anybody?
Posted by "Ph. Marek" <ph...@bmlv.gv.at>.
Hello John!
On Montag, 2. Juli 2007, John Peacock wrote:
> Perhaps if you described what you actually hope to accomplish (e.g. keep
> a remote filesystem in sync with a local repository), rather than a
> specific mechanism (like pipe or tar), someone could make a more
> intelligent suggestion.
Ok, here is it.
I'd like to push the contents of some repository (say, 2GB) to many (order of
several hundred) machines at once, with a granularity of, say, a day.
Believing (but not having benchmarked) that restoring file data from a
compressed repository is *much* more CPU-using than simply streaming from a
RAID array, I thought that the easy way would be to generate a tar file, and
let the clients fetch it.
Of course, I can do an export and tar that, but I wanted to ask because the
filesystem fragmentation is much lower if I don't need to do all that into a
temporary directory, and it could be done much *faster* too, if it didn't
need to write the data twice.
(Just get the data from the repos, and write a single stream, possibly even
into a raw device - which gets me easily more than 100MByte/sec transfer
rate).
> p.s. check out socat:
>
> http://www.dest-unreach.org/socat/doc/socat.html
Thank you, I already know netcat which does more or less the same.
> if you want to do something like this:
>
> svn cat /repository/path | socat [options]
Thank you, but that works only for files, not entire directory structures.
Regards,
Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svntar, anybody?
Posted by John Peacock <jp...@rowman.com>.
Ph. Marek wrote:
> Hello everybody,
>
> I'd like to pipe parts of a repository over some channel to another box.
Perhaps if you described what you actually hope to accomplish (e.g. keep
a remote filesystem in sync with a local repository), rather than a
specific mechanism (like pipe or tar), someone could make a more
intelligent suggestion. I suspect this should also happen on the user
rather than the dev list...
John
p.s. check out socat:
http://www.dest-unreach.org/socat/doc/socat.html
if you want to do something like this:
svn cat /repository/path | socat [options]
--
John Peacock
Director of Information Research and Technology
Rowman & Littlefield Publishing Group
4501 Forbes Boulevard
Suite H
Lanham, MD 20706
301-459-3366 x.5010
fax 301-429-5748
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svntar, anybody?
Posted by "Ph. Marek" <ph...@bmlv.gv.at>.
On Mittwoch, 4. Juli 2007, Martin Furter wrote:
> On Wed, 4 Jul 2007, Ph. Marek wrote:
> > - Does it handle long names, like GNU tar?
> I found a description of "ustar" format and implemented that. It handles
> filenames up to 253 chars (old tar handles only 99).
Thank you!
> > - Could you parse the properties used by the meta-data branch and FSVS,
> > to save the stored owner, group, mode and timestamp, please?
> >
> > (http://fsvs.tigris.org/source/browse/*checkout*/fsvs/trunk/www/doxygen/h
> >tml/props_8c.html#293b9c3d742482839335a0631d8affab)
>
> I don't know (yet) if I get the properties for free but it's definitely
> possible. Though there may be a problem with uid and gid, tar stores them
> as integers and as string. I'll have to find out how it behaves if I set
> only one of those.
>
> The fsvs documentation doesn't say if group and owner are stored as int or
> string (actually it says "Modification time - svn:group." ;).
It has both - First a number, then whitespace, and the rest is the name :-)
> > - Support for device nodes would be nice, too.
>
> How are they stored in the repos? Similar to symlinks?
Yes, with "[bc]dev:MAJOR:MINOR" for block/device nodes.
> > * Which license does apply?? GPL? BSD? Something else?
>
> Oooh, I still didn't add one...
> Is the subversion license OK or do you prefer another one?
Well, for FSVS I use GPL.
> > Otherwise, it does what I need ... Even includes compression, although
> > some "-h" or "-?" switch would be nice :-)
> I thought I get that for free... Will add it.
>
> Hmm, I guess it's time to get it into a repos and add a version number
> too...
Thank you very much!
Regards,
Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svntar, anybody?
Posted by Martin Furter <mf...@rola.ch>.
Hi Phil
On Wed, 4 Jul 2007, Ph. Marek wrote:
> Hello Martin!
>
> On Mittwoch, 4. Juli 2007, Martin Furter wrote:
>> I wrote one a while ago: http://svn.borg.ch/tmp/svntar/0.0.2/
>> It seems to do it's job but i'm sure there are tons of bugs in it.
>>
>> I was able to tar working copies an URLs with it (but forgot how it
>> works ;)
>>
>> If you try it please give me some feedback and i'll try to improve it.
> It seems to work just fine.
>
> Some points:
> - Does it handle long names, like GNU tar?
I found a description of "ustar" format and implemented that. It handles
filenames up to 253 chars (old tar handles only 99).
> - Could you parse the properties used by the meta-data branch and FSVS,
> to save the stored owner, group, mode and timestamp, please?
>
> (http://fsvs.tigris.org/source/browse/*checkout*/fsvs/trunk/www/doxygen/html/props_8c.html#293b9c3d742482839335a0631d8affab)
I don't know (yet) if I get the properties for free but it's definitely
possible. Though there may be a problem with uid and gid, tar stores them
as integers and as string. I'll have to find out how it behaves if I set
only one of those.
The fsvs documentation doesn't say if group and owner are stored as int or
string (actually it says "Modification time - svn:group." ;).
> - Support for device nodes would be nice, too.
How are they stored in the repos? Similar to symlinks?
> * Which license does apply?? GPL? BSD? Something else?
Oooh, I still didn't add one...
Is the subversion license OK or do you prefer another one?
> Otherwise, it does what I need ... Even includes compression, although
> some "-h" or "-?" switch would be nice :-)
I thought I get that for free... Will add it.
Hmm, I guess it's time to get it into a repos and add a version number
too...
Thanks.
Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svntar, anybody?
Posted by "Ph. Marek" <ph...@bmlv.gv.at>.
Hello Martin!
On Mittwoch, 4. Juli 2007, Martin Furter wrote:
> I wrote one a while ago: http://svn.borg.ch/tmp/svntar/0.0.2/
> It seems to do it's job but i'm sure there are tons of bugs in it.
>
> I was able to tar working copies an URLs with it (but forgot how it
> works ;)
>
> If you try it please give me some feedback and i'll try to improve it.
It seems to work just fine.
Some points:
- Does it handle long names, like GNU tar?
- Could you parse the properties used by the meta-data branch and FSVS,
to save the stored owner, group, mode and timestamp, please?
(http://fsvs.tigris.org/source/browse/*checkout*/fsvs/trunk/www/doxygen/html/props_8c.html#293b9c3d742482839335a0631d8affab)
- Support for device nodes would be nice, too.
* Which license does apply?? GPL? BSD? Something else?
Otherwise, it does what I need ... Even includes compression, although
some "-h" or "-?" switch would be nice :-)
Thank you!
Regards,
Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svntar, anybody?
Posted by Martin Furter <mf...@rola.ch>.
Hi Phil
I wrote one a while ago: http://svn.borg.ch/tmp/svntar/0.0.2/
It seems to do it's job but i'm sure there are tons of bugs in it.
I was able to tar working copies an URLs with it (but forgot how it
works ;)
If you try it please give me some feedback and i'll try to improve it.
HTH
Martin
On Mon, 2 Jul 2007, Ph. Marek wrote:
> Hello everybody,
>
> I'd like to pipe parts of a repository over some channel to another box.
>
> I found "svntar" mentioned here:
> http://svn.haxx.se/users/archive-2006-03/1310.shtml
>
> But that thread ends with a PHP solution that exports into a temporary
> directory, which is not feasible for my needs.
>
>
> Does someone know if such a thing does already exist?
>
>
> Regards,
>
> Phil
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svntar, anybody?
Posted by "Ph. Marek" <ph...@bmlv.gv.at>.
On Montag, 2. Juli 2007, Marcus Rueckert wrote:
> On 2007-07-02 13:58:43 +0200, Ph. Marek wrote:
> > I'd like to pipe parts of a repository over some channel to another box.
> >
> > I found "svntar" mentioned here:
> > http://svn.haxx.se/users/archive-2006-03/1310.shtml
> >
> > But that thread ends with a PHP solution that exports into a temporary
> > directory, which is not feasible for my needs.
> >
> >
> > Does someone know if such a thing does already exist?
>
> uhm i think you will always need a temporary export.
Why should I? There's a smbtar, after all, too.
(http://us1.samba.org/samba/docs/man/manpages-3/smbtar.1.html).
It's just about getting the files from the repository and streaming them on as
a tar file.
Regards,
Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svntar, anybody?
Posted by Marcus Rueckert <da...@web.de>.
On 2007-07-02 13:58:43 +0200, Ph. Marek wrote:
> I'd like to pipe parts of a repository over some channel to another box.
>
> I found "svntar" mentioned here:
> http://svn.haxx.se/users/archive-2006-03/1310.shtml
>
> But that thread ends with a PHP solution that exports into a temporary
> directory, which is not feasible for my needs.
>
>
> Does someone know if such a thing does already exist?
uhm i think you will always need a temporary export.
darix
--
openSUSE - SUSE Linux is my linux
openSUSE is good for you
www.opensuse.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: RE : svntar, anybody?
Posted by "Ph. Marek" <ph...@bmlv.gv.at>.
On Montag, 2. Juli 2007, Ionel GARDAIS wrote:
> Hi,
>
> With SVN 1.4, you should take a look at svnsync :
> http://svnbook.red-bean.com/nightly/en/svn.reposadmin.maint.html#svn.reposa
>dmin.maint.tk.svnsync
Thank you, but that doesn't help if there's only a tar at the remote site.
Regards,
Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
RE : svntar, anybody?
Posted by Ionel GARDAIS <ig...@yahoo.fr>.
Hi,
With SVN 1.4, you should take a look at svnsync :
http://svnbook.red-bean.com/nightly/en/svn.reposadmin.maint.html#svn.reposadmin.maint.tk.svnsync
Ionel
--- "Ph. Marek" <ph...@bmlv.gv.at> a écrit :
> Hello everybody,
>
> I'd like to pipe parts of a repository over some
> channel to another box.
>
> I found "svntar" mentioned here:
>
> http://svn.haxx.se/users/archive-2006-03/1310.shtml
>
> But that thread ends with a PHP solution that
> exports into a temporary
> directory, which is not feasible for my needs.
>
>
> Does someone know if such a thing does already
> exist?
>
>
> Regards,
>
> Phil
>
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail:
> dev-help@subversion.tigris.org
>
>
_____________________________________________________________________________
Ne gardez plus qu'une seule adresse mail ! Copiez vos mails vers Yahoo! Mail
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org