Ev2 question regarding properties?

[Justin Erenkrantz <ju...@erenkrantz.com>]

Ev2 folks,

I've been thinking a little bit about how to optimize the PROPFIND
requests in ra_serf.  The core issue we had with Ev1 was that we had
to delay setting any of the properties until we had the file handle
open - this is why, way back in 2006, I just coded it to have a
parallel HTTP request per entry and had to do tracking to ensure we
did the right thing about ordering of the HTTP requests/responses.

Will Ev2 offer some lightweight way to set properties on a bunch of
local WC paths?  Do the textbases already need to be present?  (I'm
thinking here about how we would handle EOL properties.)  Will we need
to keep the WC file handle open once we set the properties?  ISTR that
a Ev1 characteristic was that you can't re-open a file multiple times
in an editor drive.  And, we couldn't keep too many file handles open
or we would exhaust ulimits and such.  Hence, thinking back on it now,
the reason we ended up where we did with ra_serf wasn't as much about
HTTP or serf - but more about the original editor API.

I'm guessing that if Ev2 doesn't cleanly support allowing properties
to be set completely independently from the textbases, we may want to
consider modifying Ev2 to support this in order to allow ra_serf to
batch up properties in one go or at least batched per-directory.

I'm happy to hear suggestions and thoughts...especially if the time
machine has solved this one already.  =)  -- justin

Re: Ev2 question regarding properties?

[Greg Stein <gs...@gmail.com>]

On Sat, May 19, 2012 at 10:15 AM, Justin Erenkrantz
<ju...@erenkrantz.com> wrote:
> On Sat, May 19, 2012 at 7:09 AM, Bert Huijben <be...@qqmail.nl> wrote:
>...
>> E.g. for directories we update the list of children and the properties in
>> one operation, to avoid the old 'incomplete' state of editor v1.
>
> Does that also mean the properties of all of the children as well?

Nope. Just the props/children of the directory. Once that has been
done, then the directory has been updated.

Children are updated separately.

When the new set of children is specified, we can create "incomplete"
child nodes for any new children. At some point in the future, they
will get added with their data. The children are otherwise ignored
(ie. we don't immediately delete any children; we wait for a delete())

> If so, this may be why Greg has suggested a PROPFIND Depth: 1 call as
> that'll mesh with the Ev2 semantics if that's the case.

Nope. Just as a mechanism for reducing traffic (due to HTTP
request/response overhead), and to reduce server CPU (fewer requests
to process).

Regarding the other stuff: there are no batched requests in Ev2,
today. And each node is updated in a single call, as Bert indicated.

File handles should not be a problem since you do one of:

add_directory(path, contents, props)
alter_directory(path, contents, props)

So the file handle for 'contents' only needs to be open for the
duration of that call. Either one (but not both) may be NULL if only
one aspect will be altered. Two calls are not allowed for a given path
(eg. you can't set contents, then later set props).

Cheers,
-g

Re: Ev2 question regarding properties?

[Justin Erenkrantz <ju...@erenkrantz.com>]

On Sat, May 19, 2012 at 7:09 AM, Bert Huijben <be...@qqmail.nl> wrote:
> No, the idea of editor v2 is that we update a node in a single atomic
> operation.
>
> If we would update just the properties or just the text, we have a node that
> is neither at its old revision, nor at its new revision. And that is
> something that we try to avoid in editor v2.
>
> E.g. for directories we update the list of children and the properties in
> one operation, to avoid the old 'incomplete' state of editor v1.

Does that also mean the properties of all of the children as well?

If so, this may be why Greg has suggested a PROPFIND Depth: 1 call as
that'll mesh with the Ev2 semantics if that's the case.

BTW, thanks for the reply!  -- justin

RE: Ev2 question regarding properties?

[Bert Huijben <be...@qqmail.nl>]

> -----Original Message-----
> From: justin.erenkrantz@gmail.com [mailto:justin.erenkrantz@gmail.com] On
> Behalf Of Justin Erenkrantz
> Sent: zaterdag 19 mei 2012 16:02
> To: Subversion Development
> Subject: Ev2 question regarding properties?
> 
> Ev2 folks,
> 
> I've been thinking a little bit about how to optimize the PROPFIND
> requests in ra_serf.  The core issue we had with Ev1 was that we had
> to delay setting any of the properties until we had the file handle
> open - this is why, way back in 2006, I just coded it to have a
> parallel HTTP request per entry and had to do tracking to ensure we
> did the right thing about ordering of the HTTP requests/responses.
> 
> Will Ev2 offer some lightweight way to set properties on a bunch of
> local WC paths?  Do the textbases already need to be present?  (I'm
> thinking here about how we would handle EOL properties.)  Will we need
> to keep the WC file handle open once we set the properties?  ISTR that
> a Ev1 characteristic was that you can't re-open a file multiple times
> in an editor drive.  And, we couldn't keep too many file handles open
> or we would exhaust ulimits and such.  Hence, thinking back on it now,
> the reason we ended up where we did with ra_serf wasn't as much about
> HTTP or serf - but more about the original editor API.

No, the idea of editor v2 is that we update a node in a single atomic
operation.

If we would update just the properties or just the text, we have a node that
is neither at its old revision, nor at its new revision. And that is
something that we try to avoid in editor v2.

E.g. for directories we update the list of children and the properties in
one operation, to avoid the old 'incomplete' state of editor v1.

> I'm guessing that if Ev2 doesn't cleanly support allowing properties
> to be set completely independently from the textbases, we may want to
> consider modifying Ev2 to support this in order to allow ra_serf to
> batch up properties in one go or at least batched per-directory.

Editor v2 was explicitly designed to *not* allow this :)

And I leave answering the rest to those that are actively working on the
editor. 
(Which I'm not. I'm still not sure that editor v2 will really gain us enough
performance, given that the current update process handles a lot of
obstruction and conflict cases on opening parent directories. We'll see if
that scales to per node updates.)

	Bert