You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spamassassin.apache.org by Justin Mason <jm...@jmason.org> on 2004/04/03 02:13:05 UTC
Re: [alexis@panix.com: Fwd: Reminder: SpamAssassin needs your help!]
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Alexis Rosen writes:
>Hey, guys. I'm just following up because I'm curious about the final
>disposition of the nfslock code. Did you add a function to correspond to
>nfsunlock? I hope the rest of my comments were helpful.
Actually, we haven't :( It dropped off my radar, unfortunately.
I'm thinking we should take a look at that before 3.0.0 release, and
probably we should also add a *non*-NFS-safe locking function too for
hosts where NFS is not involved -- as I think we could cut down
lock overhead quite a lot that way.
- --j.
>/a
>
>On Fri, Oct 31, 2003 at 05:55:02PM -0500, Alexis Rosen said:
>> On Fri, Oct 31, 2003 at 10:29:12AM -0800, Justin Mason said:
>> > >That said, I've glanced at the code, and I see a couple of potential problems.
>> > >
>> > >I may be completely missing something here, but it looks like the lockfile
>> > >is uniqued by hostname. In that case, it's not locking anything, unless you
>> > >have some other means of guaranteeing that the file being locked will never
>> > >be touched by more than one host.
>> >
>> > I don't think so, at least -- the code is (as far as I can see):
>> >
>> > - generate random file we can always create: $path.lock.$hname.$$
>>
>> Right. Sorry, I was just looking at it quickly. Too quickly...
>>
>> However... I now remember thinking about using the hostname when I was
>> writing this. I decided that nfslock was already pretty expensive, and
>> that adding the hostname was really not an advantage.
>>
>> > >There is also the tricky problem that you need to have a corresponding unlock
>> > >function or you run the risk of trashing locks you don't own. See comments
>> > >in my code for details... wait, you probably don't have it. I'll send it
>> > >to you shortly.
>> >
>> > Interesting; we'd better check that.
>>
>> OK, here's the source. It's attached (only ~10k). A few things to think about:
>>
>> - Some of the issues involved are *really* subtle. Don't throw out the sleep
>> in nfsunlock, for example.
>> - The race condition that made me write nfsunlock may seem a virtual
>> impossibility, but if you are actually running across NFS, it's not the most
>> unlikely thing in the world. All you need is a small network problem at the
>> wrong time.
>> - nfs{,un}lock4 was written to deal with a bug in netbsd. It had a hideously
>> broken nfs client that didn't act atomically even for link(). Nothing you
>> can do about that except work around it as best you can. I encourage you to
>> NOT use this workaround- it's very expensive, and the problem no longer exists,
>> I think. nfstest is a tiny bit of code that can turn up systems with that
>> problem. I know of no other Unixes that have that problem.
>>
>> Any other questions, just ask.
>>
>> /a
>
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Exmh CVS
iD8DBQFAbgGRQTcbUG5Y7woRAnDAAKDFeym/cqnvL0A3NYPpzGOI/QzZewCgkZ/V
xNPkki2QsXaH/w1A0b5IiYA=
=h/xe
-----END PGP SIGNATURE-----
Re: [alexis@panix.com: Fwd: Reminder: SpamAssassin needs your help!]
Posted by Kelsey Cummings <kg...@sonic.net>.
On Mon, Apr 05, 2004 at 12:28:00PM -0700, Justin Mason wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Kelsey Cummings writes:
> > And, for that matter, at this point most (all?) NFS implementations
> > implement locking anyway. It'd be fairly easy to add this to the tests.
>
> Yes, but do they implement it *correctly*? ;) as far as I know that's
> the problem.
True that. Do you think that file locking tests in the make procedure
would be able to properly test implementation? In any case, it'd be nice
to see selectable locking code. Most installations can use flock on local
disks without any problem and would probably gain some performance. People
on NFS (I imagine a small number) could use whichever they felt comfortable
with.
--
Kelsey Cummings - kgc@sonic.net sonic.net, inc.
System Administrator 2260 Apollo Way
707.522.1000 (Voice) Santa Rosa, CA 95407
707.547.2199 (Fax) http://www.sonic.net/
Fingerprint = D5F9 667F 5D32 7347 0B79 8DB7 2B42 86B6 4E2C 3896
Re: [alexis@panix.com: Fwd: Reminder: SpamAssassin needs your help!]
Posted by Kelsey Cummings <kg...@sonic.net>.
On Mon, Apr 05, 2004 at 05:37:11PM -0400, Alexis Rosen wrote:
> On Sun, Apr 04, 2004 at 05:40:35PM -0700, Kelsey Cummings said:
> > And, for that matter, at this point most (all?) NFS implementations
> > implement locking anyway. It'd be fairly easy to add this to the tests.
>
> Yes, but be cautious. Locking over NFS has been claimed to work successfully
> for many years, and yet odd failures would pop up again and again. I'm
> thinking back to the SunOS 4 days, of course, but I'm still skeptical. IIRC,
> someone wrote a well-received paper many years ago (12? 15?) that proved
> mathematically that nfs locking was impossible to do correctly 100% of the
> time. Best to offer people the choice. This is mail and users/customers tend
> to get bent out of shape when it gets lost...
We were always cautious of nfslocking, which is why we contributed the
existing code as a safe method. However, since then, we've proven that
flock works for us just fine. This would be NFSv3 with linux clients and
NetApp filers.
As you say, YMMV.
--
Kelsey Cummings - kgc@sonic.net sonic.net, inc.
System Administrator 2260 Apollo Way
707.522.1000 (Voice) Santa Rosa, CA 95407
707.547.2199 (Fax) http://www.sonic.net/
Fingerprint = D5F9 667F 5D32 7347 0B79 8DB7 2B42 86B6 4E2C 3896
Re: [alexis@panix.com: Fwd: Reminder: SpamAssassin needs your help!]
Posted by Alexis Rosen <al...@panix.com>.
On Sun, Apr 04, 2004 at 05:40:35PM -0700, Kelsey Cummings said:
> And, for that matter, at this point most (all?) NFS implementations
> implement locking anyway. It'd be fairly easy to add this to the tests.
Yes, but be cautious. Locking over NFS has been claimed to work successfully
for many years, and yet odd failures would pop up again and again. I'm
thinking back to the SunOS 4 days, of course, but I'm still skeptical. IIRC,
someone wrote a well-received paper many years ago (12? 15?) that proved
mathematically that nfs locking was impossible to do correctly 100% of the
time. Best to offer people the choice. This is mail and users/customers tend
to get bent out of shape when it gets lost...
(BTW, I have no idea if the same issues exist in NFSv4. I would be unsurprised
either way.)
/a
Re: [alexis@panix.com: Fwd: Reminder: SpamAssassin needs your help!]
Posted by Kelsey Cummings <kg...@sonic.net>.
On Sun, Apr 04, 2004 at 05:00:55PM -0400, Alexis Rosen wrote:
> On Fri, Apr 02, 2004 at 04:13:05PM -0800, Justin Mason said:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> >
> > Alexis Rosen writes:
> > >Hey, guys. I'm just following up because I'm curious about the final
> > >disposition of the nfslock code. Did you add a function to correspond to
> > >nfsunlock? I hope the rest of my comments were helpful.
> >
> > Actually, we haven't :( It dropped off my radar, unfortunately.
> >
> > I'm thinking we should take a look at that before 3.0.0 release, and
>
> You *really* should.
>
> > probably we should also add a *non*-NFS-safe locking function too for
> > hosts where NFS is not involved -- as I think we could cut down
> > lock overhead quite a lot that way.
>
> Absoultely. NFSlock is at least an order of magnitude more expensive than
> using OS-supported locking schemes.
And, for that matter, at this point most (all?) NFS implementations
implement locking anyway. It'd be fairly easy to add this to the tests.
--
Kelsey Cummings - kgc@sonic.net sonic.net, inc.
System Administrator 2260 Apollo Way
707.522.1000 (Voice) Santa Rosa, CA 95407
707.547.2199 (Fax) http://www.sonic.net/
Fingerprint = D5F9 667F 5D32 7347 0B79 8DB7 2B42 86B6 4E2C 3896
Re: [alexis@panix.com: Fwd: Reminder: SpamAssassin needs your help!]
Posted by Alexis Rosen <al...@panix.com>.
On Fri, Apr 02, 2004 at 04:13:05PM -0800, Justin Mason said:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
> Alexis Rosen writes:
> >Hey, guys. I'm just following up because I'm curious about the final
> >disposition of the nfslock code. Did you add a function to correspond to
> >nfsunlock? I hope the rest of my comments were helpful.
>
> Actually, we haven't :( It dropped off my radar, unfortunately.
>
> I'm thinking we should take a look at that before 3.0.0 release, and
You *really* should.
> probably we should also add a *non*-NFS-safe locking function too for
> hosts where NFS is not involved -- as I think we could cut down
> lock overhead quite a lot that way.
Absoultely. NFSlock is at least an order of magnitude more expensive than
using OS-supported locking schemes.
/a