You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Dirk-Willem van Gulik <di...@webweaving.org> on 2011/08/24 13:33:12 UTC
Mitigation Range header (Was: DoS with mod_deflate & range requests)
Folks,
This issue is now active in the wild. So some unified/simple comms is needed.
What is the wisdom on mitigation advise/briefing until a proper fix it out - in order of ease:
-> Where possible - disable mod_deflate
=> we sure this covers all cases - or this is a good stopgap ?
-> Where possible - set LimitRequestFieldSize to a small value
-> Suggesting of 128 fine ?
-> Where this is not possible (e.g. long cookies, auth headers of serious size) consider using
mod_rewrite to not accept more than a few commas
=> anyone a config snipped for this ?
-> Perhaps a stop gap module
http://people.apache.org/~dirkx/mod_rangecnt.c (is this kosher??)
-> Apply patch XXX from the mailing list
Any thoughts ? Followed by a - upgrade as soon as a release is made
Thanks,
Dw
RE: Mitigation Range header
Posted by "Plüm, Rüdiger, VF-Group" <ru...@vodafone.com>.
________________________________
From: Greg Ames [mailto:ames.greg@gmail.com]
Sent: Mittwoch, 24. August 2011 18:20
To: dev@httpd.apache.org
Subject: Re: Mitigation Range header
On Wed, Aug 24, 2011 at 10:33 AM, "Plüm, Rüdiger, VF-Group" <ru...@vodafone.com> wrote:
I think mod_deflate is just the tool to convert an O(N^2) data size problem into an O(N^2) CPU usage problem, where N is some function of LimitRequestLine. If the file size is smaller than the largest range end used in the attack, it may reduce the amount of data actually going down the filter chain.
Greg
I don't think so. The compression happens before the byterange filter and the byterange filter just hacks the already compressed brigade into more buckets and rearranges them.
mod_deflate does not do more work if it is a range request. It does the same amount of work as for the non range request.
OK, thanks for the clarification, Rüdiger. Then I don't understand why mod_deflate seems to be an important factor in killing the server.
If the DEFLATE filter runs first, can you do anything useful with a subrange of its output? i.e., could a client decompress a subrange that starts in the middle of the compressed version and get a subrange of the original uncompressed data?
Greg
It depends. Think of a download that was broken in the middle and that was compressed by mod_deflate on the fly. A client could just request the missing bits and decompress the whole thing
afterwards.
I guess in general it only works if the client has all the missing pieces (or at least the ones before the contents of the partial response) that are not sent via the partial response at hand to
decompress the whole stream (or decompress at least until the last part of the partial response).
Regards
Rüdiger
Re: Mitigation Range header
Posted by Greg Ames <am...@gmail.com>.
On Wed, Aug 24, 2011 at 10:33 AM, "Plüm, Rüdiger, VF-Group" <
ruediger.pluem@vodafone.com> wrote:
> **
>
>
> I think mod_deflate is just the tool to convert an O(N^2) data size problem
> into an O(N^2) CPU usage problem, where N is some function of
> LimitRequestLine. If the file size is smaller than the largest range end
> used in the attack, it may reduce the amount of data actually going down the
> filter chain.
>
> Greg
>
>
>
> I don't think so. The compression happens before the byterange filter and
> the byterange filter just hacks the already compressed brigade into more
> buckets and rearranges them.
> mod_deflate does not do more work if it is a range request. It does the
> same amount of work as for the non range request.
>
> OK, thanks for the clarification, Rüdiger. Then I don't understand why
mod_deflate seems to be an important factor in killing the server.
If the DEFLATE filter runs first, can you do anything useful with a subrange
of its output? i.e., could a client decompress a subrange that starts in
the middle of the compressed version and get a subrange of the original
uncompressed data?
Greg
RE: Mitigation Range header
Posted by "Plüm, Rüdiger, VF-Group" <ru...@vodafone.com>.
________________________________
From: Greg Ames Sent: Mittwoch, 24. August 2011 16:05
To: dev@httpd.apache.org
Subject: Re: Mitigation Range header
On Wed, Aug 24, 2011 at 9:01 AM, "Plüm, Rüdiger, VF-Group" <ru...@vodafone.com> wrote:
> Hmm - when I remove mod_deflate (i.e. explicitly as it is the
> default in all our installs) and test on a / entry which is a
> static file which is large (100k)* - then I cannot get apache
> on its knees on a freebsd machine - saturating the 1Gbit
> connection it has (Note: the attack machines *are* getting
> saturated). The moment i put in mod_deflate, mod_external
> filter, etc - it is much easier to get deplete enough
> resources to notice.
>
> Dw.
>
> *: as I cannot reproduce the issue with very small index.html files.
Have you tried if the same happens with mod_deflate, but with one of the
the proposed mitigations in place?
As said my guess is that this might be an issue with mod_deflate that
is unrelated to the Range request issue.
I think mod_deflate is just the tool to convert an O(N^2) data size problem into an O(N^2) CPU usage problem, where N is some function of LimitRequestLine. If the file size is smaller than the largest range end used in the attack, it may reduce the amount of data actually going down the filter chain.
Greg
I don't think so. The compression happens before the byterange filter and the byterange filter just hacks the already compressed brigade into more buckets and rearranges them.
mod_deflate does not do more work if it is a range request. It does the same amount of work as for the non range request.
Regards
Rüdiger
Re: Mitigation Range header
Posted by Greg Ames <am...@gmail.com>.
On Wed, Aug 24, 2011 at 9:01 AM, "Plüm, Rüdiger, VF-Group" <
ruediger.pluem@vodafone.com> wrote:
>
> > Hmm - when I remove mod_deflate (i.e. explicitly as it is the
> > default in all our installs) and test on a / entry which is a
> > static file which is large (100k)* - then I cannot get apache
> > on its knees on a freebsd machine - saturating the 1Gbit
> > connection it has (Note: the attack machines *are* getting
> > saturated). The moment i put in mod_deflate, mod_external
> > filter, etc - it is much easier to get deplete enough
> > resources to notice.
> >
> > Dw.
> >
> > *: as I cannot reproduce the issue with very small index.html files.
>
> Have you tried if the same happens with mod_deflate, but with one of the
> the proposed mitigations in place?
> As said my guess is that this might be an issue with mod_deflate that
> is unrelated to the Range request issue.
>
I think mod_deflate is just the tool to convert an O(N^2) data size problem
into an O(N^2) CPU usage problem, where N is some function of
LimitRequestLine. If the file size is smaller than the largest range end
used in the attack, it may reduce the amount of data actually going down the
filter chain.
Greg
Re: Mitigation Range header
Posted by Dirk-WIllem van Gulik <di...@webweaving.org>.
On 24 Aug 2011, at 14:01, Plüm, Rüdiger, VF-Group wrote:
> Have you tried if the same happens with mod_deflate, but with one of the
> the proposed mitigations in place?
As soon as I reject/block on the range header - all is fine again.
> As said my guess is that this might be an issue with mod_deflate that
> is unrelated to the Range request issue.
Good point.
Dw
RE: Mitigation Range header
Posted by "Plüm, Rüdiger, VF-Group" <ru...@vodafone.com>.
> -----Original Message-----
> From: Dirk-WIllem van Gulik [mailto:dirkx@webweaving.org]
> Sent: Mittwoch, 24. August 2011 14:40
> To: dev@httpd.apache.org
> Cc: Plüm, Rüdiger, VF-Group
> Subject: Re: Mitigation Range header
>
>
> On 24 Aug 2011, at 13:22, Florian Weimer wrote:
>
> > * Plüm, Rüdiger, VF-Group:
> >
> >> As said this has *nothing* to do with mod_deflate. This
> was IMHO just
> >> a guess by the original author of the tool.
> >
> > This matches my testing, too. I see a significant peak in
> RAM usage on
> > a server where "apachectl -M" does not print anything with
> the string
> > "deflate" (so I assume that mod_deflate is not enabled).
> This is with
> > 2.2.9-10+lenny9 on Debian.
> >
> > If it is more difficult to check if mod_deflate is enabled,
> the advisory
> > should tell how to check your server. If the method I used is the
> > correct one, I don't think it's reasonable to suggest disabling
> > mod_deflate as a mitigation because it does not seem to
> make much of a
> > difference.
>
> Hmm - when I remove mod_deflate (i.e. explicitly as it is the
> default in all our installs) and test on a / entry which is a
> static file which is large (100k)* - then I cannot get apache
> on its knees on a freebsd machine - saturating the 1Gbit
> connection it has (Note: the attack machines *are* getting
> saturated). The moment i put in mod_deflate, mod_external
> filter, etc - it is much easier to get deplete enough
> resources to notice.
>
> Dw.
>
> *: as I cannot reproduce the issue with very small index.html files.
Have you tried if the same happens with mod_deflate, but with one of the
the proposed mitigations in place?
As said my guess is that this might be an issue with mod_deflate that
is unrelated to the Range request issue.
Regards
Rüdiger
Re: Mitigation Range header
Posted by Florian Weimer <fw...@bfk.de>.
* Dirk-WIllem van Gulik:
> On 24 Aug 2011, at 13:43, Florian Weimer wrote:
>
>> * Dirk-WIllem van Gulik:
>>
>>> Hmm - when I remove mod_deflate (i.e. explicitly as it is the default
>>> in all our installs) and test on a / entry which is a static file
>>> which is large (100k)* - then I cannot get apache on its knees on a
>>> freebsd machine - saturating the 1Gbit connection it has (Note: the
>>> attack machines *are* getting saturated). The moment i put in
>>> mod_deflate, mod_external filter, etc - it is much easier to get
>>> deplete enough resources to notice.
>>
>> Oh. Have you checked memory usage on the server?
>
> I had not - and you are right - quite high. I also tried it on a
> Ubuntu machine - and that one dies right out of the gate - regardless
> as to wether deflate is on- or off.
It seems that this reflects different approaches to memory
overcommitment. I didn't see any crashes with vm.overcommit_memory=2 on
Linux, either. I wouldn't mention this in the advisory, though, because
even if no critical process is terminated due to the out-of-memory
condition, thrashing could still push the system beyond the point of
recovery.
Including the increased memory usage in the adviosry, as a potential
attack indicator, would make sense, IMHO.
--
Florian Weimer <fw...@bfk.de>
BFK edv-consulting GmbH http://www.bfk.de/
Kriegsstraße 100 tel: +49-721-96201-1
D-76133 Karlsruhe fax: +49-721-96201-99
Re: Mitigation Range header
Posted by Dirk-WIllem van Gulik <di...@webweaving.org>.
On 24 Aug 2011, at 13:43, Florian Weimer wrote:
> * Dirk-WIllem van Gulik:
>
>> Hmm - when I remove mod_deflate (i.e. explicitly as it is the default
>> in all our installs) and test on a / entry which is a static file
>> which is large (100k)* - then I cannot get apache on its knees on a
>> freebsd machine - saturating the 1Gbit connection it has (Note: the
>> attack machines *are* getting saturated). The moment i put in
>> mod_deflate, mod_external filter, etc - it is much easier to get
>> deplete enough resources to notice.
>
> Oh. Have you checked memory usage on the server?
I had not - and you are right - quite high. I also tried it on a Ubuntu machine - and that one dies right out of the gate - regardless as to wether deflate is on- or off.
So I guess this is somewhat OS specific - but indeed - not overly deflate specific. Deflate just does something.
Ok - let me rewrite advisory draft !
Dw
Re: Mitigation Range header
Posted by Florian Weimer <fw...@bfk.de>.
* Dirk-WIllem van Gulik:
> Hmm - when I remove mod_deflate (i.e. explicitly as it is the default
> in all our installs) and test on a / entry which is a static file
> which is large (100k)* - then I cannot get apache on its knees on a
> freebsd machine - saturating the 1Gbit connection it has (Note: the
> attack machines *are* getting saturated). The moment i put in
> mod_deflate, mod_external filter, etc - it is much easier to get
> deplete enough resources to notice.
Oh. Have you checked memory usage on the server?
--
Florian Weimer <fw...@bfk.de>
BFK edv-consulting GmbH http://www.bfk.de/
Kriegsstraße 100 tel: +49-721-96201-1
D-76133 Karlsruhe fax: +49-721-96201-99
Re: Mitigation Range header
Posted by Dirk-WIllem van Gulik <di...@webweaving.org>.
On 24 Aug 2011, at 13:22, Florian Weimer wrote:
> * Plüm, Rüdiger, VF-Group:
>
>> As said this has *nothing* to do with mod_deflate. This was IMHO just
>> a guess by the original author of the tool.
>
> This matches my testing, too. I see a significant peak in RAM usage on
> a server where "apachectl -M" does not print anything with the string
> "deflate" (so I assume that mod_deflate is not enabled). This is with
> 2.2.9-10+lenny9 on Debian.
>
> If it is more difficult to check if mod_deflate is enabled, the advisory
> should tell how to check your server. If the method I used is the
> correct one, I don't think it's reasonable to suggest disabling
> mod_deflate as a mitigation because it does not seem to make much of a
> difference.
Hmm - when I remove mod_deflate (i.e. explicitly as it is the default in all our installs) and test on a / entry which is a static file which is large (100k)* - then I cannot get apache on its knees on a freebsd machine - saturating the 1Gbit connection it has (Note: the attack machines *are* getting saturated). The moment i put in mod_deflate, mod_external filter, etc - it is much easier to get deplete enough resources to notice.
Dw.
*: as I cannot reproduce the issue with very small index.html files.
Re: Mitigation Range header
Posted by Florian Weimer <fw...@bfk.de>.
* Plüm, Rüdiger, VF-Group:
> As said this has *nothing* to do with mod_deflate. This was IMHO just
> a guess by the original author of the tool.
This matches my testing, too. I see a significant peak in RAM usage on
a server where "apachectl -M" does not print anything with the string
"deflate" (so I assume that mod_deflate is not enabled). This is with
2.2.9-10+lenny9 on Debian.
If it is more difficult to check if mod_deflate is enabled, the advisory
should tell how to check your server. If the method I used is the
correct one, I don't think it's reasonable to suggest disabling
mod_deflate as a mitigation because it does not seem to make much of a
difference.
--
Florian Weimer <fw...@bfk.de>
BFK edv-consulting GmbH http://www.bfk.de/
Kriegsstraße 100 tel: +49-721-96201-1
D-76133 Karlsruhe fax: +49-721-96201-99
RE: Mitigation Range header (Was: DoS with mod_deflate & range requests)
Posted by "Plüm, Rüdiger, VF-Group" <ru...@vodafone.com>.
> -----Original Message-----
> From: Eric Covener [mailto:covener@gmail.com]
> Sent: Mittwoch, 24. August 2011 14:59
> To: dev@httpd.apache.org
> Subject: Re: Mitigation Range header (Was: DoS with
> mod_deflate & range requests)
>
> > Of course it should have been:
> >
> > RewriteCond %{HTTP:range} !^bytes=[^,]+(,[^,]+){0,4}$
> > RewriteRule .* - [F]
>
> The problem with the negation is you need an addl rule to handle an
> empty range header, this would forbid normal non-range requests.
Damn it. Got me again. How about this:
RewriteCond %{HTTP:range} !(^bytes=[^,]+(,[^,]+){0,4}$|^$)
RewriteRule .* - [F]
Regards
Rüdiger
Re: Mitigation Range header (Was: DoS with mod_deflate & range requests)
Posted by Eric Covener <co...@gmail.com>.
> Of course it should have been:
>
> RewriteCond %{HTTP:range} !^bytes=[^,]+(,[^,]+){0,4}$
> RewriteRule .* - [F]
The problem with the negation is you need an addl rule to handle an
empty range header, this would forbid normal non-range requests.
--
Eric Covener
covener@gmail.com
RE: Mitigation Range header (Was: DoS with mod_deflate & range requests)
Posted by "Plüm, Rüdiger, VF-Group" <ru...@vodafone.com>.
> -----Original Message-----
> From: Eric Covener [mailto:covener@gmail.com]
> Sent: Mittwoch, 24. August 2011 14:05
> To: dev@httpd.apache.org
> Subject: Re: Mitigation Range header (Was: DoS with
> mod_deflate & range requests)
>
> On Wed, Aug 24, 2011 at 7:57 AM, "Plüm, Rüdiger, VF-Group"
> <ru...@vodafone.com> wrote:
> >
> >
> >> -----Original Message-----
> >> From: Dirk-Willem van Gulik
> >> Sent: Mittwoch, 24. August 2011 13:33
> >> To: dev@httpd.apache.org
> >> Subject: Mitigation Range header (Was: DoS with mod_deflate &
> >> range requests)
> >>
> >> Folks,
> >>
> >> This issue is now active in the wild. So some unified/simple
> >> comms is needed.
> >>
> >> What is the wisdom on mitigation advise/briefing until a
> >> proper fix it out - in order of ease:
> >>
> >> -> Where possible - disable mod_deflate
> >>
> >> => we sure this covers all cases - or this is a good
> stopgap ?
> >
> > As said this has *nothing* to do with mod_deflate. This was
> IMHO just
> > a guess by the original author of the tool.
> >
> >>
> >> -> Where possible - set LimitRequestFieldSize to a small value
> >>
> >> -> Suggesting of 128 fine ?
> >>
> >> -> Where this is not possible (e.g. long cookies, auth
> >> headers of serious size) consider using
> >> mod_rewrite to not accept more than a few commas
> >>
> >> => anyone a config snipped for this ?
> >
> > How about the following (untested) rewrite rule. It should
> only allow 5
> > ranges at max.
> >
> > RewriteCond %{HTTP:range} ^bytes=[^,]+(,[^,]+){0,4}$
> > RewriteRule .* - [F]
>
> Is [E=no-gzip] enough to avoid the downward spiral, for the sake of
> false positives?
As said it has IMHO nothing to do with mod_deflate. It is an issue of the byte range filter.
>
> But your regex matches when there's just a couple of ranges -- maybe
> {4} and no $?
Of course it should have been:
RewriteCond %{HTTP:range} !^bytes=[^,]+(,[^,]+){0,4}$
RewriteRule .* - [F]
As said untested. Thanks for remote eyes.
Regards
Rüdiger
Re: Mitigation Range header (Was: DoS with mod_deflate & range requests)
Posted by Eric Covener <co...@gmail.com>.
On Wed, Aug 24, 2011 at 7:57 AM, "Plüm, Rüdiger, VF-Group"
<ru...@vodafone.com> wrote:
>
>
>> -----Original Message-----
>> From: Dirk-Willem van Gulik
>> Sent: Mittwoch, 24. August 2011 13:33
>> To: dev@httpd.apache.org
>> Subject: Mitigation Range header (Was: DoS with mod_deflate &
>> range requests)
>>
>> Folks,
>>
>> This issue is now active in the wild. So some unified/simple
>> comms is needed.
>>
>> What is the wisdom on mitigation advise/briefing until a
>> proper fix it out - in order of ease:
>>
>> -> Where possible - disable mod_deflate
>>
>> => we sure this covers all cases - or this is a good stopgap ?
>
> As said this has *nothing* to do with mod_deflate. This was IMHO just
> a guess by the original author of the tool.
>
>>
>> -> Where possible - set LimitRequestFieldSize to a small value
>>
>> -> Suggesting of 128 fine ?
>>
>> -> Where this is not possible (e.g. long cookies, auth
>> headers of serious size) consider using
>> mod_rewrite to not accept more than a few commas
>>
>> => anyone a config snipped for this ?
>
> How about the following (untested) rewrite rule. It should only allow 5
> ranges at max.
>
> RewriteCond %{HTTP:range} ^bytes=[^,]+(,[^,]+){0,4}$
> RewriteRule .* - [F]
Is [E=no-gzip] enough to avoid the downward spiral, for the sake of
false positives?
But your regex matches when there's just a couple of ranges -- maybe
{4} and no $?
RE: Mitigation Range header (Was: DoS with mod_deflate & range requests)
Posted by "Plüm, Rüdiger, VF-Group" <ru...@vodafone.com>.
> -----Original Message-----
> From: Dirk-Willem van Gulik [mailto:dirkx@webweaving.org]
> Sent: Mittwoch, 24. August 2011 14:14
> To: dev@httpd.apache.org
> Subject: Re: Mitigation Range header (Was: DoS with
> mod_deflate & range requests)
>
>
> On 24 Aug 2011, at 12:57, Plüm, Rüdiger, VF-Group wrote:
>
> >> -> Where possible - disable mod_deflate
> >>
> >> => we sure this covers all cases - or this is a good stopgap ?
> >
> > As said this has *nothing* to do with mod_deflate. This was
> IMHO just
> > a guess by the original author of the tool.
>
> Ok - but when I try it on my servers (with the check of the
> tool removed) - it seems quite impotent unless mod_deflate
> is in the wire.
Hm, weird. I would guess that mod_deflate could even mitigate this attack as the byterange
filter only does something if it sees the whole response in the brigade the first time it
is called. Having mod_deflate compressing larger chunks of data causes the byterange filter
to be called multiple times with only parts of the response in the brigade.
So the byte range filter should only be applied with responses whose compressed response
fits into the zlibs output filter.
Depending on the size of the input and the number of parallel requests it might be possible
that a lot of memory is consumed by mod_deflate anyway. But I would expect the same behviour
without range requests as well.
>
> And it seems a bit more potent when there is other 'keep in
> the air' modules around.
>
> So I guess mod_deflate is right now the largest 'plug' we
> have in the server which can cause this backup ?
>
> Or is that totally wrong. Happy to stand correctede !
>
>
> >> -> Where possible - set LimitRequestFieldSize to a small value
> >>
> >> -> Suggesting of 128 fine ?
> >>
> >> -> Where this is not possible (e.g. long cookies, auth
> >> headers of serious size) consider using
> >> mod_rewrite to not accept more than a few commas
> >>
> >> => anyone a config snipped for this ?
> >
> > How about the following (untested) rewrite rule. It should
> only allow 5
> > ranges at max.
> >
> > RewriteCond %{HTTP:range} ^bytes=[^,]+(,[^,]+){0,4}$
> > RewriteRule .* - [F]
>
>
> Sounds like a plan ! This mail crossed one I just sent out -
> lemme update that too.
Please see my response to Eric. He detected an error in the above.
Regards
Rüdiger
Re: Mitigation Range header (Was: DoS with mod_deflate & range requests)
Posted by Dirk-Willem van Gulik <di...@webweaving.org>.
On 24 Aug 2011, at 12:57, Plüm, Rüdiger, VF-Group wrote:
>> -> Where possible - disable mod_deflate
>>
>> => we sure this covers all cases - or this is a good stopgap ?
>
> As said this has *nothing* to do with mod_deflate. This was IMHO just
> a guess by the original author of the tool.
Ok - but when I try it on my servers (with the check of the tool removed) - it seems quite impotent unless mod_deflate is in the wire.
And it seems a bit more potent when there is other 'keep in the air' modules around.
So I guess mod_deflate is right now the largest 'plug' we have in the server which can cause this backup ?
Or is that totally wrong. Happy to stand correctede !
>> -> Where possible - set LimitRequestFieldSize to a small value
>>
>> -> Suggesting of 128 fine ?
>>
>> -> Where this is not possible (e.g. long cookies, auth
>> headers of serious size) consider using
>> mod_rewrite to not accept more than a few commas
>>
>> => anyone a config snipped for this ?
>
> How about the following (untested) rewrite rule. It should only allow 5
> ranges at max.
>
> RewriteCond %{HTTP:range} ^bytes=[^,]+(,[^,]+){0,4}$
> RewriteRule .* - [F]
Sounds like a plan ! This mail crossed one I just sent out - lemme update that too.
Dw.
RE: Mitigation Range header (Was: DoS with mod_deflate & range requests)
Posted by "Plüm, Rüdiger, VF-Group" <ru...@vodafone.com>.
> -----Original Message-----
> From: Dirk-Willem van Gulik
> Sent: Mittwoch, 24. August 2011 13:33
> To: dev@httpd.apache.org
> Subject: Mitigation Range header (Was: DoS with mod_deflate &
> range requests)
>
> Folks,
>
> This issue is now active in the wild. So some unified/simple
> comms is needed.
>
> What is the wisdom on mitigation advise/briefing until a
> proper fix it out - in order of ease:
>
> -> Where possible - disable mod_deflate
>
> => we sure this covers all cases - or this is a good stopgap ?
As said this has *nothing* to do with mod_deflate. This was IMHO just
a guess by the original author of the tool.
>
> -> Where possible - set LimitRequestFieldSize to a small value
>
> -> Suggesting of 128 fine ?
>
> -> Where this is not possible (e.g. long cookies, auth
> headers of serious size) consider using
> mod_rewrite to not accept more than a few commas
>
> => anyone a config snipped for this ?
How about the following (untested) rewrite rule. It should only allow 5
ranges at max.
RewriteCond %{HTTP:range} ^bytes=[^,]+(,[^,]+){0,4}$
RewriteRule .* - [F]
Regards
Rüdiger