a load_response transaction-type thing

[Karl Lehenbauer <ka...@gmail.com>]

This talk of new capabilities for a new release of Rivet has had me revisiting something I've been thinking about for quite a while...

It's really easy for developers to use forms in a way that they shouldn't, putting stuff in hidden fields, etc, that could easily be monkeyed with by an attacker.  That data needs to be kept server side and accessed with an opaque, unguessable session ID cookie or equivalent.  Yes, yes… clear, but not simple, and that friction helps to cause the observed problem.

It's also super common for developers to accept input from forms with insufficient checks on the validity of the input data, the source of SQL injection attacks and so forth.

I'm thinking of what you might call a response broker.  (This isn't an original idea -- I've read about stuff like this.)

Rather than doing a load_response, the page handling the response would explicitly name the fields you expect to get from the form, you invoke a proc specifying the fields you expect to find, their data types, optional code to validate them, and an array to stuff the validated fields into.

set wantVarList {{username string} {id integer} {uid check_routine validate_uid} {password check_routine validate_password} {hash base64} {email email}}

set status [response_broker $wantVarList response]

Since every page using the response broker would need to check for response broker parse failures it would probably be nice to be able to specify a general handler routine that would run and then abort the page, removing the need to check the return.

Now if I run…

response_broker $wantVarList response

…if the page continues then the response array would contain validated fields found in the form for the variables named in wantVarList and no others.  You might want the presence of unexpected fields to also blow out, but that would be likely to bite you pretty often when the reasons are harmless.  Maybe log them and include a {field ignore} option that will inhibit logging for expected-but-ignored fields.

We've done a form package at FlightAware that has a specific look and feel that allows you to specify both Tcl and Javascript validation code.  Of course you still need the Tcl code on the server but it's nice in the modern era to also provide Javascript validation on the browser.  We could probably open source if its appearance was genericized or something, but I mention it mainly to point out the usefulness of such an approach both for the developer and the users and a likely need to push Rivet into providing more stuff to support developers making modern websites.

Re: a load_response transaction-type thing

[Massimo Manghi <mx...@apache.org>]

  Some thoughts about Karl's proposal for an exception based broker 
handling wrong or malformed or even forged form data.  I'm reckoning 
from the module's point of view to understand where the handler of such 
exception should be placed.

  As I said the similarities with AbortScript are many. In fact 
mod_rivet chains together various scripts into a single script for 
evaluation. If the script thus composed fails either AbortScript or 
ErrorScript catch the execution provided they're defined. In every 
circumstance AfterEveryScript works as the finale of a content 
generation. Unless we break the chain BeforeScript-> Referenced script 
-> AfterScript into 2 or 3 script evaluations we are left with the only 
chance to handle this sort of exception at the same stage AbortScript is 
evaluated. This would be largely an overlapping role leading to 
redundant code.

  The simplest alternative is to exploit abort_page ability of accepting 
as argument whatever lvalue Tcl can produce, either string, list or 
dictionary. The broker might work as a wrapper around abort_script 
stuffing in some dictionary information about what piece of information 
didn't pass the tests and some other metadata identifying the origin of 
the exception. The benefit would be simple: we need just to implement 
response_broker and handle the abort condition properly, perhaps with 
some specialized interface to hide from the application layer the actual 
implementation of message passing through the dictionary.

An alternative or a complement would be to generalize the exception 
handling making AbortScript a special case of a new handler definition.

RivetServerConf ExceptionHandler <EXCEPTION-CODE> "handler code"

AbortScript would be simply equivalent to

RivetServerConf ExceptionHandler ABORTPAGE "abort script code"

implementation of this handler could be done by stuffing in the 
configuration a hash table associating <EXCEPTION-CODE> to the handler 
code. To exploit completely this new way to trigger abnormal condition 
handling we should also extend abort_page to accept some extra argument. E.g

abort_script -code MYHANDLERCODE $exceptionCode

and it will trigger the handler corresponding to MYHANDLERCODE

abort_page $abortCode

would by default by equivalent to

abort_page -code ABORTPAGE $abortCode


This message has grown perhaps too long and it's become a sort of short TIP

thoughts?

  -- Massimo



On 01/08/2013 08:39 AM, Karl Lehenbauer wrote:
> This talk of new capabilities for a new release of Rivet has had me
> revisiting something I've been thinking about for quite a while...
>
> It's really easy for developers to use forms in a way that they
> shouldn't, putting stuff in hidden fields, etc, that could easily be
> monkeyed with by an attacker. That data needs to be kept server side and
> accessed with an opaque, unguessable session ID cookie or equivalent.
> Yes, yes… clear, but not simple, and that friction helps to cause the
> observed problem.
>
> It's also super common for developers to accept input from forms with
> insufficient checks on the validity of the input data, the source of SQL
> injection attacks and so forth.
>
> I'm thinking of what you might call a response broker. (This isn't an
> original idea -- I've read about stuff like this.)
>
> Rather than doing a load_response, the page handling the response would
> explicitly name the fields you expect to get from the form, you invoke a
> proc specifying the fields you expect to find, their data types,
> optional code to validate them, and an array to stuff the validated
> fields into.
>
> set wantVarList {{username string} {id integer} {uid check_routine
> validate_uid} {password check_routine validate_password} {hash base64}
> {email email}}
>
> set status [response_broker $wantVarList response]
>
> Since every page using the response broker would need to check for
> response broker parse failures it would probably be nice to be able to
> specify a general handler routine that would run and then abort the
> page, removing the need to check the return.
>
> Now if I run…
>
> response_broker $wantVarList response
>
> …if the page continues then the response array would contain validated
> fields found in the form for the variables named in wantVarList and no
> others. You might want the presence of unexpected fields to also blow
> out, but that would be likely to bite you pretty often when the reasons
> are harmless. Maybe log them and include a {field ignore} option that
> will inhibit logging for expected-but-ignored fields.
>
> We've done a form package at FlightAware that has a specific look and
> feel that allows you to specify both Tcl and Javascript validation code.
> Of course you still need the Tcl code on the server but it's nice in the
> modern era to also provide Javascript validation on the browser. We
> could probably open source if its appearance was genericized or
> something, but I mention it mainly to point out the usefulness of such
> an approach both for the developer and the users and a likely need to
> push Rivet into providing more stuff to support developers making modern
> websites.
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: rivet-dev-unsubscribe@tcl.apache.org
For additional commands, e-mail: rivet-dev-help@tcl.apache.org

Re: a load_response transaction-type thing

[Karl Lehenbauer <ka...@gmail.com>]

>
>
>
>I reiterate Damon's question: is there something that could be shared in
>your tools on which we may start some analysis and development?

Let me talk to Daniel and get back to you.  I'm not concerned about the
code -- I don't think we'll have any problem shaking it loose.  The only
concern I have is with the collateral, which there are several ways that
could be handled.




---------------------------------------------------------------------
To unsubscribe, e-mail: rivet-dev-unsubscribe@tcl.apache.org
For additional commands, e-mail: rivet-dev-help@tcl.apache.org

Re: a load_response transaction-type thing

[Massimo Manghi <mx...@apache.org>]

On 01/08/2013 08:39 AM, Karl Lehenbauer wrote:
> This talk of new capabilities for a new release of Rivet has had me
> revisiting something I've been thinking about for quite a while...
>
> It's really easy for developers to use forms in a way that they
> shouldn't, putting stuff in hidden fields, etc, that could easily be
> monkeyed with by an attacker. That data needs to be kept server side and
> accessed with an opaque, unguessable session ID cookie or equivalent.
> Yes, yes… clear, but not simple, and that friction helps to cause the
> observed problem.

I wasn't in the team when Session package was designed but I understand 
it was shaped with an eye on this issue. Methods 'fetch' and 'store' 
take 2 keys and it's quite natural to have the form name as one of the 
keys for storing/retrieving data. Maybe the interface could be more 
extended to offer some 'form' oriented method.

>
> It's also super common for developers to accept input from forms with
> insufficient checks on the validity of the input data, the source of SQL
> injection attacks and so forth.
>
> I'm thinking of what you might call a response broker. (This isn't an
> original idea -- I've read about stuff like this.)
>
> Rather than doing a load_response, the page handling the response would
> explicitly name the fields you expect to get from the form, you invoke a
> proc specifying the fields you expect to find, their data types,
> optional code to validate them, and an array to stuff the validated
> fields into.

I ended up having something like this as a utility for consistently 
squeezing data from forms into array to be stored in a table. It's 
something much less ambitious

>
> set wantVarList {{username string} {id integer} {uid check_routine
> validate_uid} {password check_routine validate_password} {hash base64}
> {email email}}
>
> set status [response_broker $wantVarList response]
>
> Since every page using the response broker would need to check for
> response broker parse failures it would probably be nice to be able to
> specify a general handler routine that would run and then abort the
> page, removing the need to check the return.
>
> Now if I run…
>
> response_broker $wantVarList response
>
> …if the page continues then the response array would contain validated
> fields found in the form for the variables named in wantVarList and no
> others. You might want the presence of unexpected fields to also blow
> out, but that would be likely to bite you pretty often when the reasons
> are harmless. Maybe log them and include a {field ignore} option that
> will inhibit logging for expected-but-ignored fields.
>

Wouldn't a generalized page abort mechanism be a workable basis for 
something like this?

> We've done a form package at FlightAware that has a specific look and
> feel that allows you to specify both Tcl and Javascript validation code.
> Of course you still need the Tcl code on the server but it's nice in the
> modern era to also provide Javascript validation on the browser. We
> could probably open source if its appearance was genericized or
> something, but I mention it mainly to point out the usefulness of such
> an approach both for the developer and the users and a likely need to
> push Rivet into providing more stuff to support developers making modern
> websites.
>
>

I reiterate Damon's question: is there something that could be shared in 
your tools on which we may start some analysis and development?

  -- Massimo


---------------------------------------------------------------------
To unsubscribe, e-mail: rivet-dev-unsubscribe@tcl.apache.org
For additional commands, e-mail: rivet-dev-help@tcl.apache.org

Re: a load_response transaction-type thing

[Damon Courtney <da...@tclhome.com>]

I have something very similar to this (haven't we all written something like this at some point?).  I would love to see your forms package that includes JS validation though.  That's the one piece I'm missing.  I know there are packages out there that do the validation, but having it included in a Tcl package for me would be really sweet.

Any chance of sharing it?


On Jan 8, 2013, at 1:39 AM, Karl Lehenbauer <ka...@gmail.com> wrote:

> This talk of new capabilities for a new release of Rivet has had me revisiting something I've been thinking about for quite a while...
> 
> It's really easy for developers to use forms in a way that they shouldn't, putting stuff in hidden fields, etc, that could easily be monkeyed with by an attacker.  That data needs to be kept server side and accessed with an opaque, unguessable session ID cookie or equivalent.  Yes, yes… clear, but not simple, and that friction helps to cause the observed problem.
> 
> It's also super common for developers to accept input from forms with insufficient checks on the validity of the input data, the source of SQL injection attacks and so forth.
> 
> I'm thinking of what you might call a response broker.  (This isn't an original idea -- I've read about stuff like this.)
> 
> Rather than doing a load_response, the page handling the response would explicitly name the fields you expect to get from the form, you invoke a proc specifying the fields you expect to find, their data types, optional code to validate them, and an array to stuff the validated fields into.
> 
> set wantVarList {{username string} {id integer} {uid check_routine validate_uid} {password check_routine validate_password} {hash base64} {email email}}
> 
> set status [response_broker $wantVarList response]
> 
> Since every page using the response broker would need to check for response broker parse failures it would probably be nice to be able to specify a general handler routine that would run and then abort the page, removing the need to check the return.
> 
> Now if I run…
> 
> response_broker $wantVarList response
> 
> …if the page continues then the response array would contain validated fields found in the form for the variables named in wantVarList and no others.  You might want the presence of unexpected fields to also blow out, but that would be likely to bite you pretty often when the reasons are harmless.  Maybe log them and include a {field ignore} option that will inhibit logging for expected-but-ignored fields.
> 
> We've done a form package at FlightAware that has a specific look and feel that allows you to specify both Tcl and Javascript validation code.  Of course you still need the Tcl code on the server but it's nice in the modern era to also provide Javascript validation on the browser.  We could probably open source if its appearance was genericized or something, but I mention it mainly to point out the usefulness of such an approach both for the developer and the users and a likely need to push Rivet into providing more stuff to support developers making modern websites.
> 
>