cthrift with state-machine based receive

[ma...@bestweb.net]

Sorry about the _LONG_ delay between releases of cthrift. I was working on
replacing the recursive deserializer with a state-machine based one. It's
now been implemented and tested to the same level as the recursive
version.

Why is this interesting? Well, the main loop looks like:
  while( have_not_finished_reading_message ) {
      process_next_arc_in_state_machine
  }
But it also lets us do the following:
  while( (status = process_next_arc(state)) != DONE ) {
     if( status == SOCKET_WOULD_HAVE_BLOCKED ) {
       save_state_and_do_something_else;
    }
    if( status == ERROR )
      use_state_to_handle_error;
   }
 }

So, we get the ability to handle part of a message, then switch to
handling another message, then come back to the message, etc.

As you might gather, this will be a nice platform for implementing
streaming messages as well.

Another thing, that is specific to the implementation. The state-machine
approach creates a data structure per type. So, its a _LOT_ less expensive
to compose types.

Re: cthrift with state-machine based receive

[ma...@bestweb.net]

Sorry, I was mistaken. I had thought that the way thrift generated the
code, it would generate unique code for each path instead of each type. It
looks like thrift _does_ generate code proportional to the number of
types. I'll have to go back and revisit why someone was complaining about
the size of the generated code.

:(
Mayan

> This sounds pretty cool - definitely an improvement over the existing
> recursive approach when it comes to interruptibility.
>
> I'm not sure I understood your discussion of composing types. Can you
> expand
> on that a little?
>
> -Bryan
>
> On Fri, Apr 2, 2010 at 3:20 AM, <ma...@bestweb.net> wrote:
>
>>
>> Sorry about the _LONG_ delay between releases of cthrift. I was working
>> on
>> replacing the recursive deserializer with a state-machine based one.
>> It's
>> now been implemented and tested to the same level as the recursive
>> version.
>>
>> Why is this interesting? Well, the main loop looks like:
>>  while( have_not_finished_reading_message ) {
>>      process_next_arc_in_state_machine
>>  }
>> But it also lets us do the following:
>>  while( (status = process_next_arc(state)) != DONE ) {
>>     if( status == SOCKET_WOULD_HAVE_BLOCKED ) {
>>       save_state_and_do_something_else;
>>    }
>>    if( status == ERROR )
>>      use_state_to_handle_error;
>>   }
>>  }
>>
>> So, we get the ability to handle part of a message, then switch to
>> handling another message, then come back to the message, etc.
>>
>> As you might gather, this will be a nice platform for implementing
>> streaming messages as well.
>>
>> Another thing, that is specific to the implementation. The state-machine
>> approach creates a data structure per type. So, its a _LOT_ less
>> expensive
>> to compose types.
>>
>>
>>
>>
>

Re: cthrift with state-machine based receive

[Bryan Duxbury <br...@rapleaf.com>]

This sounds pretty cool - definitely an improvement over the existing
recursive approach when it comes to interruptibility.

I'm not sure I understood your discussion of composing types. Can you expand
on that a little?

-Bryan

On Fri, Apr 2, 2010 at 3:20 AM, <ma...@bestweb.net> wrote:

>
> Sorry about the _LONG_ delay between releases of cthrift. I was working on
> replacing the recursive deserializer with a state-machine based one. It's
> now been implemented and tested to the same level as the recursive
> version.
>
> Why is this interesting? Well, the main loop looks like:
>  while( have_not_finished_reading_message ) {
>      process_next_arc_in_state_machine
>  }
> But it also lets us do the following:
>  while( (status = process_next_arc(state)) != DONE ) {
>     if( status == SOCKET_WOULD_HAVE_BLOCKED ) {
>       save_state_and_do_something_else;
>    }
>    if( status == ERROR )
>      use_state_to_handle_error;
>   }
>  }
>
> So, we get the ability to handle part of a message, then switch to
> handling another message, then come back to the message, etc.
>
> As you might gather, this will be a nice platform for implementing
> streaming messages as well.
>
> Another thing, that is specific to the implementation. The state-machine
> approach creates a data structure per type. So, its a _LOT_ less expensive
> to compose types.
>
>
>
>