DistributedLambda

[Peter Firmstone <ji...@zeus.net.au>]

The present serialized form of Lambda's don't access captured fields 
until after deserialization, making them dependant on the enclosing 
classes serial form, its deserialized object state and requiring the 
enclosing class to be Serializable.

This appears to be a fundamental mistake in the design of lambda 
serialization.

I don't believe this will be changed now, this decision is the result of 
an expert group and the release date for Java 8 draws near.

One possibility of fixing this is to create a marker interface, eg 
DistributedLambda.

By implementing a new serialization proxy for DistributedLambda that 
doesn't refer to the enclosing class, but instead captures fields at the 
time of serialization instead of after deserialization, we can provide 
independant short lived functional immutable objects, enabling lambda 
parameter arguments for remote objects that don't require codebase 
downloads at the exporting node.

My reasoning and consequences:

   1. Anonomous classes introduce brittleness in a distributed
      environment, they really should be avoided.
   2. Capturing state at the time of execution from deserialized
      encapsulating object state, invites mutability and side effects
      that diverge from the state of the original caller object and
      intent of functional programming.
   3. Storing a lambda into a @serialField within a Serializable object,
      would also mean that the lamdba parameters would be captured at
      the time of serialization, not at some later point of execution
      after deserialization, so a DistributedLambda wouldn't see updates
      to the encapsulating object after deserialization.
   4. Lamdba's are functional programming constructs and functional
      programming is founded on immutability, so DistributedLambda would
      honour that heritage.
   5. A DistributedLambda instance becomes an independant immutable
      implementation of a functional interface at the time of
      serialization, to avoid surprises, fields referred to by a lambda
      should be effectively final and side effects should be avoided.

Thoughts?

Regards,

Peter.

Re: DistributedLambda

[Peter <ji...@zeus.net.au>]

Actually Lambda serialization is a good effort, I think we can leverage it by removing the dependency on the enclosing class, but only for short lived objects intended as arguments for remote objects.

ASM could be used to capture invokedynamic instructions used by the jvm to create lambdas, this could then be marshalled along with any parameters used to recreate the lambda remotely with ASM's assistance.  A new class object can be created using a hash class name derived from the original instructions for the sole purpose of holding the lambda instructions (purpose is similar to the original encapsulating class) and only containing instructions necessary for lambda bytecode generation.

So if a client has numerous uses of lambda's throughout it's calling class, a separate holding class would be created remotely for each lambda block.

This would only occur when a functional interface extends DistributedLambda.

Regards,

Peter.

----- Original message -----
> To enable remote clients to invoke processing at the server with lambda 
> expression invocation on remote objects, without code downloads.
> 
> Presently the enclosing class is serialized along with the lambda, 
> because it contains the receipe generated by the static compiler. I'm 
> investigating if it's possible to send only the receipe along with any 
> parameter objects, for dynamic code generatation at the server.
> 
> I want to completely avoid code downloading, lambda's could potentially 
> allow significant performance and latency benefits by filtering and 
> processing at the server, prior to returning results to clients.
> 
> Cheers,
> 
> Peter.
> 
> 
> On 9/03/2014 6:02 PM, Michał Kłeczek wrote:
> > Peter,
> > 
> > I'm still trying to grasp what you want to achieve...
> > 
> > Is it simply in-band code downloading?
> > 
> > Regards,
> > 
> 

Re: DistributedLambda

[Peter Firmstone <ji...@zeus.net.au>]

To enable remote clients to invoke processing at the server with lambda 
expression invocation on remote objects, without code downloads.

Presently the enclosing class is serialized along with the lambda, 
because it contains the receipe generated by the static compiler. I'm 
investigating if it's possible to send only the receipe along with any 
parameter objects, for dynamic code generatation at the server.

I want to completely avoid code downloading, lambda's could potentially 
allow significant performance and latency benefits by filtering and 
processing at the server, prior to returning results to clients.

Cheers,

Peter.


On 9/03/2014 6:02 PM, Michał Kłeczek wrote:
> Peter,
>
> I'm still trying to grasp what you want to achieve...
>
> Is it simply in-band code downloading?
>
> Regards,
>

Re: DistributedLambda

[Michał Kłeczek <mi...@xpro.biz>]

Peter,

I'm still trying to grasp what you want to achieve...

Is it simply in-band code downloading?

Regards,

-- 
Michał Kłeczek
XPro Sp. z o. o.
ul. Borowskiego 2
03-475 Warszawa
Polska

Re: DistributedLambda

[Peter Firmstone <ji...@zeus.net.au>]

An update:

Lambda arguments are captured at serialization time, this is a good 
thing!  A static method is generated by the compiler on the enclosing 
class and this is called during deserialization.

I'm investigating how to capture the lambda receipe, so that it and it's 
arguments can be serialized, without requiring the enclosing class, so 
the lambda receipe can be used to create an object independant of it's 
original enclosing class during deserialization.

Will keep you posted.

Any thoughts appreciated.

Regards,

Peter.

On 9/03/2014 11:41 AM, Peter Firmstone wrote:
> The present serialized form of Lambda's don't access captured fields 
> until after deserialization, making them dependant on the enclosing 
> classes serial form, its deserialized object state and requiring the 
> enclosing class to be Serializable.
>
> This appears to be a fundamental mistake in the design of lambda 
> serialization.
>
> I don't believe this will be changed now, this decision is the result 
> of an expert group and the release date for Java 8 draws near.
>
> One possibility of fixing this is to create a marker interface, eg 
> DistributedLambda.
>
> By implementing a new serialization proxy for DistributedLambda that 
> doesn't refer to the enclosing class, but instead captures fields at 
> the time of serialization instead of after deserialization, we can 
> provide independant short lived functional immutable objects, enabling 
> lambda parameter arguments for remote objects that don't require 
> codebase downloads at the exporting node.
>
> My reasoning and consequences:
>
>   1. Anonomous classes introduce brittleness in a distributed
>      environment, they really should be avoided.
>   2. Capturing state at the time of execution from deserialized
>      encapsulating object state, invites mutability and side effects
>      that diverge from the state of the original caller object and
>      intent of functional programming.
>   3. Storing a lambda into a @serialField within a Serializable object,
>      would also mean that the lamdba parameters would be captured at
>      the time of serialization, not at some later point of execution
>      after deserialization, so a DistributedLambda wouldn't see updates
>      to the encapsulating object after deserialization.
>   4. Lamdba's are functional programming constructs and functional
>      programming is founded on immutability, so DistributedLambda would
>      honour that heritage.
>   5. A DistributedLambda instance becomes an independant immutable
>      implementation of a functional interface at the time of
>      serialization, to avoid surprises, fields referred to by a lambda
>      should be effectively final and side effects should be avoided.
>
> Thoughts?
>
> Regards,
>
> Peter.