Re: [scala-internals] New Lift Actor code

[Philipp Haller <ph...@epfl.ch>]

Hi all,

I have been looking at scala.actors to see how far we are from meeting
Lift's requirements, and what LiftActor provides that scala.actors
don't. I split my reply into two mails for better modularity. In the
next installment you can read about how (something like) LiftActor could
be integrated into scala.actors.

To do this, let me first address some of David's points. Disclaimer: I
don't want to argue that scala.actors is perfect. There are some
problems and we are fixing them as we speak. (Kudos to Erik, Rich and
Mirco!)

>     * Lift creates/destroys an Actor for each Comet request.  This rapid
>       creation/destruction of Actors caused memory back-ups, and the
>       existing Actor code seems to be oriented to long running Actors
>       rather than Actors with Object-length lifespans.

scala.actors (Actors in the following) are designed to support these
short object-length lifespans. Indeed, creating an Actor is very cheap.
If the user chooses to shut down the underlying thread pool manually,
destruction of an Actor amounts to garbage-collecting it. Destruction
only involves (little) more if the library should shut down the thread
pool automatically, or Actors are linked together.

>     * The FJ libraries used for Actor scheduling have problems on
>       multi-core machines and are also a source of memory retention issues.

We should have replaced the old (pre-JDK7) FJ framework earlier. Note
that Actors can override the scheduler used to execute them:

  object MyExecutorScheduler extends SchedulerAdapter {
    val pool = Executors.newCachedThreadPool() // for example
    def execute(block: => Unit) =
      pool.execute(new Runnable {
        def run() { block }
      })
  }

  trait MyActor extends Actor {
    override def scheduler = MyExecutorScheduler
  }

In 2.8, we intend to use a scheduler based on j.u.c.ThreadPoolExecutor
as a default.

>     * Replacing the FJ libraries with a scheduler based on
>       java.util.concurrent exposes race/deadlock conditions related to
>       the fact that some parts of the Actor processing (e.g., testing
>       mailbox items against partial functions while the Actor itself is
>       synchronized)

If I understand correctly this is the subject of ticket #2009:

  https://lampsvn.epfl.ch/trac/scala/ticket/2009

Here, the reasoning is indeed as Erik suggests, namely that in the send
method
  - the check for a matching message is done on the sender's tread while
    the receiver's lock is held,
  - the check is only done if the receiver is guaranteed to wait for a
    message (therefore, it does not touch any local state), and
  - multiple senders are serialized using the lock of the receiver.

Indeed, the guarantee that we want to provide is that Actors only
execute on one thread at a time. So, if there is a problem, I suppose it
must be somewhere else.

>     * The Actors require external threads to function and it's not
>       possible to create external threads in the Google App Engine
>       (making Actor-based functionality including CometActors
>       non-functioning in GAE apps)

As I mentioned before, Actors can be made to use schedulers that do not
create external threads. So, in principle this should make it possible
to run Actors on GAE.

>     * Actors are fragile when exceptions are thrown

Here, I believe we can work out a good solution. You have already given
some valuable input, David, with your LiftActor class. So, either we
extend the programming model in that direction, or we refine
trapExit/link etc. and make them more robust.

>     * Actors have running and not running states (as compared with
>       objects which can always respond to message sends).  In practice,
>       managing the running and not running states is as hard as managing
>       memory in C.

Actually, Actors only have to be started to execute the code in act()
before the first `react`. Upon hitting the first `react`, Actor and
LiftActor behave essentially the same with respect to running states.
And, again, you don't have to terminate your Actors if you are OK with
shutting down the thread pool yourself.

>     * There are hidden actors associated with each thread which display
>       the above fragility and state management issues

Basically, what we have is that once a non-actor thread calls an
actor-based operation, an ActorProxy instance is created (which
basically holds the mailbox) that is stored in a ThreadLocal (see the
Actor object). It is used to enable a normal thread to receive messages.
 Also, it is created when a non-actor thread sends a message to another
actor, so that the receiver always has access to the sender of a message.
It should always be possible to write an application so that none of
these ActorProxy instances are created. However, even if they are
created I currently don't see how they could result in memory retention
problems.

I hope this clarifies some (important) points that David raises. The
bottom line is that our goals are not so far apart. I believe with some
tuning effort scala.actors could meet the requirements of Lift.

Cheers,
Philipp