[jira] [Created] (MESOS-8587) Introduce a parallel for each loop (and other parallel algorithms).

["Benjamin Mahler (JIRA)" <ji...@apache.org>]

Benjamin Mahler created MESOS-8587:
--------------------------------------

             Summary: Introduce a parallel for each loop (and other parallel algorithms).
                 Key: MESOS-8587
                 URL: https://issues.apache.org/jira/browse/MESOS-8587
             Project: Mesos
          Issue Type: Improvement
          Components: libprocess
            Reporter: Benjamin Mahler


Consider the following code:

{code}
SomeProcess::func()
{
  foreach (const Item& item, items) {
    // Perform some const work on item.
  }
}
{code}

When {{items}} becomes very large, this code would benefit from some parallelism. With a parallel loop construct, we could improve the performance of this type of code significantly:

{code}
SomeProcess::func()
{
  foreach_parallel (items, [=](const Item& item) {
    // Perform some const work on item.
  });
}
{code}

Ideally, this could enforce const-access to the current Process for safety. An implementation of this would need to do something like:

# Split the iteration of {{items}} into 1 <= N <= num_worker_threads segments.
# Spawn N-1 additional temporary execution Processes (or re-use from a pool)
# Dispatch to these N-1 additional processes for them to perform their segment of the iteration.
# Perform the 1st segment on the current Process.
# Have the current Process block to wait for the others to finish. (note need to avoid deadlocking the worker threads here!)

This generalizes to many other algorithms rather than just iteration. It may be good to align this with the C++ Parallelism TS, which shows how many of the C++ algorithms have potential for parallel counterparts.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)