You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@reef.apache.org by John Yang <jo...@gmail.com> on 2014/11/11 07:51:10 UTC
A question on the class NumberOfProcesses in reef-runtime-local
Hello,
I have a question on the class NumberOfProcesses in reef-runtime-local.
An intern in our lab got confused about the different names. The class name
is NumberOfProcesses, but the short_name is nThreads.
Also in LocalDriverConfiguration we have NUMBER_OF_PROCESSES and in
LocalRuntimeConfiguration we have NUMBER_OF_THREADS which are basically the
same thing.
He was confused particularly because he did not know about RunnableProcess
which runs and monitors an evaluator process as a thread.
The comment for LocalRuntimeConfiguration.NUMBER_OF_THREADS does explain
clearly what the number means, but I guess the naming inconsistency can be
confusing to newcomers.
What are your thoughts?
Best regards,
John
Re: A question on the class NumberOfProcesses in reef-runtime-local
Posted by John Yang <jo...@gmail.com>.
Hi Markus,
Thank you for the explanation. I filed an issue at
https://issues.apache.org/jira/browse/REEF-37.
Thanks,
John
On Wed, Nov 12, 2014 at 1:03 AM, Markus Weimer <ma...@weimo.de> wrote:
> Hi,
>
> tl;dr: there is really no justification for the current state of confusing
> names. Please file a JIRA such that we get around to fix it eventually.
> Ideally, we'd refer to this number as "NUMBER_OF_CONTAINERS" or such.
>
> Now, how did we get here? The local runtime evolved quite a bit.
> Initially, it was used more for API validation than for actual REEF
> application development. At that time, it didn't actually spawn processes
> for each Evaluator. Instead, each Evaluator was run on its own thread
> (pool). That became impractical when we moved on to building actual
> applications that needed testing, as this execution model allowed Tasks to
> reference Driver state and vice versa. In order to unearth those bugs, we
> moved to the current execution model of one process per Evaluator. And of
> course, not all places were properly updated.
>
> Hope this helps explain the confusion,
>
> Markus
>
>
> On 2014-11-10 22:51, John Yang wrote:
>
>> Hello,
>>
>>
>> I have a question on the class NumberOfProcesses in reef-runtime-local.
>>
>> An intern in our lab got confused about the different names. The class
>> name
>> is NumberOfProcesses, but the short_name is nThreads.
>>
>> Also in LocalDriverConfiguration we have NUMBER_OF_PROCESSES and in
>> LocalRuntimeConfiguration we have NUMBER_OF_THREADS which are basically
>> the
>> same thing.
>>
>> He was confused particularly because he did not know about RunnableProcess
>> which runs and monitors an evaluator process as a thread.
>>
>>
>> The comment for LocalRuntimeConfiguration.NUMBER_OF_THREADS does explain
>> clearly what the number means, but I guess the naming inconsistency can be
>> confusing to newcomers.
>>
>> What are your thoughts?
>>
>>
>> Best regards,
>> John
>>
>>
>
Re: A question on the class NumberOfProcesses in reef-runtime-local
Posted by Markus Weimer <ma...@weimo.de>.
Hi,
tl;dr: there is really no justification for the current state of
confusing names. Please file a JIRA such that we get around to fix it
eventually. Ideally, we'd refer to this number as "NUMBER_OF_CONTAINERS"
or such.
Now, how did we get here? The local runtime evolved quite a bit.
Initially, it was used more for API validation than for actual REEF
application development. At that time, it didn't actually spawn
processes for each Evaluator. Instead, each Evaluator was run on its own
thread (pool). That became impractical when we moved on to building
actual applications that needed testing, as this execution model allowed
Tasks to reference Driver state and vice versa. In order to unearth
those bugs, we moved to the current execution model of one process per
Evaluator. And of course, not all places were properly updated.
Hope this helps explain the confusion,
Markus
On 2014-11-10 22:51, John Yang wrote:
> Hello,
>
>
> I have a question on the class NumberOfProcesses in reef-runtime-local.
>
> An intern in our lab got confused about the different names. The class name
> is NumberOfProcesses, but the short_name is nThreads.
>
> Also in LocalDriverConfiguration we have NUMBER_OF_PROCESSES and in
> LocalRuntimeConfiguration we have NUMBER_OF_THREADS which are basically the
> same thing.
>
> He was confused particularly because he did not know about RunnableProcess
> which runs and monitors an evaluator process as a thread.
>
>
> The comment for LocalRuntimeConfiguration.NUMBER_OF_THREADS does explain
> clearly what the number means, but I guess the naming inconsistency can be
> confusing to newcomers.
>
> What are your thoughts?
>
>
> Best regards,
> John
>