You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Soheil Pourbafrani <so...@gmail.com> on 2018/06/24 09:36:15 UTC
Is it reasonable to use Flink in a local machine
Hi,
My purpose is to generate data in a process and process those data in
another process, something like stream processing, but all done in just one
node, not cluster!
The rate of generating data is 12000 sample per second. I want the
processing phase to done in parallel, So as Flink use node CPU cores to
distribute the task on them and do the process, I'm wondering is it
reasonable to use Flink in such a scenario for processing phase?
The node specification is 32GB RAM and 12 CPU cores.
Re: Is it reasonable to use Flink in a local machine
Posted by zhangminglei <18...@163.com>.
You are welcome, please let me know if you still have question.
> 在 2018年6月25日，上午1:43，Soheil Pourbafrani <so...@gmail.com> 写道：
>
> Thanks for verification!
>
> On Sun, Jun 24, 2018 at 2:25 PM, zhangminglei <18717838093@163.com <ma...@163.com>> wrote:
> Hi, Soheil
>
> You can set up several taskmanager processes in one node. I think it is reasonable to use Flink like this if you do not have enough machines.
>
> Generate data we can think it is source operator and you can set the parallelism for this operator.
> Process to those data we can think, like a flatmap function apply on a datastream. You can also set the parallelism for this operator.
>
> Cheers
> Minglei
>
> > 在 2018年6月24日，下午5:36，Soheil Pourbafrani <soheil.ir08@gmail.com <ma...@gmail.com>> 写道：
> >
> > Hi,
> >
> > My purpose is to generate data in a process and process those data in another process, something like stream processing, but all done in just one node, not cluster!
> >
> > The rate of generating data is 12000 sample per second. I want the processing phase to done in parallel, So as Flink use node CPU cores to distribute the task on them and do the process, I'm wondering is it reasonable to use Flink in such a scenario for processing phase?
> >
> > The node specification is 32GB RAM and 12 CPU cores.
>
>
>
Re: Is it reasonable to use Flink in a local machine
Posted by zhangminglei <18...@163.com>.
Hi, Soheil
You can set up several taskmanager processes in one node. I think it is reasonable to use Flink like this if you do not have enough machines.
Generate data we can think it is source operator and you can set the parallelism for this operator.
Process to those data we can think, like a flatmap function apply on a datastream. You can also set the parallelism for this operator.
Cheers
Minglei
> 在 2018年6月24日，下午5:36，Soheil Pourbafrani <so...@gmail.com> 写道：
>
> Hi,
>
> My purpose is to generate data in a process and process those data in another process, something like stream processing, but all done in just one node, not cluster!
>
> The rate of generating data is 12000 sample per second. I want the processing phase to done in parallel, So as Flink use node CPU cores to distribute the task on them and do the process, I'm wondering is it reasonable to use Flink in such a scenario for processing phase?
>
> The node specification is 32GB RAM and 12 CPU cores.