You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by 大森林 <ap...@foxmail.com> on 2020/10/06 03:44:06 UTC

what's the example for datastream data generator?

I want to simulate the scene"back pressure"could you tell me
what's the example for datastream datagen?
Thanks for your help.

回复： what's the example for datastream data generator?

Posted by 大森林 <ap...@foxmail.com>.
My java is poor


I have tried like this,but it's wrong,Could you tell me how to fix this...
import org.apache.flink.streaming.api.datastream.DataStream;
import org.apache.flink.streaming.api.environment.LocalStreamEnvironment;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.api.functions.source.datagen.DataGeneratorSource;
import org.apache.flink.streaming.api.functions.source.datagen.RandomGenerator;

public class datagen {


    public static void main(String[] args) throws Exception {

        LocalStreamEnvironment env = StreamExecutionEnvironment.createLocalEnvironment();


        DataStream datasets = new DataStream(env, env.addSource(new DataGeneratorSource(RandomGenerator.longGenerator(0, 1000))).getTransformation());
    }

}




I want to use this to simulate "back pressure" scene,do you have some advices or guide?thanks~!


------------------&nbsp;原始邮件&nbsp;------------------
发件人:                                                                                                                        "Shengkai Fang"                                                                                    <fskmine@gmail.com&gt;;
发送时间:&nbsp;2020年10月6日(星期二) 晚上11:06
收件人:&nbsp;"大森林"<appleyuchi@foxmail.com&gt;;
抄送:&nbsp;"user"<user@flink.apache.org&gt;;
主题:&nbsp;Re: what's the example for datastream data generator?



Hi, I think you can take a look at org.apache.flink.table.planner.plan.nodes.common.CommonPhysicalTableSourceScan#createSourceTransformation, which will tell you how to get transformations by source function and stream execution environment. In datastream api, we also have a DataGen that is org.apache.flink.streaming.api.functions.source.datagen.DataGeneratorSource. Therefore, I think we can do as follows to create a DataGen:&nbsp;


new DataStream(env, env.addSource(new DataGeneratorSource(RandomGenerator.longGenerator(0, 1000))).getTransformation())&nbsp;


I am not sure whether this is the best method. Welcome for any options.
&nbsp; 大森林 <appleyuchi@foxmail.com&gt; 于2020年10月6日周二 上午11:44写道：

I want to simulate the scene"back pressure"could you tell me
what's the example for datastream datagen?
Thanks for your help.

Re: what's the example for datastream data generator?

Posted by Shengkai Fang <fs...@gmail.com>.
Hi, I think you can take a look at
*org.apache.flink.table.planner.plan.nodes.common.CommonPhysicalTableSourceScan#createSourceTransformation*,
which will tell you how to get transformations by source function and
stream execution environment. In datastream api, we also have a DataGen
that is
*org.apache.flink.streaming.api.functions.source.datagen.DataGeneratorSource*.
Therefore, I think we can do as follows to create a DataGen:

new DataStream(env, env.addSource(new
DataGeneratorSource(RandomGenerator.longGenerator(0,
1000))).getTransformation())

I am not sure whether this is the best method. Welcome for any options.

大森林 <ap...@foxmail.com> 于2020年10月6日周二 上午11:44写道：

> I want to simulate the scene"back pressure"
> could you tell me
> what's the example for datastream datagen?
> Thanks for your help.
>