You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by igyu <ig...@21cn.com> on 2021/09/01 03:28:46 UTC
when does app to print
this is my data
{"timestamp":"2021-08-01 15:31:56,895","msg":" INFO org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-9-thread-97]: 88: get_table : db=hivetest tbl=chinese_part"}
{"timestamp":"2021-08-02 15:31:56,895","msg":" INFO org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-9-thread-97]: 88: get_table : db=hivetest tbl=chinese_part"}
{"timestamp":"2021-08-03 15:31:56,895","msg":" INFO org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-9-thread-97]: 88: get_table : db=hivetest tbl=chinese_part"}
{"timestamp":"2021-08-04 15:31:56,895","msg":" INFO org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-9-thread-97]: 88: get_table : db=hivetest tbl=chinese_part"}
{"timestamp":"2021-08-05 15:31:56,895","msg":" INFO org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-9-thread-97]: 88: get_table : db=hivetest tbl=chinese_part"}
{"timestamp":"2021-08-06 15:31:56,895","msg":" INFO org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-9-thread-97]: 88: get_table : db=hivetest tbl=chinese_part"}
nc -lk 6666
env.socketTextStream("127.0.0.1", 6666)
.map(line => {.......})
.assignAscendingTimestamps(_.times) //事务时间
.keyBy(k => (k.db, k.tbl))
.window(SlidingEventTimeWindows.of(Time.days(3), Time.days(1)))
.reduce((x, y) => {TableCall(x.db, x.tbl, x.conuts + 1, y.times)})
.print()
when does app to print
when I input data of 2021-08-03 15:31:56,895, app doesn't print
I input
{"timestamp":"2021-08-04 15:31:56,895","msg":" INFO org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-9-thread-97]: 88: get_table : db=hivetest tbl=chinese_part"}
{"timestamp":"2021-08-05 15:31:56,895","msg":" INFO org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-9-thread-97]: 88: get_table : db=hivetest tbl=chinese_part"}
{"timestamp":"2021-08-06 15:31:56,895","msg":" INFO org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-9-thread-97]: 88: get_table : db=hivetest tbl=chinese_part"}
app aslo doesn't print
I don't know when does app to print
swith to jdbc I have the same problem
igyu
Re: when does app to print
Posted by Caizhi Weng <ts...@gmail.com>.
Hi!
With streaming API you'll need to call env.execute() at the end to start
the job.
igyu <ig...@21cn.com> 于2021年9月1日周三 上午11:29写道：
> this is my data
>
> {"timestamp":"2021-08-01 15:31:56,895","msg":" INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-9-thread-97]: 88:
> get_table : db=hivetest tbl=chinese_part"}
>
> {"timestamp":"2021-08-02 15:31:56,895","msg":" INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-9-thread-97]: 88:
> get_table : db=hivetest tbl=chinese_part"}
>
> {"timestamp":"2021-08-03 15:31:56,895","msg":" INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-9-thread-97]: 88:
> get_table : db=hivetest tbl=chinese_part"}
> {"timestamp":"2021-08-04 15:31:56,895","msg":" INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-9-thread-97]: 88:
> get_table : db=hivetest tbl=chinese_part"}
> {"timestamp":"2021-08-05 15:31:56,895","msg":" INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-9-thread-97]: 88:
> get_table : db=hivetest tbl=chinese_part"}
> {"timestamp":"2021-08-06 15:31:56,895","msg":" INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-9-thread-97]: 88:
> get_table : db=hivetest tbl=chinese_part"}
>
>
> nc -lk 6666
>
>
> env.socketTextStream("127.0.0.1", 6666)
>
> .map(line => {.......})
>
> .assignAscendingTimestamps(_.times) //事务时间
>
> .keyBy(k => (k.db, k.tbl))
>
> .window(SlidingEventTimeWindows.of(Time.days(3), Time.days(1)))
>
> .reduce((x, y) => {TableCall(x.db, x.tbl, x.conuts + 1, y.times)})
>
> .print()
>
> when does app to print
>
> when I input data of 2021-08-03 15:31:56,895, app doesn't print
>
> I input
> {"timestamp":"2021-08-04 15:31:56,895","msg":" INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-9-thread-97]: 88:
> get_table : db=hivetest tbl=chinese_part"}
> {"timestamp":"2021-08-05 15:31:56,895","msg":" INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-9-thread-97]: 88:
> get_table : db=hivetest tbl=chinese_part"}
> {"timestamp":"2021-08-06 15:31:56,895","msg":" INFO
> org.apache.hadoop.hive.metastore.HiveMetaStore: [pool-9-thread-97]: 88:
> get_table : db=hivetest tbl=chinese_part"}
>
> app aslo doesn't print
>
> I don't know when does app to print
>
> swith to jdbc I have the same problem
>
> ------------------------------
> igyu
>