You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Fan weiwen (JIRA)" <ji...@apache.org> on 2018/11/01 02:06:00 UTC
[jira] [Updated] (FLINK-10673) Table API / SQL UIDs not the only
one
[ https://issues.apache.org/jira/browse/FLINK-10673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Fan weiwen updated FLINK-10673:
-------------------------------
Description:
a job have two sql
source is kafka
sink is redis or other sink
Asql
{code:java}
//代码占位符
select
reqIp as factorContenta,
count(*) as eCount,
60 * 60 as expire
from
kafka_source
where
uri is not null
group by
hop(
rowtime,
interval '2' second,
interval '60' minute
),
reqIp
{code}
Bsql
{code:java}
//代码占位符
select
uid as factorContentb,
count(*) as eCount,
60 * 60 as expire
from
kafka_source
where
uri is not null
group by
hop(
rowtime,
interval '2' second,
interval '60' minute
),
uid
{code}
now only start Asql stop Bsql sink have key 656.19.173.34
then stop Asql and savepoint hdfs now del key 656.19.173.34( if sink is
kafka Don't delete)
start Bsql from savepoint
you will find sink have key 656.19.173.34 and 6630519 all exist
Bsql fetch Asql savepoint result
i think sql uids not the only one
my test data is
{code:java}
//代码占位符
{
"reqIp" : "656.19.173.34",
"rowtime" : 1537950912546,
"uid" : 6630519,
"uri" : "/web"
}
{code}
was:
a job have two sql
source is kafka
sink is redis or other sink
Asql
{code:java}
//代码占位符
select
reqIp as factorContenta,
count(*) as eCount,
60 * 60 as expire
from
kafka_source
where
uri is not null
group by
hop(
rowtime,
interval '2' second,
interval '60' minute
),
reqIp
{code}
Bsql
{code:java}
//代码占位符
select
uid as factorContentb,
count(*) as eCount,
60 * 60 as expire
from
kafka_source
where
uri is not null
group by
hop(
rowtime,
interval '2' second,
interval '60' minute
),
uid
{code}
now only start Asql stop Bsql sink have key 656.19.173.34
then stop Asql and savepoint hdfs now del key 656.19.173.34( if sink is
kafka Don't delete)
start Bsql from savepoint
you will find sink have key 656.19.173.34 and 6630519 all exist
Bsql fetch Asql savepoint result
i think sql uids not the only one
Who can help me see this problem?
my test data is
{code:java}
//代码占位符
{
"reqIp" : "656.19.173.34",
"rowtime" : 1537950912546,
"uid" : 6630519,
"uri" : "/web"
}
{code}
> Table API / SQL UIDs not the only one
> -------------------------------------
>
> Key: FLINK-10673
> URL: https://issues.apache.org/jira/browse/FLINK-10673
> Project: Flink
> Issue Type: Bug
> Components: Table API & SQL
> Affects Versions: 1.5.4, 1.6.1
> Environment: flink 1.5.0
> Reporter: Fan weiwen
> Assignee: winifredtang
> Priority: Major
>
> a job have two sql
> source is kafka
> sink is redis or other sink
> Asql
> {code:java}
> //代码占位符
> select
> reqIp as factorContenta,
> count(*) as eCount,
> 60 * 60 as expire
> from
> kafka_source
> where
> uri is not null
> group by
> hop(
> rowtime,
> interval '2' second,
> interval '60' minute
> ),
> reqIp
> {code}
> Bsql
> {code:java}
> //代码占位符
> select
> uid as factorContentb,
> count(*) as eCount,
> 60 * 60 as expire
> from
> kafka_source
> where
> uri is not null
> group by
> hop(
> rowtime,
> interval '2' second,
> interval '60' minute
> ),
> uid
> {code}
> now only start Asql stop Bsql sink have key 656.19.173.34
> then stop Asql and savepoint hdfs now del key 656.19.173.34( if sink is
> kafka Don't delete)
> start Bsql from savepoint
> you will find sink have key 656.19.173.34 and 6630519 all exist
> Bsql fetch Asql savepoint result
> i think sql uids not the only one
> my test data is
> {code:java}
> //代码占位符
> {
> "reqIp" : "656.19.173.34",
> "rowtime" : 1537950912546,
> "uid" : 6630519,
> "uri" : "/web"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)