You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by WeiWen Fan <yu...@gmail.com> on 2018/10/23 02:06:56 UTC
Table API / SQL UIDs not the only one
a job have two sql
source is kafka
sink is redis or other sink
Asql
Bsql
now only start Asql stop Bsql sink have key 656.19.173.34
then stop Asql and savepoint hdfs now del key 656.19.173.34( if sink is
kafka Don't delete)
start Bsql from savepoint
you will find sink have key 656.19.173.34 and 6630519 all exist
Bsql fetch Asql savepoint result
i think sql uids not the only one
Who can help me see this problem?
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Re: Table API / SQL UIDs not the only one
Posted by WeiWen Fan <yu...@gmail.com>.
i delete ‘row‘ flag
it seems good looking
--------------------------------------
a job have two sql
source is kafka
sink is redis or other sink
Asql
select
reqIp as factorContenta,
count(*) as eCount,
60 * 60 as expire
from
kafka_source
where
uri is not null
group by
hop(
rowtime,
interval '2' second,
interval '60' minute
),
reqIp
Bsql
select
uid as factorContentb,
count(*) as eCount,
60 * 60 as expire
from
kafka_source
where
uri is not null
group by
hop(
rowtime,
interval '2' second,
interval '60' minute
),
uid
now only start Asql stop Bsql sink have key 656.19.173.34
then stop Asql and savepoint hdfs now del key 656.19.173.34( if sink is
kafka Don't delete)
start Bsql from savepoint
you will find sink have key 656.19.173.34 and 6630519 all exist
Bsql fetch Asql savepoint result
i think sql uids not the only one
Who can help me see this problem?
my test data is
{
"reqIp" : "656.19.173.34",
"rowtime" : 1537950912546,
"uid" : 6630519,
"uri" : "/web"
}
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Re: Table API / SQL UIDs not the only one
Posted by WeiWen Fan <yu...@gmail.com>.
hi
i delete flag
it seems good looking
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Re: Table API / SQL UIDs not the only one
Posted by WeiWen Fan <yu...@gmail.com>.
hi
i delete flag
it seems good looking
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Re: Table API / SQL UIDs not the only one
Posted by Dawid Wysakowicz <dw...@apache.org>.
Hi,
Could you rephrase your question? I think some parts of the question are
missing. It would be also easier to help you if you could state the
final problem a bit more clearly.
Best,
Dawid
On 23/10/2018 04:06, WeiWen Fan wrote:
> a job have two sql
> source is kafka
> sink is redis or other sink
>
> Asql
>
>
> Bsql
>
>
> now only start Asql stop Bsql sink have key 656.19.173.34
>
> then stop Asql and savepoint hdfs now del key 656.19.173.34( if sink is
> kafka Don't delete)
>
> start Bsql from savepoint
> you will find sink have key 656.19.173.34 and 6630519 all exist
>
> Bsql fetch Asql savepoint result
>
> i think sql uids not the only one
>
> Who can help me see this problem?
>
>
>
>
> --
> Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/