You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "linweijiang (Jira)" <ji...@apache.org> on 2022/11/04 09:48:00 UTC
[jira] [Updated] (FLINK-29887) Lookup cache in JDBC table connector is not each process (i.e. TaskManager) will hold a cache
[ https://issues.apache.org/jira/browse/FLINK-29887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
linweijiang updated FLINK-29887:
--------------------------------
Description:
hi~ I saw the description of "When lookup cache is enabled, each process (i.e. TaskManager) will hold a cache" on the website. But when I print out the hashCode of cache in each slot’s thread, I find that they are inconsistent. Can you help me explain? thks~
{code:java}
//org.apache.flink.connector.jdbc.table.JdbcRowDataLookupFunction # eval() add a line of code:
LOG.info("cache hashCode is: {}", cache);
//log
2022-11-04 17:22:53,118 INFO org.apache.flink.connector.jdbc.table.JdbcRowDataLookupFunction [Source: daily[1] -> Calc[2] -> LookupJoin[3] -> Calc[4] -> ConstraintEnforcer[5] (8/8)#0] [] - cache hashCode is: org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache@656ae7d9
2022-11-04 17:22:53,118 INFO org.apache.flink.connector.jdbc.table.JdbcRowDataLookupFunction [Source: daily[1] -> Calc[2] -> LookupJoin[3] -> Calc[4] -> ConstraintEnforcer[5] (6/8)#0] [] - cache hashCode is: org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache@5c3a31c2
2022-11-04 17:22:53,118 INFO org.apache.flink.connector.jdbc.table.JdbcRowDataLookupFunction [Source: daily[1] -> Calc[2] -> LookupJoin[3] -> Calc[4] -> ConstraintEnforcer[5] (5/8)#0] [] - cache hashCode is: org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache@598a856e
2022-11-04 17:22:53,118 INFO org.apache.flink.connector.jdbc.table.JdbcRowDataLookupFunction [Source: daily[1] -> Calc[2] -> LookupJoin[3] -> Calc[4] -> ConstraintEnforcer[5] (7/8)#0] [] - cache hashCode is: org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache@765328ef
2022-11-04 17:22:53,118 INFO org.apache.flink.connector.jdbc.table.JdbcRowDataLookupFunction [Source: daily[1] -> Calc[2] -> LookupJoin[3] -> Calc[4] -> ConstraintEnforcer[5] (3/8)#0] [] - cache hashCode is: org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache@47f36967
2022-11-04 17:22:53,118 INFO org.apache.flink.connector.jdbc.table.JdbcRowDataLookupFunction [Source: daily[1] -> Calc[2] -> LookupJoin[3] -> Calc[4] -> ConstraintEnforcer[5] (1/8)#0] [] - cache hashCode is: org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache@2b2ea2f
2022-11-04 17:22:53,118 INFO org.apache.flink.connector.jdbc.table.JdbcRowDataLookupFunction [Source: daily[1] -> Calc[2] -> LookupJoin[3] -> Calc[4] -> ConstraintEnforcer[5] (4/8)#0] [] - cache hashCode is: org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache@1901ad34
2022-11-04 17:22:53,118 INFO org.apache.flink.connector.jdbc.table.JdbcRowDataLookupFunction [Source: daily[1] -> Calc[2] -> LookupJoin[3] -> Calc[4] -> ConstraintEnforcer[5] (2/8)#0] [] - cache hashCode is: org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache@6c441f09
{code}
was:
hi~ I saw the description of "When lookup cache is enabled, each process (i.e. TaskManager) will hold a cache" on the website. But when I print out the hashCode of cache in each slot’s thread, I find that they are inconsistent. Can you help me explain? thks~
{code:java}
//代码占位符
2022-11-04 17:22:53,118 INFO org.apache.flink.connector.jdbc.table.JdbcRowDataLookupFunction [Source: daily[1] -> Calc[2] -> LookupJoin[3] -> Calc[4] -> ConstraintEnforcer[5] (8/8)#0] [] - cache hashCode is: org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache@656ae7d9
2022-11-04 17:22:53,118 INFO org.apache.flink.connector.jdbc.table.JdbcRowDataLookupFunction [Source: daily[1] -> Calc[2] -> LookupJoin[3] -> Calc[4] -> ConstraintEnforcer[5] (6/8)#0] [] - cache hashCode is: org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache@5c3a31c2
2022-11-04 17:22:53,118 INFO org.apache.flink.connector.jdbc.table.JdbcRowDataLookupFunction [Source: daily[1] -> Calc[2] -> LookupJoin[3] -> Calc[4] -> ConstraintEnforcer[5] (5/8)#0] [] - cache hashCode is: org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache@598a856e
2022-11-04 17:22:53,118 INFO org.apache.flink.connector.jdbc.table.JdbcRowDataLookupFunction [Source: daily[1] -> Calc[2] -> LookupJoin[3] -> Calc[4] -> ConstraintEnforcer[5] (7/8)#0] [] - cache hashCode is: org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache@765328ef
2022-11-04 17:22:53,118 INFO org.apache.flink.connector.jdbc.table.JdbcRowDataLookupFunction [Source: daily[1] -> Calc[2] -> LookupJoin[3] -> Calc[4] -> ConstraintEnforcer[5] (3/8)#0] [] - cache hashCode is: org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache@47f36967
2022-11-04 17:22:53,118 INFO org.apache.flink.connector.jdbc.table.JdbcRowDataLookupFunction [Source: daily[1] -> Calc[2] -> LookupJoin[3] -> Calc[4] -> ConstraintEnforcer[5] (1/8)#0] [] - cache hashCode is: org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache@2b2ea2f
2022-11-04 17:22:53,118 INFO org.apache.flink.connector.jdbc.table.JdbcRowDataLookupFunction [Source: daily[1] -> Calc[2] -> LookupJoin[3] -> Calc[4] -> ConstraintEnforcer[5] (4/8)#0] [] - cache hashCode is: org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache@1901ad34
2022-11-04 17:22:53,118 INFO org.apache.flink.connector.jdbc.table.JdbcRowDataLookupFunction [Source: daily[1] -> Calc[2] -> LookupJoin[3] -> Calc[4] -> ConstraintEnforcer[5] (2/8)#0] [] - cache hashCode is: org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache@6c441f09
{code}
> Lookup cache in JDBC table connector is not each process (i.e. TaskManager) will hold a cache
> -----------------------------------------------------------------------------------------------
>
> Key: FLINK-29887
> URL: https://issues.apache.org/jira/browse/FLINK-29887
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / JDBC
> Reporter: linweijiang
> Priority: Major
>
> hi~ I saw the description of "When lookup cache is enabled, each process (i.e. TaskManager) will hold a cache" on the website. But when I print out the hashCode of cache in each slot’s thread, I find that they are inconsistent. Can you help me explain? thks~
>
> {code:java}
> //org.apache.flink.connector.jdbc.table.JdbcRowDataLookupFunction # eval() add a line of code:
> LOG.info("cache hashCode is: {}", cache);
> //log
> 2022-11-04 17:22:53,118 INFO org.apache.flink.connector.jdbc.table.JdbcRowDataLookupFunction [Source: daily[1] -> Calc[2] -> LookupJoin[3] -> Calc[4] -> ConstraintEnforcer[5] (8/8)#0] [] - cache hashCode is: org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache@656ae7d9
> 2022-11-04 17:22:53,118 INFO org.apache.flink.connector.jdbc.table.JdbcRowDataLookupFunction [Source: daily[1] -> Calc[2] -> LookupJoin[3] -> Calc[4] -> ConstraintEnforcer[5] (6/8)#0] [] - cache hashCode is: org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache@5c3a31c2
> 2022-11-04 17:22:53,118 INFO org.apache.flink.connector.jdbc.table.JdbcRowDataLookupFunction [Source: daily[1] -> Calc[2] -> LookupJoin[3] -> Calc[4] -> ConstraintEnforcer[5] (5/8)#0] [] - cache hashCode is: org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache@598a856e
> 2022-11-04 17:22:53,118 INFO org.apache.flink.connector.jdbc.table.JdbcRowDataLookupFunction [Source: daily[1] -> Calc[2] -> LookupJoin[3] -> Calc[4] -> ConstraintEnforcer[5] (7/8)#0] [] - cache hashCode is: org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache@765328ef
> 2022-11-04 17:22:53,118 INFO org.apache.flink.connector.jdbc.table.JdbcRowDataLookupFunction [Source: daily[1] -> Calc[2] -> LookupJoin[3] -> Calc[4] -> ConstraintEnforcer[5] (3/8)#0] [] - cache hashCode is: org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache@47f36967
> 2022-11-04 17:22:53,118 INFO org.apache.flink.connector.jdbc.table.JdbcRowDataLookupFunction [Source: daily[1] -> Calc[2] -> LookupJoin[3] -> Calc[4] -> ConstraintEnforcer[5] (1/8)#0] [] - cache hashCode is: org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache@2b2ea2f
> 2022-11-04 17:22:53,118 INFO org.apache.flink.connector.jdbc.table.JdbcRowDataLookupFunction [Source: daily[1] -> Calc[2] -> LookupJoin[3] -> Calc[4] -> ConstraintEnforcer[5] (4/8)#0] [] - cache hashCode is: org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache@1901ad34
> 2022-11-04 17:22:53,118 INFO org.apache.flink.connector.jdbc.table.JdbcRowDataLookupFunction [Source: daily[1] -> Calc[2] -> LookupJoin[3] -> Calc[4] -> ConstraintEnforcer[5] (2/8)#0] [] - cache hashCode is: org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache@6c441f09
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)