[jira] [Created] (FLINK-31686) All of connector should replace the shallow copy with a deep copy

["tanjialiang (Jira)" <ji...@apache.org>]

tanjialiang created FLINK-31686:
-----------------------------------

             Summary: All of connector should replace the shallow copy with a deep copy
                 Key: FLINK-31686
                 URL: https://issues.apache.org/jira/browse/FLINK-31686
             Project: Flink
          Issue Type: Bug
          Components: Connectors / FileSystem
    Affects Versions: 1.16.1
            Reporter: tanjialiang
         Attachments: image-2023-04-01-16-18-48-762.png, image-2023-04-01-16-18-56-075.png

Hi team, when i using the following sql
{code:java}
CREATE TABLE student (
    `id` STRING,
    `name` STRING,
    `age` INT
) WITH (
  'connector' = 'filesystem',
  'path' = '...',
  'format' = 'orc'
);

select
    t1.total,
    t2.total
from
    (
        select
            count(*) as total,
            1 as join_key
        from student
        where name = 'tanjialiang'
    ) t1
    LEFT JOIN (
        select
            count(*) as total,
            1 as join_key
        from student;
    ) t2 
    ON t1.join_key = t2.join_key; {code}
 

it will throw an error

!image-2023-04-01-15-53-40-060.png!

 

I tried to solve it, and i found filesystem connector's copy function using a shallow copy instread of deep copy.   It lead to all of query from a same table source reuse the same bulkWriterFormat, and my query have filter condition, which will push down into the bulkWriterFormat, so the filter condition maybe reuse.   

I found the DynamicTableSource and DynamicTableSink's copy function comment to ask we should impletement it with deep copy, but i found every connector are using shallow copy to impletement it.   So i think not only the filesystem connector have this problem.

!image-2023-04-01-16-12-29-927.png!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)