You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by "Alex Aranovsky (Jira)" <ji...@apache.org> on 2025/02/18 09:23:00 UTC

[jira] [Created] (FLINK-37343) Support for Dynamic Table Selection in DynamoDB Sink Connector

Alex Aranovsky created FLINK-37343:
--------------------------------------

             Summary: Support for Dynamic Table Selection in DynamoDB Sink Connector
                 Key: FLINK-37343
                 URL: https://issues.apache.org/jira/browse/FLINK-37343
             Project: Flink
          Issue Type: New Feature
          Components: Connectors / DynamoDB
            Reporter: Alex Aranovsky


Hey, we have a use-case where we need to sink record into multiple ddb tables without a job reset (topology change), we use the DynamicKafka source and we've essentially forked the regular ddb connector and added support for providing a table as part of the ElementsConverter interface. Since DDB batchWriteItem supports multiple items in a single request, I don't really see a downside into not including it. There is some serdes cost associated with providing the table name as a string on each element we provide to the sink, I'm not sure how significant this is.


This is both a question and a feature request: I can merge the fork upstream, is this something that makes sense to include in the default connector?

Is the serdes cost negligible?

Can I add the table name field as part of the DynamoDbWriteRequest class? it will be a breaking change.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)