You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Ismaël Mejía (JIRA)" <ji...@apache.org> on 2019/06/25 10:38:00 UTC
[jira] [Comment Edited] (BEAM-7615) Cloud Bigtable IO connector can
read from multiple tables using SDF (Java)
[ https://issues.apache.org/jira/browse/BEAM-7615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16872254#comment-16872254 ]
Ismaël Mejía edited comment on BEAM-7615 at 6/25/19 10:37 AM:
--------------------------------------------------------------
You can probably benefit of the work done to add SDF for HBaseIO. So reuse ByteKeyRangeTracker and co BEAM-4020. Please remember that at this moment SDF does not support Dynamic Work Rebalancing with any runner so the SDF based transform should not be the default yet.
If you want to address the unbounded case too do not hesitate to take a look to the recent work on HCatalogIO to reuse the Wait transform to discover new partitions See BEAM-7450. But maybe you can try to tackle directly if the BigTable APIs allow to do so.
was (Author: iemejia):
You can probably benefit of the work done to add SDF for HBaseIO. So reuse ByteKeyRangeTracker and co BEAM-4020. Please remember that at this moment SDF does not support Dynamic Work Rebalancing with any runner so the SDF based transform should not be the default yet.
If you want to address the unbounded case too do not hesitate to take a look to the recent work on HCatalogIO to reuse the Wait transform to discover new partitions See BEAM-7450. But maybe you can try to tackle directly if the BigTable IOs allow to do so.
> Cloud Bigtable IO connector can read from multiple tables using SDF (Java)
> --------------------------------------------------------------------------
>
> Key: BEAM-7615
> URL: https://issues.apache.org/jira/browse/BEAM-7615
> Project: Beam
> Issue Type: Improvement
> Components: io-java-gcp
> Reporter: Ahmet Altay
> Assignee: Igor Bernstein
> Priority: Major
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)