You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gobblin.apache.org by "Hung Tran (Jira)" <ji...@apache.org> on 2019/12/12 06:15:00 UTC
[jira] [Resolved] (GOBBLIN-865) Add feature that enables
PK-chunking in partition
[ https://issues.apache.org/jira/browse/GOBBLIN-865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hung Tran resolved GOBBLIN-865.
-------------------------------
Fix Version/s: 0.15.0
Resolution: Fixed
Issue resolved by pull request #2722
[https://github.com/apache/incubator-gobblin/pull/2722]
> Add feature that enables PK-chunking in partition
> --------------------------------------------------
>
> Key: GOBBLIN-865
> URL: https://issues.apache.org/jira/browse/GOBBLIN-865
> Project: Apache Gobblin
> Issue Type: Task
> Reporter: Alex Li
> Priority: Major
> Labels: salesforce
> Fix For: 0.15.0
>
> Time Spent: 12h 20m
> Remaining Estimate: 0h
>
> In SFDC(salesforce) connector, we have partitioning mechanisms to split a giant query to multiple sub queries. There are 3 mechanisms:
> * simple partition (equally split by time)
> * dynamic pre-partition (generate histogram and split by row numbers)
> * user specified partition (set up time range in job file)
> However there are tables like Task and Contract are failing time to time to fetch full data.
> We may want to utilize PK-chunking to partition the query.
>
> The pk-chunking doc from SFDC - [https://developer.salesforce.com/docs/atlas.en-us.api_asynch.meta/api_asynch/async_api_headers_enable_pk_chunking.htm]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)