You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2020/02/21 05:28:00 UTC
[jira] [Commented] (ATLAS-3320) Import Service: Support Concurrent
Ingest
[ https://issues.apache.org/jira/browse/ATLAS-3320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17041559#comment-17041559 ]
ASF subversion and git services commented on ATLAS-3320:
--------------------------------------------------------
Commit a2ccfb9f3577e911103041d8d4b91c169697f6a1 in atlas's branch refs/heads/master from Ashutosh Mestry
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=a2ccfb9 ]
ATLAS-3320: Import Service. Support concurrent ingest.
> Import Service: Support Concurrent Ingest
> -----------------------------------------
>
> Key: ATLAS-3320
> URL: https://issues.apache.org/jira/browse/ATLAS-3320
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
> Affects Versions: trunk
> Reporter: Ashutosh Mestry
> Assignee: Ashutosh Mestry
> Priority: Major
> Fix For: trunk
>
> Attachments: ATLAS-3320-ImportService-Concurrent-ingest.patch
>
>
> *Background*
> Existing implementation of import supports single entity creation. This greatly limits the rate of ingest.
> *Solution Approach*
> * Use existing producer-consumer framework to allow for configurable consumers of entities from the input stream.
> * Modify _BulkImporterImpl_ to implement the consumer.
> * Modify _AtlasImportRequest_ to specify the batch size and number of workers.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)