You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@dlab.apache.org by "Vira Vitanska (JIRA)" <ji...@apache.org> on 2018/12/19 10:56:00 UTC
[jira] [Closed] (DLAB-89) [AWS][SSN performance optimization]:
Refactor pricing retrieval logic to optimize RAM usage on SSN
[ https://issues.apache.org/jira/browse/DLAB-89?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vira Vitanska closed DLAB-89.
-----------------------------
> [AWS][SSN performance optimization]: Refactor pricing retrieval logic to optimize RAM usage on SSN
> --------------------------------------------------------------------------------------------------
>
> Key: DLAB-89
> URL: https://issues.apache.org/jira/browse/DLAB-89
> Project: Apache DLab
> Issue Type: Bug
> Reporter: Vira Vitanska
> Assignee: Oleh Moskovych
> Priority: Major
> Labels: DevOps
> Fix For: 2.1 release
>
>
> *Preconditions:*
> 1. Environment is created
>
> *Steps to reproduce:*
> 1. Create EMR with spot
>
> *Actual result:*
> EMR creation fails
>
> *Expected result:*
> EMR is created
>
> Currently method _*get_ec2_price*_ uses downloading of big json file (~484 Mb) during creating EMR cluster with sport instances.
> Python required ~ _*3.2Gb + 484*_ Mb on SSN node for 1 launch, which can cause memory error on small shapes (like _*t2.medium*_).
> Suggestion: rewrite method using python client to avoid such resource (memory/cpu) overheads.
> Links:
> [https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeSpotPriceHistory.html]
> [https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ec2.html#EC2.Client.describe_spot_price_history]
> [https://aws.amazon.com/blogs/aws/new-aws-price-list-api/]
> [https://aws.amazon.com/blogs/aws/aws-price-list-api-update-new-query-and-metadata-functions/]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@dlab.apache.org
For additional commands, e-mail: dev-help@dlab.apache.org