You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@dlab.apache.org by "Oleh Moskovych (JIRA)" <ji...@apache.org> on 2018/11/26 21:06:00 UTC
[jira] [Commented] (DLAB-89) [AWS]: Improve method to receive
prices of spot instances
[ https://issues.apache.org/jira/browse/DLAB-89?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16699581#comment-16699581 ]
Oleh Moskovych commented on DLAB-89:
------------------------------------
Refactored code for getting price for on demand instances using boto3 client in scope of this task.
Method will receive full price for on demand instance as in DLab this value used for calculating max price for spot instances for EMR clusters.
# Used botocore/botocore/data/endpoints.json file as data source for getting region description, which should be in a format, for example: EU (Ireland) for eu-west-1
# Updated missed parameters in Jenkins job for EMR cluster creating
Created PR: [https://github.com/epam/DLab/pull/702] and assigned [~omartushevskyi] to review/merge.
> [AWS]: Improve method to receive prices of spot instances
> ---------------------------------------------------------
>
> Key: DLAB-89
> URL: https://issues.apache.org/jira/browse/DLAB-89
> Project: Apache DLab
> Issue Type: Bug
> Reporter: Vira Vitanska
> Assignee: Oleh Moskovych
> Priority: Major
> Labels: DevOps
>
> *Preconditions:*
> 1. Environment is created
>
> *Steps to reproduce:*
> 1. Create EMR with spot
>
> *Actual result:*
> EMR creation fails
>
> *Expected result:*
> EMR is created
>
> Currently method _*get_ec2_price*_ uses downloading of big json file (~484 Mb) during creating EMR cluster with sport instances.
> Python required ~ _*3.2Gb + 484*_ Mb on SSN node for 1 launch, which can cause memory error on small shapes (like _*t2.medium*_).
> Suggestion: rewrite method using python client to avoid such resource (memory/cpu) overheads.
> Links:
> [https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeSpotPriceHistory.html]
> [https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ec2.html#EC2.Client.describe_spot_price_history]
> [https://aws.amazon.com/blogs/aws/new-aws-price-list-api/]
> [https://aws.amazon.com/blogs/aws/aws-price-list-api-update-new-query-and-metadata-functions/]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@dlab.apache.org
For additional commands, e-mail: dev-help@dlab.apache.org