You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@openwhisk.apache.org by "Dominic Kim (Confluence)" <no...@apache.org> on 2021/03/08 09:10:06 UTC
[CONF] OpenWhisk > DataManagementService
|
[](https://cwiki.apache.org/confluence/display/~style95?src=mail&src.mail.product=confluence-
server&src.mail.timestamp=1615194606559&src.mail.notification=com.atlassian.confluence.plugins.confluence-
content-notifications-plugin%3Apage-created-
notification&src.mail.recipient=8aa980875bf24635015c9267bc8e02f6) | Dominic
Kim **created** a page
---|---
|
---
| | [](https://cwiki.apache.org/confluence/display/OPENWHISK/DataManagementService?src=mail&src.mail.product=confluence-
server&src.mail.timestamp=1615194606559&src.mail.notification=com.atlassian.confluence.plugins.confluence-
content-notifications-plugin%3Apage-created-
notification&src.mail.recipient=8aa980875bf24635015c9267bc8e02f6&src.mail.action=view
"page icon")
---
[DataManagementService](https://cwiki.apache.org/confluence/display/OPENWHISK/DataManagementService?src=mail&src.mail.product=confluence-
server&src.mail.timestamp=1615194606559&src.mail.notification=com.atlassian.confluence.plugins.confluence-
content-notifications-plugin%3Apage-created-
notification&src.mail.recipient=8aa980875bf24635015c9267bc8e02f6&src.mail.action=view
"DataManagementService")
|
# 1\. Introduction
As the new scheduler depends on ETCD, it is necessary to manage data
insertion/deletion/restoration to/from ETCD.
For example, when a new queue is created, the queue endpoint should be
included in ETCD.
When a new container is created for a given action, the container information
should be stored in ETCD to properly calculate the required number of
containers.
When a scheduler is up and running, the endpoint of the scheduler should also
be stored in ETCD so that a new queue can be created in the queue.
If such data are removed for some reason even though they are actually running
well, data should be restored.
Since many components like MemoryQueue, ContainerProxy, Scheduler, etc,
require the same logic, we introduced a dedicated service to manage this kind
of workload.
So the DataManagementService is in charge of communicating with ETCD.
It stores data to ETCD, if it couldn't store it for some reason, it retries
until succeeds. So it guarantees the data is eventually stored in ETCD.
# 2\. Detail Design
This service receives three types of events,
1. Event to store to ETCD.
2. Watch-related event.
3. Transaction(leader election) event.
4. ETC.
## 2.1 DataStorage event
This type of event is mostly sent from MemoryQueue and ContainerProxy.
This event is to let DataManagementService store data to ETCD.
And there are two events of this type.
1. RegisterData: Store data to overwrite the given key.
2. RegisterInitialData: Store data if and only if there is no data in ETCD.
3. UnregisterData: Remove data with the given key.
For example, when we store data that is used by only one component, we can
make sure no other component is updating the data.
The status of a certain MemoryQueue or ContainerProxy would never be updated
by the others.
In this case we can use RegisterData to update the data.
But for some data such as namespace throttling status which is shared by
multiple queues in a namespace, if there is already data, it should be
respected.
In this case, we can use RegisterInitialData. So if one queue already stores
the namespace throttling data and updated it to "throttled" status, when
another queue is created, it should not overwrite it with other data.
When storing data, a caller can select whether to enable fail-over or not.
When fail-over is enabled, DataManagementService watches the given key and
restores data when it is unexpectedly removed.
When removing the data, it tries to unwatch the given key first, and unwatch
operation works, then remove the data in ETCD.
So no restoration happens.
## 2.2 Watch-related event.
DataManagementService is supposed to restore data if it is removed by
something other than itself.
For this feature, DataManagementService is watching some ETCD keys.
When the data is removed for some reason The WatchEndpointRemoved event will
be sent and it should try to restore the data.
## 2.3 Transaction event.
This is used when a scheduler tries to create a new queue.
In the current version, there is only one queue running for a given action.
So we need to guarantee a queue is created only in one scheduler.
So when a scheduler receives a queue creation request, it first tries to do
the ETCD transaction for a MemoryQueue key.
If the transaction succeeds, other schedulers' transactions would fail and
only one scheduler creates a new queue.
The transaction is generally an expensive operation but queue creation happens
only once for each action if activations keep coming.
## 2.4 ETC
Sometimes, a component such as InvokerHealthManager repeatedly stores data to
ETCD.
InvokerHealthManager keeps reporting the invoker status(free memory,
busyMemory, etc).
If the data is same, we don't need to store the same data to ETCD multiple
times.
SO we have UpdateDataOnChange event to store the data only if there is any
change.
# 3\. EtcdWorker
Actual ETCD-related operations take place in EtcdWorker only.
So that we can isolate the ETCD dependency to EtcdWorker only.
This worker is only used by DataManagementService and it stores/removes data
to/from ETCD according to requests from DataManagementService.
---
| | [](https://cwiki.apache.org/confluence/display/OPENWHISK/DataManagementService?src=mail&src.mail.product=confluence-
server&src.mail.timestamp=1615194606559&src.mail.notification=com.atlassian.confluence.plugins.confluence-
content-notifications-plugin%3Apage-created-
notification&src.mail.recipient=8aa980875bf24635015c9267bc8e02f6&src.mail.action=view
"View page Icon") | [View
page](https://cwiki.apache.org/confluence/display/OPENWHISK/DataManagementService?src=mail&src.mail.product=confluence-
server&src.mail.timestamp=1615194606559&src.mail.notification=com.atlassian.confluence.plugins.confluence-
content-notifications-plugin%3Apage-created-
notification&src.mail.recipient=8aa980875bf24635015c9267bc8e02f6&src.mail.action=view
"View page") | •
---|---|---
[](https://cwiki.apache.org/confluence/plugins/likes/like.action?contentId=177046251&src=mail&src.mail.product=confluence-
server&src.mail.timestamp=1615194606559&src.mail.notification=com.atlassian.confluence.plugins.confluence-
content-notifications-plugin%3Apage-created-
notification&src.mail.recipient=8aa980875bf24635015c9267bc8e02f6&src.mail.action=like&jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ4c3JmOjhhYTk4MDg3NWJmMjQ2MzUwMTVjOTI2N2JjOGUwMmY2IiwicXNoIjoiY2E0ODdmMzI2NjNjMjIyMTFjYjIyY2Q1NmM2Y2UyY2Y1NGExODI5ODMxODFiMzZmZGMxYzc1MzIxYTA0MTI0YSIsImlzcyI6ImNvbmZsdWVuY2Vfbm90aWZpY2F0aW9uc0FSRUgtWFVEMS1QT1FHLUNTQU8iLCJleHAiOjE2MTU3OTk0MDYsImlhdCI6MTYxNTE5NDYwNn0.ocgsnT4vw-
_dZYMJFyeohe_faBrIPa36px6W8U-OWLk "Like Icon") |
[Like](https://cwiki.apache.org/confluence/plugins/likes/like.action?contentId=177046251&src=mail&src.mail.product=confluence-
server&src.mail.timestamp=1615194606559&src.mail.notification=com.atlassian.confluence.plugins.confluence-
content-notifications-plugin%3Apage-created-
notification&src.mail.recipient=8aa980875bf24635015c9267bc8e02f6&src.mail.action=like&jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ4c3JmOjhhYTk4MDg3NWJmMjQ2MzUwMTVjOTI2N2JjOGUwMmY2IiwicXNoIjoiY2E0ODdmMzI2NjNjMjIyMTFjYjIyY2Q1NmM2Y2UyY2Y1NGExODI5ODMxODFiMzZmZGMxYzc1MzIxYTA0MTI0YSIsImlzcyI6ImNvbmZsdWVuY2Vfbm90aWZpY2F0aW9uc0FSRUgtWFVEMS1QT1FHLUNTQU8iLCJleHAiOjE2MTU3OTk0MDYsImlhdCI6MTYxNTE5NDYwNn0.ocgsnT4vw-
_dZYMJFyeohe_faBrIPa36px6W8U-OWLk "Like")
---|---
| | [Stop watching
space](https://cwiki.apache.org/confluence/users/removespacenotification.action?spaceKey=OPENWHISK&src=mail&src.mail.product=confluence-
server&src.mail.timestamp=1615194606559&src.mail.notification=com.atlassian.confluence.plugins.confluence-
content-notifications-plugin%3Apage-created-
notification&src.mail.recipient=8aa980875bf24635015c9267bc8e02f6&src.mail.action=stop-
watching&jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ4c3JmOjhhYTk4MDg3NWJmMjQ2MzUwMTVjOTI2N2JjOGUwMmY2IiwicXNoIjoiOGY2YTMyNDY1NTUzZThiNDUyYTM3NGE5NjAxZjIxOWQ0ZGUyYzcyYWRiNDlhYmE1NWRmZmMyMTQ2NmM5ZmNhNiIsImlzcyI6ImNvbmZsdWVuY2Vfbm90aWZpY2F0aW9uc0FSRUgtWFVEMS1QT1FHLUNTQU8iLCJleHAiOjE2MTU3OTk0MDYsImlhdCI6MTYxNTE5NDYwNn0.NfrjDBavY4Fui-
fKmncg0-VuuHtpshsZH9_4o8qVe5A) | •
---|---
[Manage
notifications](https://cwiki.apache.org/confluence/users/editmyemailsettings.action?src=mail&src.mail.product=confluence-
server&src.mail.timestamp=1615194606559&src.mail.notification=com.atlassian.confluence.plugins.confluence-
content-notifications-plugin%3Apage-created-
notification&src.mail.recipient=8aa980875bf24635015c9267bc8e02f6&src.mail.action=manage)
---
| 
---
This message was sent by Atlassian Confluence 7.5.0
---