You are viewing a plain text version of this content. The canonical link for it is here.
Posted to custos@airavata.apache.org by Isuru Ranawaka <ir...@gmail.com> on 2019/12/01 01:13:35 UTC
Re: Microservices Based Custos Implementation
Hi Amila,
Thank you very much for your comments and really helpful for further
enhancements. please check my inline comments. Let me know your thoughts.
On Sat, Nov 30, 2019 at 4:33 PM Thejaka Amila J Kanewala <
thejaka.amila@gmail.com> wrote:
> Hi Isuru,
>
> First, sorry for my lack of understanding about some of the bus-words used
> in this design/discussion (e.g., gRPC, Saga, Axon, Camunda etc.)
> Few questions and comments.
>
> 1. As per the design, we have two services (integration and atomic) and
> atomic services have their own database. My assumption is you will have all
> tenant information in the tenant db and all user information in the iam db.
> When you say two dbs, did you mean two MySQL instances or two databases in
> a single MySQL instance? Why we need two databases (even if it is running
> in a single MySQL instance)?
>
The atomic services represents a set of functions which can be grouped
together to do some independent work . Tenant related functions are
grouped to
tenant service and IAM related functions are grouped to IAM service. Data
related to tenant service are located in tenant db and all the db
operations of tenant db is only done through tenant service.
If tenant services requires an information related to a tenant which
resides in iam db, should call to IAM service and fetch. Tenant service is
not allowed to call to iam db directly. The main reasons for this kind of
decoupling is for independent development, deployment and release of each
versions of services. For instance, if tenant schema is changed and want
to do some change in tenant db, data migration or rollback it should only
affect tenant db. it should not affect iam db. I think we can use single
MySQL instance. Moreover, we can use different database technologies for
different services, for e.g if tenant service only has direct insertions
and retrievals of data with less interdependencies among data we can use
mongoDB over mySQL. If iam service deal with data having interdependencies
and transactions we can use MySQL over mongoDB. This may help agile
development as well like adding new services and removing services easily.
> 2. Can you conceptually explain how transaction management works (Saga,
> Axon, Camunda) in this solution? Preferably using some examples (e.g.,
> tenant registration, user registration, use deletion, etc.)
>
Let's consider tenant registration message flow, firstly, tenant
registration integration service will talk to tenant registration core
service to create tenant id and save tenant profile information in tenant
db and subsequently tenant registration integration service will call to
iam service with tenant id to create keycloak realm and save credentials.
Let's say call to iam service is failed. Then we have two options to do
roll back tenant info from tenant db or updated tenant info with tenant
creation failed status. As in monolithic architecture we cannot perform two
phase commits here. We can only achieved eventual consistency. Saga is
architectural pattern which keeps track of all passed service calls and
failed service calls. So we can rollback passed service calls using saga
logs. Axon is a framework which supports saga pattern.
> 3. At first, I thought "API gateway" refers to "Science Gateway" in your
> document, apparently that is not the case (as per description API gateway
> is for API management capabilities....). In that case, what is "Consumer"
> in your diagram? Is that the actual "science gateway" (e.g., SeaGrid) or
> "science gateway middleware" (e.g., Airavata)?
>
> Actual consumers would be integration services. API gateway is not doing
any data related operation. It is just for handle SSL termination,
service discovery, throttling, ..etc.
4. Is this design only for provisioning? In the sense, are users also
> authenticated/authorized through this architecture?
>
Users are authenticated and authorized. Actually, issuing access tokens or
client credentials are done through custos services.
> 5. Regarding the asynchronous model: You probably need to consider a way to
> avoid duplicate requests and should have an API call to check the status of
> an asynchronous request.
>
Yes we need to handle it. May be we may need some background tasks to retry
for failure and scenarios and update status for each task.
> I have a few more questions but let me wait for answers to the above
> questions.
>
> Thank you.
> Best Regards,
> Thejaka Amila Kanewala, PhD
> https://github.com/thejkane/agm
>
> On Wed, Nov 27, 2019 at 4:36 PM Isuru Ranawaka <ir...@gmail.com> wrote:
>
> > Hi Dimuthu,
> >
> > For instance, in tenant registration when client requests to register a
> > tenant, the api should return with 200 OK saying tenant requested and
> > subsequently calling to keycloak, vault and iam services should be run in
> > background and update the status.
> >
> > thanks
> >
> >
> > On Wed, Nov 27, 2019 at 4:07 PM DImuthu Upeksha <
> > dimuthu.upeksha2@gmail.com>
> > wrote:
> >
> > > Hi Isuru,
> > >
> > > What are the use cases for asynchronous communication?
> > >
> > > Thanks
> > > Dimuthu
> > >
> > > On Wed, Nov 27, 2019 at 3:39 PM Isuru Ranawaka <ir...@gmail.com>
> > wrote:
> > >
> > > > Hi all,
> > > >
> > > > Just to give an update on the project,
> > > >
> > > > As per the discussion had with Suresh, Custos has synchronous
> > > communication
> > > > as well as asynchronous communication. For synchronous, there is one
> to
> > > one
> > > > mapping between integration service endpoint to core service
> endpoint.
> > > For
> > > > asynchronous communication there are many services communicate
> > > > asynchronously and get work done.
> > > >
> > > > I have modeled and implemented asynchronous communication as a
> service
> > > > chain and each core service is invoked via service task and
> integration
> > > > service builds a service task pipeline which is modeled as service
> > chain.
> > > > Responses are communicated back via callbacks.
> > > >
> > > > Moreover, gRPC zipkin interceptors for tracing and Prometheus related
> > > > configurations for metrics are added. Docker images for each service
> is
> > > > built and published using docker maven plugin and helm charts are
> > > packaged
> > > > using helm maven plugin.
> > > >
> > > > thanks
> > > >
> > > > Isuru
> > > >
> > > >
> > > > On Wed, Nov 20, 2019 at 3:12 PM Isuru Ranawaka <ir...@gmail.com>
> > > wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > I have implemented a basic prototype [1] according to the
> > architecture
> > > > > mentioned above.
> > > > >
> > > > > custos
> > > > > |
> > > > > |_ custos-core-services
> > > > > | |
> > > > > | | _ tenant-profile-core-service
> > > > > |
> > > > > |_ java
> > > > > |_proto
> > > > > |_ resources
> > > > > |
> > > > > |_custos-core-services-client-stubs
> > > > > |
> > > > > |_ tenant-profile-core-service-client-stub
> > > > > |
> > > > > |_custos-integration-services
> > > > > |
> > > > > | _ tenant-registration-service
> > > > >
> > > > >
> > > > > Basically core services are gRPC services and their stubs and
> clients
> > > are
> > > > > generated in client-stubs module. Integration services use those
> > stubs
> > > to
> > > > > talk to core services. This is currently working locally and as
> next
> > > > step,
> > > > > need to develop docker images, k8s artifacts and run on K8s cluster
> > to
> > > > > check whether integration services resolves core services
> addresses
> > > > > seamlessly.
> > > > >
> > > > > [1]https://github.com/apache/airavata-custos/pull/14
> > > > >
> > > > > thanks
> > > > > Isuru
> > > > >
> > > > >
> > > > >
> > > > > On Mon, Nov 18, 2019 at 9:49 PM Isuru Ranawaka <irjanith@gmail.com
> >
> > > > wrote:
> > > > >
> > > > >> Hi Suresh,
> > > > >>
> > > > >> Thanks for your feedback. I have updated the doc [1]. For
> > > observability
> > > > >> purposes we can use spring-cloud-sleuth and servers such as Zipkin
> > and
> > > > >> those are capable of distributed tracing. For vault operations we
> > have
> > > > >> separate audit feature.
> > > > >>
> > > > >> [1]
> > > > >>
> > > >
> > >
> >
> https://docs.google.com/document/d/1QtUKTfAnIrErvtYnKo3FizCRL2GUiWikEzvBnHPXfWw/edit?usp=sharing
> > > > >>
> > > > >> thanks
> > > > >>
> > > > >> On Mon, Nov 18, 2019 at 6:47 PM Suresh Marru <sm...@apache.org>
> > > wrote:
> > > > >>
> > > > >>> Hi Isuru,
> > > > >>>
> > > > >>> Looks like your embedded architecture diagram did not make it
> with
> > > the
> > > > >>> email. I will request a wiki space for Custos so you can upload
> > there
> > > > for
> > > > >>> future discussions. For now, can you add it to the google doc?
> > > > >>>
> > > > >>> Your overall architecture proposal seems to be good. Can you also
> > > > >>> consider adding logging and detailed instrumentation as part of
> the
> > > > core
> > > > >>> architecture? This will help to have a detailed audit of each
> and
> > > > every
> > > > >>> action on who did what and when.
> > > > >>>
> > > > >>> Cheers,
> > > > >>> Suresh
> > > > >>>
> > > > >>> > On Nov 18, 2019, at 3:41 PM, Isuru Ranawaka <
> irjanith@gmail.com>
> > > > >>> wrote:
> > > > >>> >
> > > > >>> > Hi all,
> > > > >>> >
> > > > >>> > I am initiating this thread to provide Custos initial version
> > > > >>> > implementation details. Since we are moving into cloud native
> > > > >>> development
> > > > >>> > of Custos. I have come up with the following structure for
> micro
> > > > >>> services
> > > > >>> > based development of Custos.
> > > > >>> >
> > > > >>> > Mainly codebase composed of the atomic services and
> integration
> > > > >>> services.
> > > > >>> >
> > > > >>> > Atomic services are independent services with minimal network
> > > > >>> communication
> > > > >>> > and they have their own database with local data.
> > > > >>> >
> > > > >>> > Integration services represents the actual business use case
> and
> > it
> > > > >>> acts as
> > > > >>> > an orchestrator between core services and external services.
> This
> > > is
> > > > >>> > exposed as a rest api via API Gateway. Integration services
> talk
> > to
> > > > >>> atomic
> > > > >>> > services via gRPC protocol. Integration services includes gRPC
> > stub
> > > > of
> > > > >>> > core services. Each integration service and core service is
> > > deployed
> > > > to
> > > > >>> > Kubernetes cluster as services and have their own service
> > > addresses.
> > > > >>> >
> > > > >>> > Main problem with this architecture is transaction management
> > and
> > > it
> > > > >>> is
> > > > >>> > planned to handle with Saga using Axon Server or Camunda like
> > > > >>> framework.
> > > > >>> > All the services are written using Spring Boot and Spring
> Cloud.
> > > > >>> >
> > > > >>> > All the Integration services are registered with the service
> > > registry
> > > > >>> such
> > > > >>> > as Consul. API Gateway is used for API Management capabilities
> > such
> > > > as
> > > > >>> SSL
> > > > >>> > Termination, throttling..etc. There are Open Source API
> Gateways
> > > such
> > > > >>> as
> > > > >>> > Ambassador, Envoy which are compatible with Kubernetes and
> > service
> > > > >>> > registries. API Gateway is responsible for service discovery.
> > > > >>> >
> > > > >>> > Vault and Keycloak are deployed in Kubernetes with HA[1].
> > > > >>> >
> > > > >>> >
> > > > >>> > Let me know your thoughts.
> > > > >>> >
> > > > >>> >
> > > > >>> >
> > > > >>> > [1]
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
> https://docs.google.com/document/d/1zGXlstt9w1ge8oRWjiUv3LdQETcSYtwDngm9grB5sbk/edit?usp=sharing
> > > > >>> >
> > > > >>> >
> > > > >>> > Thanks
> > > > >>> >
> > > > >>> > Isuru
> > > > >>>
> > > > >>>
> > > > >>
> > > > >> --
> > > > >> Senior Engineer
> > > > >> WSO2 Lanka (pvt) Ltd
> > > > >> Blog:- http://isurur.blogspot.com/
> > > > >> Phone:- +94714629880
> > > > >>
> > > > >
> > > > >
> > > > > --
> > > > > Senior Engineer
> > > > > WSO2 Lanka (pvt) Ltd
> > > > > Blog:- http://isurur.blogspot.com/
> > > > > Phone:- +94714629880
> > > > >
> > > >
> > > >
> > > > --
> > > > Senior Engineer
> > > > WSO2 Lanka (pvt) Ltd
> > > > Blog:- http://isurur.blogspot.com/
> > > > Phone:- +94714629880
> > > >
> > >
> >
> >
> > --
> > Senior Engineer
> > WSO2 Lanka (pvt) Ltd
> > Blog:- http://isurur.blogspot.com/
> > Phone:- +94714629880
> >
>
>
> --
> <https://github.com/thejkane/agm>
>
--
Senior Engineer
WSO2 Lanka (pvt) Ltd
Blog:- http://isurur.blogspot.com/
Phone:- +94714629880
Re: Microservices Based Custos Implementation
Posted by Isuru Ranawaka <ir...@gmail.com>.
Hi all,
@Marcus,
Thanks for your input. It answered some issues I had.
Tenant Registration, User Registration, and Authentication Flow is as
follows.
Tenant Registration
POST https://custos.scigap.org:32036/tenant-management/v1.0.0/tenant
{
"tenantName": "sampletenant",
"requesterEmail": "irjanith@gmail.com",
"requesterUsername": "isjarana",
"adminFirstName": "Janith",
"adminLastName": "Ranawaka",
"adminEmail": "isjarana@iu.edu",
"contacts": [
"+1812345386"
],
"redirectURIs": [
"http://tenantsample.com/callback"
],
"tenantURI": "http://tenantsample.com",
"scope": "email",
"domain": "tenantsample.com",
"adminPassword": "1234",
"adminUsername": "admin@custos.iu.edu"
}
Response {
{
"tenantId": "10000001",
"clientId": "custos/HUGxHVZbFgY8fPiM5QeJ/10000001",
"clientSecret": "403Rhpy4Z0LKoFa6Nt3uk1EsQ93h6kJy51JCZ9Ra",
"msg":
"Use Base64 encoded clientId:clientSecret as auth token for
authorization, Credentials are activated after admin approval"
}
}
Tenant Approval
POST https://custos.scigap.org:32036/tenant-management/v1.0.0/status
{
"tenantId":10000001,
"status":"APPROVED",
"updatedBy":"Custosadmin"
}
{
"tenantId": "10000001",
"status": "APPROVED"
}
Add User to Tenant
POST https://custos.scigap.org:32036/user-management/v1.0.0/user
Header :- authorization base64Encode(custosClientId:CustosClientSec)
{
"username": "janith123",
"firstName": "Janith",
"lastName": "Ranawaka",
"password": "janith123",
"email": "fernado@iu.edu"
}
{
"isRegistered": true
}
Enable User
POST https://custos.scigap.org:32036//user-management/v1.0.0/user/activation
Header :- authorization base64Encode(custosClientId:CustosClientSec)
{
"username": "janith123"
}
{
"internalUserId": "janith123@10000002",
"username": "janith123",
"tenantId": "10000002",
"firstName": "Janith",
"lastName": "Ranawaka",
"email": "fernado@iu.edu",
"creationTime": 1.5790367e+12,
"state": "ACTIVE"
}
Authenticate
POST https://custos.scigap.org:32036/identity-management/v1.0.0/authenticate
Header :- authorization base64Encode(custosClientId:CustosClientSec)
{
"username":"janith123",
"password":"janith123"
}
{
"accessToken":
"eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJlOHY0cVpYWVVMbWNsZDQ0OGplUW9EMzE0NUppSVREYm45TTJ1bHVnSlFRIn0.eyJqdGkiOiI4OWRhNjc4Ny1iZjBmLTQ4OGYtYTJhNy01YWVhZjBmOWE3NmQiLCJleHAiOjE1NzkwMzg3NTAsIm5iZiI6MCwiaWF0IjoxNTc5MDM2OTUwLCJpc3MiOiJodHRwczovL2tleWNsb2FrLmN1c3Rvcy5zY2lnYXAub3JnOjMxMDAwL2F1dGgvcmVhbG1zLzEwMDAwMDAyIiwiYXVkIjoiYWNjb3VudCIsInN1YiI6ImQxZmVkMTQxLTAzMjktNDFiMy1hZWFkLTJmNGZlYmI0ZjQ3YyIsInR5cCI6IkJlYXJlciIsImF6cCI6InBnYSIsImF1dGhfdGltZSI6MCwic2Vzc2lvbl9zdGF0ZSI6IjAyYTkxMjUxLTQxMDEtNGU2Yi1iN2UxLTU2ZmIxMjFlNTllZiIsImFjciI6IjEiLCJhbGxvd2VkLW9yaWdpbnMiOlsiaHR0cDovL3RlbmFudHNhbXBsZS5jb20iXSwicmVhbG1fYWNjZXNzIjp7InJvbGVzIjpbIm9mZmxpbmVfYWNjZXNzIiwidW1hX2F1dGhvcml6YXRpb24iXX0sInJlc291cmNlX2FjY2VzcyI6eyJhY2NvdW50Ijp7InJvbGVzIjpbIm1hbmFnZS1hY2NvdW50IiwibWFuYWdlLWFjY291bnQtbGlua3MiLCJ2aWV3LXByb2ZpbGUiXX19LCJzY29wZSI6ImVtYWlsIHByb2ZpbGUiLCJlbWFpbF92ZXJpZmllZCI6dHJ1ZSwibmFtZSI6Ikphbml0aCBSYW5hd2FrYSIsInByZWZlcnJlZF91c2VybmFtZSI6Imphbml0aDEyMyIsImdpdmVuX25hbWUiOiJKYW5pdGgiLCJmYW1pbHlfbmFtZSI6IlJhbmF3YWthIiwiZW1haWwiOiJmZXJuYWRvQGl1LmVkdSJ9.c6NwgtzMeG33_K-JQo9ijOBPkOc-el-DB4aSSuKi_V7on8FttTvfzoYrnnwoxGXFocUVeJMdSsOVsriXu_YtPzUgYueVf0bYpPgoNQWltzGPC3jmaq1OLKlydDYbojwCoN8IhqlXjTPvKB_0L3lycaTLKFFssaZXBa0C2_cs2hnZGcE8li8Pu8FyWGB6oJMsny4sv1z_KuAYmRwou-dGPeVzTKFFiywqXRGxRqvSv_Nqc_ol54q1baS8HJWHvz98DGrhShr4EV0XUQJgo3iULQdRWvo_hmVxYJhWMBAZwmuwfJY1E2Sr_fc_IxOeHwSaPkTfdZ2fx4u-3HEHJvTNkg"
,
"claims": [
{
"key": "username",
"value": "janith123"
},
{
"key": "tenantId",
"value": "10000002"
}
]
}
On Mon, Jan 6, 2020 at 4:51 PM Christie, Marcus Aaron <ma...@iu.edu>
wrote:
> Hi Isuru,
>
> Sorry for the late reply. Answers inline.
>
> > On Dec 16, 2019, at 5:17 PM, Isuru Ranawaka <ir...@gmail.com> wrote:
> >
> > Hi Aarushi, Suresh
> >
> > I am trying to bring back the functionalities of Tenant Profile Service,
> > IAM Service , User Profile Service and Group\ Sharing Service into
> Custos.
> > Once I am going through the existing code in airavata, I came across few
> > unclear parts. I have created some ER diagrams and sequence diagrams to
> > further realize the design. See DB Layer Design Part of [1].
> >
> >
> > 1. In tenant creation flow, tenant creation function of the
> > IAMAdminService creates an admin user at keycloak and enable the user,
> but
> > that user is not saved in UserProfileRespository, where all keycloak
> > enabled users are saved. Is this a bug ?
>
> In Airavata the user profile is typically created when the user first logs
> in. For Custos though, it is perhaps a bug to not create the user profile
> for the admin user.
>
> >
> > 2. How do we get tenant admin password in above case ?
> >
>
> Not quite sure what you are asking for. The tenant admin password
> shouldn't be stored in plain text, but maybe you are asking for something
> else. Why do you want to get the tenant admin password?
>
> > 3. Does tenant requester need to be registered with custos before
> > requesting a tenant. (Otherwise how to access custos API without access
> > token)
> >
>
> In Airavata, a tenant requester effectively has a user account with the
> super-tenant. I think something similar could be done for Custos.
>
> > 3. The users in UserProfileRepository (Keycloak enabled users) are not
> sync
> > with UserRepository and they are only synced when creating a group for a
> > particular domain. Is this ok ?
> >
>
> In Airavata, user profile is synced via event-based replication. When you
> say they are only synced when creating a group, I'm not familiar with that,
> so I'm not sure.
>
> > 4. What is the relationship between domainId and tenantId ? According to
> > the current code tenantId is treated as domainId. See
> > (GroupManagerServiceHandler -> createGroup)
> >
>
> Yeah they are the same thing. Historically these identifiers are called
> gatewayId or domainId or tenantId in Airavata. In Custos they should
> probably be standardized to just "tenantId".
>
> >
> >
> > thanks
> > Isuru
> >
> > [1]
> >
> https://docs.google.com/document/d/1QtUKTfAnIrErvtYnKo3FizCRL2GUiWikEzvBnHPXfWw/edit?usp=sharing
> >
> >
> >
> >
> >
> > On Sat, Nov 30, 2019 at 8:13 PM Isuru Ranawaka <ir...@gmail.com>
> wrote:
> >
> >>
> >> Hi Amila,
> >>
> >> Thank you very much for your comments and really helpful for further
> >> enhancements. please check my inline comments. Let me know your
> thoughts.
> >>
> >> On Sat, Nov 30, 2019 at 4:33 PM Thejaka Amila J Kanewala <
> >> thejaka.amila@gmail.com> wrote:
> >>
> >>> Hi Isuru,
> >>>
> >>> First, sorry for my lack of understanding about some of the bus-words
> used
> >>> in this design/discussion (e.g., gRPC, Saga, Axon, Camunda etc.)
> >>> Few questions and comments.
> >>>
> >>> 1. As per the design, we have two services (integration and atomic) and
> >>> atomic services have their own database. My assumption is you will have
> >>> all
> >>> tenant information in the tenant db and all user information in the iam
> >>> db.
> >>> When you say two dbs, did you mean two MySQL instances or two
> databases in
> >>> a single MySQL instance? Why we need two databases (even if it is
> running
> >>> in a single MySQL instance)?
> >>>
> >>
> >> The atomic services represents a set of functions which can be grouped
> >> together to do some independent work . Tenant related functions are
> >> grouped to
> >> tenant service and IAM related functions are grouped to IAM service.
> Data
> >> related to tenant service are located in tenant db and all the db
> >> operations of tenant db is only done through tenant service.
> >> If tenant services requires an information related to a tenant which
> >> resides in iam db, should call to IAM service and fetch. Tenant service
> is
> >> not allowed to call to iam db directly. The main reasons for this kind
> of
> >> decoupling is for independent development, deployment and release of
> each
> >> versions of services. For instance, if tenant schema is changed and
> want
> >> to do some change in tenant db, data migration or rollback it should
> only
> >> affect tenant db. it should not affect iam db. I think we can use single
> >> MySQL instance. Moreover, we can use different database technologies for
> >> different services, for e.g if tenant service only has direct
> insertions
> >> and retrievals of data with less interdependencies among data we can use
> >> mongoDB over mySQL. If iam service deal with data having
> interdependencies
> >> and transactions we can use MySQL over mongoDB. This may help agile
> >> development as well like adding new services and removing services
> easily.
> >>
> >>
> >>
> >>
> >>> 2. Can you conceptually explain how transaction management works (Saga,
> >>> Axon, Camunda) in this solution? Preferably using some examples (e.g.,
> >>> tenant registration, user registration, use deletion, etc.)
> >>>
> >>
> >> Let's consider tenant registration message flow, firstly, tenant
> >> registration integration service will talk to tenant registration core
> >> service to create tenant id and save tenant profile information in
> tenant
> >> db and subsequently tenant registration integration service will call
> to
> >> iam service with tenant id to create keycloak realm and save
> credentials.
> >> Let's say call to iam service is failed. Then we have two options to do
> >> roll back tenant info from tenant db or updated tenant info with tenant
> >> creation failed status. As in monolithic architecture we cannot perform
> two
> >> phase commits here. We can only achieved eventual consistency. Saga is
> >> architectural pattern which keeps track of all passed service calls and
> >> failed service calls. So we can rollback passed service calls using saga
> >> logs. Axon is a framework which supports saga pattern.
> >>
> >>
> >>> 3. At first, I thought "API gateway" refers to "Science Gateway" in
> your
> >>> document, apparently that is not the case (as per description API
> gateway
> >>> is for API management capabilities....). In that case, what is
> "Consumer"
> >>> in your diagram? Is that the actual "science gateway" (e.g., SeaGrid)
> or
> >>> "science gateway middleware" (e.g., Airavata)?
> >>>
> >>> Actual consumers would be integration services. API gateway is not
> doing
> >> any data related operation. It is just for handle SSL termination,
> >> service discovery, throttling, ..etc.
> >>
> >> 4. Is this design only for provisioning? In the sense, are users also
> >>> authenticated/authorized through this architecture?
> >>>
> >>
> >> Users are authenticated and authorized. Actually, issuing access tokens
> or
> >> client credentials are done through custos services.
> >>
> >>
> >>> 5. Regarding the asynchronous model: You probably need to consider a
> way
> >>> to
> >>> avoid duplicate requests and should have an API call to check the
> status
> >>> of
> >>> an asynchronous request.
> >>>
> >>
> >> Yes we need to handle it. May be we may need some background tasks to
> >> retry for failure and scenarios and update status for each task.
> >>
> >>
> >>> I have a few more questions but let me wait for answers to the above
> >>> questions.
> >>>
> >>> Thank you.
> >>> Best Regards,
> >>> Thejaka Amila Kanewala, PhD
> >>> https://github.com/thejkane/agm
> >>>
> >>> On Wed, Nov 27, 2019 at 4:36 PM Isuru Ranawaka <ir...@gmail.com>
> >>> wrote:
> >>>
> >>>> Hi Dimuthu,
> >>>>
> >>>> For instance, in tenant registration when client requests to register
> a
> >>>> tenant, the api should return with 200 OK saying tenant requested and
> >>>> subsequently calling to keycloak, vault and iam services should be run
> >>> in
> >>>> background and update the status.
> >>>>
> >>>> thanks
> >>>>
> >>>>
> >>>> On Wed, Nov 27, 2019 at 4:07 PM DImuthu Upeksha <
> >>>> dimuthu.upeksha2@gmail.com>
> >>>> wrote:
> >>>>
> >>>>> Hi Isuru,
> >>>>>
> >>>>> What are the use cases for asynchronous communication?
> >>>>>
> >>>>> Thanks
> >>>>> Dimuthu
> >>>>>
> >>>>> On Wed, Nov 27, 2019 at 3:39 PM Isuru Ranawaka <ir...@gmail.com>
> >>>> wrote:
> >>>>>
> >>>>>> Hi all,
> >>>>>>
> >>>>>> Just to give an update on the project,
> >>>>>>
> >>>>>> As per the discussion had with Suresh, Custos has synchronous
> >>>>> communication
> >>>>>> as well as asynchronous communication. For synchronous, there is
> >>> one to
> >>>>> one
> >>>>>> mapping between integration service endpoint to core service
> >>> endpoint.
> >>>>> For
> >>>>>> asynchronous communication there are many services communicate
> >>>>>> asynchronously and get work done.
> >>>>>>
> >>>>>> I have modeled and implemented asynchronous communication as a
> >>> service
> >>>>>> chain and each core service is invoked via service task and
> >>> integration
> >>>>>> service builds a service task pipeline which is modeled as service
> >>>> chain.
> >>>>>> Responses are communicated back via callbacks.
> >>>>>>
> >>>>>> Moreover, gRPC zipkin interceptors for tracing and Prometheus
> >>> related
> >>>>>> configurations for metrics are added. Docker images for each
> >>> service is
> >>>>>> built and published using docker maven plugin and helm charts are
> >>>>> packaged
> >>>>>> using helm maven plugin.
> >>>>>>
> >>>>>> thanks
> >>>>>>
> >>>>>> Isuru
> >>>>>>
> >>>>>>
> >>>>>> On Wed, Nov 20, 2019 at 3:12 PM Isuru Ranawaka <ir...@gmail.com>
> >>>>> wrote:
> >>>>>>
> >>>>>>> Hi all,
> >>>>>>>
> >>>>>>> I have implemented a basic prototype [1] according to the
> >>>> architecture
> >>>>>>> mentioned above.
> >>>>>>>
> >>>>>>> custos
> >>>>>>> |
> >>>>>>> |_ custos-core-services
> >>>>>>> | |
> >>>>>>> | | _ tenant-profile-core-service
> >>>>>>> |
> >>>>>>> |_ java
> >>>>>>> |_proto
> >>>>>>> |_ resources
> >>>>>>> |
> >>>>>>> |_custos-core-services-client-stubs
> >>>>>>> |
> >>>>>>> |_ tenant-profile-core-service-client-stub
> >>>>>>> |
> >>>>>>> |_custos-integration-services
> >>>>>>> |
> >>>>>>> | _ tenant-registration-service
> >>>>>>>
> >>>>>>>
> >>>>>>> Basically core services are gRPC services and their stubs and
> >>> clients
> >>>>> are
> >>>>>>> generated in client-stubs module. Integration services use those
> >>>> stubs
> >>>>> to
> >>>>>>> talk to core services. This is currently working locally and as
> >>> next
> >>>>>> step,
> >>>>>>> need to develop docker images, k8s artifacts and run on K8s
> >>> cluster
> >>>> to
> >>>>>>> check whether integration services resolves core services
> >>> addresses
> >>>>>>> seamlessly.
> >>>>>>>
> >>>>>>> [1]https://github.com/apache/airavata-custos/pull/14
> >>>>>>>
> >>>>>>> thanks
> >>>>>>> Isuru
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Mon, Nov 18, 2019 at 9:49 PM Isuru Ranawaka <
> >>> irjanith@gmail.com>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>>> Hi Suresh,
> >>>>>>>>
> >>>>>>>> Thanks for your feedback. I have updated the doc [1]. For
> >>>>> observability
> >>>>>>>> purposes we can use spring-cloud-sleuth and servers such as
> >>> Zipkin
> >>>> and
> >>>>>>>> those are capable of distributed tracing. For vault operations we
> >>>> have
> >>>>>>>> separate audit feature.
> >>>>>>>>
> >>>>>>>> [1]
> >>>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> https://docs.google.com/document/d/1QtUKTfAnIrErvtYnKo3FizCRL2GUiWikEzvBnHPXfWw/edit?usp=sharing
> >>>>>>>>
> >>>>>>>> thanks
> >>>>>>>>
> >>>>>>>> On Mon, Nov 18, 2019 at 6:47 PM Suresh Marru <sm...@apache.org>
> >>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Hi Isuru,
> >>>>>>>>>
> >>>>>>>>> Looks like your embedded architecture diagram did not make it
> >>> with
> >>>>> the
> >>>>>>>>> email. I will request a wiki space for Custos so you can upload
> >>>> there
> >>>>>> for
> >>>>>>>>> future discussions. For now, can you add it to the google doc?
> >>>>>>>>>
> >>>>>>>>> Your overall architecture proposal seems to be good. Can you
> >>> also
> >>>>>>>>> consider adding logging and detailed instrumentation as part of
> >>> the
> >>>>>> core
> >>>>>>>>> architecture? This will help to have a detailed audit of each
> >>> and
> >>>>>> every
> >>>>>>>>> action on who did what and when.
> >>>>>>>>>
> >>>>>>>>> Cheers,
> >>>>>>>>> Suresh
> >>>>>>>>>
> >>>>>>>>>> On Nov 18, 2019, at 3:41 PM, Isuru Ranawaka <
> >>> irjanith@gmail.com>
> >>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Hi all,
> >>>>>>>>>>
> >>>>>>>>>> I am initiating this thread to provide Custos initial version
> >>>>>>>>>> implementation details. Since we are moving into cloud native
> >>>>>>>>> development
> >>>>>>>>>> of Custos. I have come up with the following structure for
> >>> micro
> >>>>>>>>> services
> >>>>>>>>>> based development of Custos.
> >>>>>>>>>>
> >>>>>>>>>> Mainly codebase composed of the atomic services and
> >>> integration
> >>>>>>>>> services.
> >>>>>>>>>>
> >>>>>>>>>> Atomic services are independent services with minimal network
> >>>>>>>>> communication
> >>>>>>>>>> and they have their own database with local data.
> >>>>>>>>>>
> >>>>>>>>>> Integration services represents the actual business use case
> >>> and
> >>>> it
> >>>>>>>>> acts as
> >>>>>>>>>> an orchestrator between core services and external services.
> >>> This
> >>>>> is
> >>>>>>>>>> exposed as a rest api via API Gateway. Integration services
> >>> talk
> >>>> to
> >>>>>>>>> atomic
> >>>>>>>>>> services via gRPC protocol. Integration services includes gRPC
> >>>> stub
> >>>>>> of
> >>>>>>>>>> core services. Each integration service and core service is
> >>>>> deployed
> >>>>>> to
> >>>>>>>>>> Kubernetes cluster as services and have their own service
> >>>>> addresses.
> >>>>>>>>>>
> >>>>>>>>>> Main problem with this architecture is transaction management
> >>>> and
> >>>>> it
> >>>>>>>>> is
> >>>>>>>>>> planned to handle with Saga using Axon Server or Camunda like
> >>>>>>>>> framework.
> >>>>>>>>>> All the services are written using Spring Boot and Spring
> >>> Cloud.
> >>>>>>>>>>
> >>>>>>>>>> All the Integration services are registered with the service
> >>>>> registry
> >>>>>>>>> such
> >>>>>>>>>> as Consul. API Gateway is used for API Management capabilities
> >>>> such
> >>>>>> as
> >>>>>>>>> SSL
> >>>>>>>>>> Termination, throttling..etc. There are Open Source API
> >>> Gateways
> >>>>> such
> >>>>>>>>> as
> >>>>>>>>>> Ambassador, Envoy which are compatible with Kubernetes and
> >>>> service
> >>>>>>>>>> registries. API Gateway is responsible for service discovery.
> >>>>>>>>>>
> >>>>>>>>>> Vault and Keycloak are deployed in Kubernetes with HA[1].
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Let me know your thoughts.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> [1]
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> https://docs.google.com/document/d/1zGXlstt9w1ge8oRWjiUv3LdQETcSYtwDngm9grB5sbk/edit?usp=sharing
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Thanks
> >>>>>>>>>>
> >>>>>>>>>> Isuru
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> Senior Engineer
> >>>>>>>> WSO2 Lanka (pvt) Ltd
> >>>>>>>> Blog:- http://isurur.blogspot.com/
> >>>>>>>> Phone:- +94714629880
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> Senior Engineer
> >>>>>>> WSO2 Lanka (pvt) Ltd
> >>>>>>> Blog:- http://isurur.blogspot.com/
> >>>>>>> Phone:- +94714629880
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Senior Engineer
> >>>>>> WSO2 Lanka (pvt) Ltd
> >>>>>> Blog:- http://isurur.blogspot.com/
> >>>>>> Phone:- +94714629880
> >>>>>>
> >>>>>
> >>>>
> >>>>
> >>>> --
> >>>> Senior Engineer
> >>>> WSO2 Lanka (pvt) Ltd
> >>>> Blog:- http://isurur.blogspot.com/
> >>>> Phone:- +94714629880
> >>>>
> >>>
> >>>
> >>> --
> >>> <https://github.com/thejkane/agm>
> >>>
> >>
> >>
> >> --
> >> Senior Engineer
> >> WSO2 Lanka (pvt) Ltd
> >> Blog:- http://isurur.blogspot.com/
> >> Phone:- +94714629880
> >>
> >
> >
> > --
> > Senior Engineer
> > WSO2 Lanka (pvt) Ltd
> > Blog:- http://isurur.blogspot.com/
> > Phone:- +94714629880
>
>
--
Senior Engineer
WSO2 Lanka (pvt) Ltd
Blog:- http://isurur.blogspot.com/
Phone:- +94714629880
Re: Microservices Based Custos Implementation
Posted by "Christie, Marcus Aaron" <ma...@iu.edu>.
Hi Isuru,
Sorry for the late reply. Answers inline.
> On Dec 16, 2019, at 5:17 PM, Isuru Ranawaka <ir...@gmail.com> wrote:
>
> Hi Aarushi, Suresh
>
> I am trying to bring back the functionalities of Tenant Profile Service,
> IAM Service , User Profile Service and Group\ Sharing Service into Custos.
> Once I am going through the existing code in airavata, I came across few
> unclear parts. I have created some ER diagrams and sequence diagrams to
> further realize the design. See DB Layer Design Part of [1].
>
>
> 1. In tenant creation flow, tenant creation function of the
> IAMAdminService creates an admin user at keycloak and enable the user, but
> that user is not saved in UserProfileRespository, where all keycloak
> enabled users are saved. Is this a bug ?
In Airavata the user profile is typically created when the user first logs in. For Custos though, it is perhaps a bug to not create the user profile for the admin user.
>
> 2. How do we get tenant admin password in above case ?
>
Not quite sure what you are asking for. The tenant admin password shouldn't be stored in plain text, but maybe you are asking for something else. Why do you want to get the tenant admin password?
> 3. Does tenant requester need to be registered with custos before
> requesting a tenant. (Otherwise how to access custos API without access
> token)
>
In Airavata, a tenant requester effectively has a user account with the super-tenant. I think something similar could be done for Custos.
> 3. The users in UserProfileRepository (Keycloak enabled users) are not sync
> with UserRepository and they are only synced when creating a group for a
> particular domain. Is this ok ?
>
In Airavata, user profile is synced via event-based replication. When you say they are only synced when creating a group, I'm not familiar with that, so I'm not sure.
> 4. What is the relationship between domainId and tenantId ? According to
> the current code tenantId is treated as domainId. See
> (GroupManagerServiceHandler -> createGroup)
>
Yeah they are the same thing. Historically these identifiers are called gatewayId or domainId or tenantId in Airavata. In Custos they should probably be standardized to just "tenantId".
>
>
> thanks
> Isuru
>
> [1]
> https://docs.google.com/document/d/1QtUKTfAnIrErvtYnKo3FizCRL2GUiWikEzvBnHPXfWw/edit?usp=sharing
>
>
>
>
>
> On Sat, Nov 30, 2019 at 8:13 PM Isuru Ranawaka <ir...@gmail.com> wrote:
>
>>
>> Hi Amila,
>>
>> Thank you very much for your comments and really helpful for further
>> enhancements. please check my inline comments. Let me know your thoughts.
>>
>> On Sat, Nov 30, 2019 at 4:33 PM Thejaka Amila J Kanewala <
>> thejaka.amila@gmail.com> wrote:
>>
>>> Hi Isuru,
>>>
>>> First, sorry for my lack of understanding about some of the bus-words used
>>> in this design/discussion (e.g., gRPC, Saga, Axon, Camunda etc.)
>>> Few questions and comments.
>>>
>>> 1. As per the design, we have two services (integration and atomic) and
>>> atomic services have their own database. My assumption is you will have
>>> all
>>> tenant information in the tenant db and all user information in the iam
>>> db.
>>> When you say two dbs, did you mean two MySQL instances or two databases in
>>> a single MySQL instance? Why we need two databases (even if it is running
>>> in a single MySQL instance)?
>>>
>>
>> The atomic services represents a set of functions which can be grouped
>> together to do some independent work . Tenant related functions are
>> grouped to
>> tenant service and IAM related functions are grouped to IAM service. Data
>> related to tenant service are located in tenant db and all the db
>> operations of tenant db is only done through tenant service.
>> If tenant services requires an information related to a tenant which
>> resides in iam db, should call to IAM service and fetch. Tenant service is
>> not allowed to call to iam db directly. The main reasons for this kind of
>> decoupling is for independent development, deployment and release of each
>> versions of services. For instance, if tenant schema is changed and want
>> to do some change in tenant db, data migration or rollback it should only
>> affect tenant db. it should not affect iam db. I think we can use single
>> MySQL instance. Moreover, we can use different database technologies for
>> different services, for e.g if tenant service only has direct insertions
>> and retrievals of data with less interdependencies among data we can use
>> mongoDB over mySQL. If iam service deal with data having interdependencies
>> and transactions we can use MySQL over mongoDB. This may help agile
>> development as well like adding new services and removing services easily.
>>
>>
>>
>>
>>> 2. Can you conceptually explain how transaction management works (Saga,
>>> Axon, Camunda) in this solution? Preferably using some examples (e.g.,
>>> tenant registration, user registration, use deletion, etc.)
>>>
>>
>> Let's consider tenant registration message flow, firstly, tenant
>> registration integration service will talk to tenant registration core
>> service to create tenant id and save tenant profile information in tenant
>> db and subsequently tenant registration integration service will call to
>> iam service with tenant id to create keycloak realm and save credentials.
>> Let's say call to iam service is failed. Then we have two options to do
>> roll back tenant info from tenant db or updated tenant info with tenant
>> creation failed status. As in monolithic architecture we cannot perform two
>> phase commits here. We can only achieved eventual consistency. Saga is
>> architectural pattern which keeps track of all passed service calls and
>> failed service calls. So we can rollback passed service calls using saga
>> logs. Axon is a framework which supports saga pattern.
>>
>>
>>> 3. At first, I thought "API gateway" refers to "Science Gateway" in your
>>> document, apparently that is not the case (as per description API gateway
>>> is for API management capabilities....). In that case, what is "Consumer"
>>> in your diagram? Is that the actual "science gateway" (e.g., SeaGrid) or
>>> "science gateway middleware" (e.g., Airavata)?
>>>
>>> Actual consumers would be integration services. API gateway is not doing
>> any data related operation. It is just for handle SSL termination,
>> service discovery, throttling, ..etc.
>>
>> 4. Is this design only for provisioning? In the sense, are users also
>>> authenticated/authorized through this architecture?
>>>
>>
>> Users are authenticated and authorized. Actually, issuing access tokens or
>> client credentials are done through custos services.
>>
>>
>>> 5. Regarding the asynchronous model: You probably need to consider a way
>>> to
>>> avoid duplicate requests and should have an API call to check the status
>>> of
>>> an asynchronous request.
>>>
>>
>> Yes we need to handle it. May be we may need some background tasks to
>> retry for failure and scenarios and update status for each task.
>>
>>
>>> I have a few more questions but let me wait for answers to the above
>>> questions.
>>>
>>> Thank you.
>>> Best Regards,
>>> Thejaka Amila Kanewala, PhD
>>> https://github.com/thejkane/agm
>>>
>>> On Wed, Nov 27, 2019 at 4:36 PM Isuru Ranawaka <ir...@gmail.com>
>>> wrote:
>>>
>>>> Hi Dimuthu,
>>>>
>>>> For instance, in tenant registration when client requests to register a
>>>> tenant, the api should return with 200 OK saying tenant requested and
>>>> subsequently calling to keycloak, vault and iam services should be run
>>> in
>>>> background and update the status.
>>>>
>>>> thanks
>>>>
>>>>
>>>> On Wed, Nov 27, 2019 at 4:07 PM DImuthu Upeksha <
>>>> dimuthu.upeksha2@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Isuru,
>>>>>
>>>>> What are the use cases for asynchronous communication?
>>>>>
>>>>> Thanks
>>>>> Dimuthu
>>>>>
>>>>> On Wed, Nov 27, 2019 at 3:39 PM Isuru Ranawaka <ir...@gmail.com>
>>>> wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> Just to give an update on the project,
>>>>>>
>>>>>> As per the discussion had with Suresh, Custos has synchronous
>>>>> communication
>>>>>> as well as asynchronous communication. For synchronous, there is
>>> one to
>>>>> one
>>>>>> mapping between integration service endpoint to core service
>>> endpoint.
>>>>> For
>>>>>> asynchronous communication there are many services communicate
>>>>>> asynchronously and get work done.
>>>>>>
>>>>>> I have modeled and implemented asynchronous communication as a
>>> service
>>>>>> chain and each core service is invoked via service task and
>>> integration
>>>>>> service builds a service task pipeline which is modeled as service
>>>> chain.
>>>>>> Responses are communicated back via callbacks.
>>>>>>
>>>>>> Moreover, gRPC zipkin interceptors for tracing and Prometheus
>>> related
>>>>>> configurations for metrics are added. Docker images for each
>>> service is
>>>>>> built and published using docker maven plugin and helm charts are
>>>>> packaged
>>>>>> using helm maven plugin.
>>>>>>
>>>>>> thanks
>>>>>>
>>>>>> Isuru
>>>>>>
>>>>>>
>>>>>> On Wed, Nov 20, 2019 at 3:12 PM Isuru Ranawaka <ir...@gmail.com>
>>>>> wrote:
>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> I have implemented a basic prototype [1] according to the
>>>> architecture
>>>>>>> mentioned above.
>>>>>>>
>>>>>>> custos
>>>>>>> |
>>>>>>> |_ custos-core-services
>>>>>>> | |
>>>>>>> | | _ tenant-profile-core-service
>>>>>>> |
>>>>>>> |_ java
>>>>>>> |_proto
>>>>>>> |_ resources
>>>>>>> |
>>>>>>> |_custos-core-services-client-stubs
>>>>>>> |
>>>>>>> |_ tenant-profile-core-service-client-stub
>>>>>>> |
>>>>>>> |_custos-integration-services
>>>>>>> |
>>>>>>> | _ tenant-registration-service
>>>>>>>
>>>>>>>
>>>>>>> Basically core services are gRPC services and their stubs and
>>> clients
>>>>> are
>>>>>>> generated in client-stubs module. Integration services use those
>>>> stubs
>>>>> to
>>>>>>> talk to core services. This is currently working locally and as
>>> next
>>>>>> step,
>>>>>>> need to develop docker images, k8s artifacts and run on K8s
>>> cluster
>>>> to
>>>>>>> check whether integration services resolves core services
>>> addresses
>>>>>>> seamlessly.
>>>>>>>
>>>>>>> [1]https://github.com/apache/airavata-custos/pull/14
>>>>>>>
>>>>>>> thanks
>>>>>>> Isuru
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Nov 18, 2019 at 9:49 PM Isuru Ranawaka <
>>> irjanith@gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Suresh,
>>>>>>>>
>>>>>>>> Thanks for your feedback. I have updated the doc [1]. For
>>>>> observability
>>>>>>>> purposes we can use spring-cloud-sleuth and servers such as
>>> Zipkin
>>>> and
>>>>>>>> those are capable of distributed tracing. For vault operations we
>>>> have
>>>>>>>> separate audit feature.
>>>>>>>>
>>>>>>>> [1]
>>>>>>>>
>>>>>>
>>>>>
>>>>
>>> https://docs.google.com/document/d/1QtUKTfAnIrErvtYnKo3FizCRL2GUiWikEzvBnHPXfWw/edit?usp=sharing
>>>>>>>>
>>>>>>>> thanks
>>>>>>>>
>>>>>>>> On Mon, Nov 18, 2019 at 6:47 PM Suresh Marru <sm...@apache.org>
>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Isuru,
>>>>>>>>>
>>>>>>>>> Looks like your embedded architecture diagram did not make it
>>> with
>>>>> the
>>>>>>>>> email. I will request a wiki space for Custos so you can upload
>>>> there
>>>>>> for
>>>>>>>>> future discussions. For now, can you add it to the google doc?
>>>>>>>>>
>>>>>>>>> Your overall architecture proposal seems to be good. Can you
>>> also
>>>>>>>>> consider adding logging and detailed instrumentation as part of
>>> the
>>>>>> core
>>>>>>>>> architecture? This will help to have a detailed audit of each
>>> and
>>>>>> every
>>>>>>>>> action on who did what and when.
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>> Suresh
>>>>>>>>>
>>>>>>>>>> On Nov 18, 2019, at 3:41 PM, Isuru Ranawaka <
>>> irjanith@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi all,
>>>>>>>>>>
>>>>>>>>>> I am initiating this thread to provide Custos initial version
>>>>>>>>>> implementation details. Since we are moving into cloud native
>>>>>>>>> development
>>>>>>>>>> of Custos. I have come up with the following structure for
>>> micro
>>>>>>>>> services
>>>>>>>>>> based development of Custos.
>>>>>>>>>>
>>>>>>>>>> Mainly codebase composed of the atomic services and
>>> integration
>>>>>>>>> services.
>>>>>>>>>>
>>>>>>>>>> Atomic services are independent services with minimal network
>>>>>>>>> communication
>>>>>>>>>> and they have their own database with local data.
>>>>>>>>>>
>>>>>>>>>> Integration services represents the actual business use case
>>> and
>>>> it
>>>>>>>>> acts as
>>>>>>>>>> an orchestrator between core services and external services.
>>> This
>>>>> is
>>>>>>>>>> exposed as a rest api via API Gateway. Integration services
>>> talk
>>>> to
>>>>>>>>> atomic
>>>>>>>>>> services via gRPC protocol. Integration services includes gRPC
>>>> stub
>>>>>> of
>>>>>>>>>> core services. Each integration service and core service is
>>>>> deployed
>>>>>> to
>>>>>>>>>> Kubernetes cluster as services and have their own service
>>>>> addresses.
>>>>>>>>>>
>>>>>>>>>> Main problem with this architecture is transaction management
>>>> and
>>>>> it
>>>>>>>>> is
>>>>>>>>>> planned to handle with Saga using Axon Server or Camunda like
>>>>>>>>> framework.
>>>>>>>>>> All the services are written using Spring Boot and Spring
>>> Cloud.
>>>>>>>>>>
>>>>>>>>>> All the Integration services are registered with the service
>>>>> registry
>>>>>>>>> such
>>>>>>>>>> as Consul. API Gateway is used for API Management capabilities
>>>> such
>>>>>> as
>>>>>>>>> SSL
>>>>>>>>>> Termination, throttling..etc. There are Open Source API
>>> Gateways
>>>>> such
>>>>>>>>> as
>>>>>>>>>> Ambassador, Envoy which are compatible with Kubernetes and
>>>> service
>>>>>>>>>> registries. API Gateway is responsible for service discovery.
>>>>>>>>>>
>>>>>>>>>> Vault and Keycloak are deployed in Kubernetes with HA[1].
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Let me know your thoughts.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> [1]
>>>>>>>>>>
>>>>>>>>>
>>>>>>
>>>>>
>>>>
>>> https://docs.google.com/document/d/1zGXlstt9w1ge8oRWjiUv3LdQETcSYtwDngm9grB5sbk/edit?usp=sharing
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>>
>>>>>>>>>> Isuru
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Senior Engineer
>>>>>>>> WSO2 Lanka (pvt) Ltd
>>>>>>>> Blog:- http://isurur.blogspot.com/
>>>>>>>> Phone:- +94714629880
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Senior Engineer
>>>>>>> WSO2 Lanka (pvt) Ltd
>>>>>>> Blog:- http://isurur.blogspot.com/
>>>>>>> Phone:- +94714629880
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Senior Engineer
>>>>>> WSO2 Lanka (pvt) Ltd
>>>>>> Blog:- http://isurur.blogspot.com/
>>>>>> Phone:- +94714629880
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Senior Engineer
>>>> WSO2 Lanka (pvt) Ltd
>>>> Blog:- http://isurur.blogspot.com/
>>>> Phone:- +94714629880
>>>>
>>>
>>>
>>> --
>>> <https://github.com/thejkane/agm>
>>>
>>
>>
>> --
>> Senior Engineer
>> WSO2 Lanka (pvt) Ltd
>> Blog:- http://isurur.blogspot.com/
>> Phone:- +94714629880
>>
>
>
> --
> Senior Engineer
> WSO2 Lanka (pvt) Ltd
> Blog:- http://isurur.blogspot.com/
> Phone:- +94714629880
Re: Microservices Based Custos Implementation
Posted by Isuru Ranawaka <ir...@gmail.com>.
Hi Aarushi, Suresh
I am trying to bring back the functionalities of Tenant Profile Service,
IAM Service , User Profile Service and Group\ Sharing Service into Custos.
Once I am going through the existing code in airavata, I came across few
unclear parts. I have created some ER diagrams and sequence diagrams to
further realize the design. See DB Layer Design Part of [1].
1. In tenant creation flow, tenant creation function of the
IAMAdminService creates an admin user at keycloak and enable the user, but
that user is not saved in UserProfileRespository, where all keycloak
enabled users are saved. Is this a bug ?
2. How do we get tenant admin password in above case ?
3. Does tenant requester need to be registered with custos before
requesting a tenant. (Otherwise how to access custos API without access
token)
3. The users in UserProfileRepository (Keycloak enabled users) are not sync
with UserRepository and they are only synced when creating a group for a
particular domain. Is this ok ?
4. What is the relationship between domainId and tenantId ? According to
the current code tenantId is treated as domainId. See
(GroupManagerServiceHandler -> createGroup)
thanks
Isuru
[1]
https://docs.google.com/document/d/1QtUKTfAnIrErvtYnKo3FizCRL2GUiWikEzvBnHPXfWw/edit?usp=sharing
On Sat, Nov 30, 2019 at 8:13 PM Isuru Ranawaka <ir...@gmail.com> wrote:
>
> Hi Amila,
>
> Thank you very much for your comments and really helpful for further
> enhancements. please check my inline comments. Let me know your thoughts.
>
> On Sat, Nov 30, 2019 at 4:33 PM Thejaka Amila J Kanewala <
> thejaka.amila@gmail.com> wrote:
>
>> Hi Isuru,
>>
>> First, sorry for my lack of understanding about some of the bus-words used
>> in this design/discussion (e.g., gRPC, Saga, Axon, Camunda etc.)
>> Few questions and comments.
>>
>> 1. As per the design, we have two services (integration and atomic) and
>> atomic services have their own database. My assumption is you will have
>> all
>> tenant information in the tenant db and all user information in the iam
>> db.
>> When you say two dbs, did you mean two MySQL instances or two databases in
>> a single MySQL instance? Why we need two databases (even if it is running
>> in a single MySQL instance)?
>>
>
> The atomic services represents a set of functions which can be grouped
> together to do some independent work . Tenant related functions are
> grouped to
> tenant service and IAM related functions are grouped to IAM service. Data
> related to tenant service are located in tenant db and all the db
> operations of tenant db is only done through tenant service.
> If tenant services requires an information related to a tenant which
> resides in iam db, should call to IAM service and fetch. Tenant service is
> not allowed to call to iam db directly. The main reasons for this kind of
> decoupling is for independent development, deployment and release of each
> versions of services. For instance, if tenant schema is changed and want
> to do some change in tenant db, data migration or rollback it should only
> affect tenant db. it should not affect iam db. I think we can use single
> MySQL instance. Moreover, we can use different database technologies for
> different services, for e.g if tenant service only has direct insertions
> and retrievals of data with less interdependencies among data we can use
> mongoDB over mySQL. If iam service deal with data having interdependencies
> and transactions we can use MySQL over mongoDB. This may help agile
> development as well like adding new services and removing services easily.
>
>
>
>
>> 2. Can you conceptually explain how transaction management works (Saga,
>> Axon, Camunda) in this solution? Preferably using some examples (e.g.,
>> tenant registration, user registration, use deletion, etc.)
>>
>
> Let's consider tenant registration message flow, firstly, tenant
> registration integration service will talk to tenant registration core
> service to create tenant id and save tenant profile information in tenant
> db and subsequently tenant registration integration service will call to
> iam service with tenant id to create keycloak realm and save credentials.
> Let's say call to iam service is failed. Then we have two options to do
> roll back tenant info from tenant db or updated tenant info with tenant
> creation failed status. As in monolithic architecture we cannot perform two
> phase commits here. We can only achieved eventual consistency. Saga is
> architectural pattern which keeps track of all passed service calls and
> failed service calls. So we can rollback passed service calls using saga
> logs. Axon is a framework which supports saga pattern.
>
>
>> 3. At first, I thought "API gateway" refers to "Science Gateway" in your
>> document, apparently that is not the case (as per description API gateway
>> is for API management capabilities....). In that case, what is "Consumer"
>> in your diagram? Is that the actual "science gateway" (e.g., SeaGrid) or
>> "science gateway middleware" (e.g., Airavata)?
>>
>> Actual consumers would be integration services. API gateway is not doing
> any data related operation. It is just for handle SSL termination,
> service discovery, throttling, ..etc.
>
> 4. Is this design only for provisioning? In the sense, are users also
>> authenticated/authorized through this architecture?
>>
>
> Users are authenticated and authorized. Actually, issuing access tokens or
> client credentials are done through custos services.
>
>
>> 5. Regarding the asynchronous model: You probably need to consider a way
>> to
>> avoid duplicate requests and should have an API call to check the status
>> of
>> an asynchronous request.
>>
>
> Yes we need to handle it. May be we may need some background tasks to
> retry for failure and scenarios and update status for each task.
>
>
>> I have a few more questions but let me wait for answers to the above
>> questions.
>>
>> Thank you.
>> Best Regards,
>> Thejaka Amila Kanewala, PhD
>> https://github.com/thejkane/agm
>>
>> On Wed, Nov 27, 2019 at 4:36 PM Isuru Ranawaka <ir...@gmail.com>
>> wrote:
>>
>> > Hi Dimuthu,
>> >
>> > For instance, in tenant registration when client requests to register a
>> > tenant, the api should return with 200 OK saying tenant requested and
>> > subsequently calling to keycloak, vault and iam services should be run
>> in
>> > background and update the status.
>> >
>> > thanks
>> >
>> >
>> > On Wed, Nov 27, 2019 at 4:07 PM DImuthu Upeksha <
>> > dimuthu.upeksha2@gmail.com>
>> > wrote:
>> >
>> > > Hi Isuru,
>> > >
>> > > What are the use cases for asynchronous communication?
>> > >
>> > > Thanks
>> > > Dimuthu
>> > >
>> > > On Wed, Nov 27, 2019 at 3:39 PM Isuru Ranawaka <ir...@gmail.com>
>> > wrote:
>> > >
>> > > > Hi all,
>> > > >
>> > > > Just to give an update on the project,
>> > > >
>> > > > As per the discussion had with Suresh, Custos has synchronous
>> > > communication
>> > > > as well as asynchronous communication. For synchronous, there is
>> one to
>> > > one
>> > > > mapping between integration service endpoint to core service
>> endpoint.
>> > > For
>> > > > asynchronous communication there are many services communicate
>> > > > asynchronously and get work done.
>> > > >
>> > > > I have modeled and implemented asynchronous communication as a
>> service
>> > > > chain and each core service is invoked via service task and
>> integration
>> > > > service builds a service task pipeline which is modeled as service
>> > chain.
>> > > > Responses are communicated back via callbacks.
>> > > >
>> > > > Moreover, gRPC zipkin interceptors for tracing and Prometheus
>> related
>> > > > configurations for metrics are added. Docker images for each
>> service is
>> > > > built and published using docker maven plugin and helm charts are
>> > > packaged
>> > > > using helm maven plugin.
>> > > >
>> > > > thanks
>> > > >
>> > > > Isuru
>> > > >
>> > > >
>> > > > On Wed, Nov 20, 2019 at 3:12 PM Isuru Ranawaka <ir...@gmail.com>
>> > > wrote:
>> > > >
>> > > > > Hi all,
>> > > > >
>> > > > > I have implemented a basic prototype [1] according to the
>> > architecture
>> > > > > mentioned above.
>> > > > >
>> > > > > custos
>> > > > > |
>> > > > > |_ custos-core-services
>> > > > > | |
>> > > > > | | _ tenant-profile-core-service
>> > > > > |
>> > > > > |_ java
>> > > > > |_proto
>> > > > > |_ resources
>> > > > > |
>> > > > > |_custos-core-services-client-stubs
>> > > > > |
>> > > > > |_ tenant-profile-core-service-client-stub
>> > > > > |
>> > > > > |_custos-integration-services
>> > > > > |
>> > > > > | _ tenant-registration-service
>> > > > >
>> > > > >
>> > > > > Basically core services are gRPC services and their stubs and
>> clients
>> > > are
>> > > > > generated in client-stubs module. Integration services use those
>> > stubs
>> > > to
>> > > > > talk to core services. This is currently working locally and as
>> next
>> > > > step,
>> > > > > need to develop docker images, k8s artifacts and run on K8s
>> cluster
>> > to
>> > > > > check whether integration services resolves core services
>> addresses
>> > > > > seamlessly.
>> > > > >
>> > > > > [1]https://github.com/apache/airavata-custos/pull/14
>> > > > >
>> > > > > thanks
>> > > > > Isuru
>> > > > >
>> > > > >
>> > > > >
>> > > > > On Mon, Nov 18, 2019 at 9:49 PM Isuru Ranawaka <
>> irjanith@gmail.com>
>> > > > wrote:
>> > > > >
>> > > > >> Hi Suresh,
>> > > > >>
>> > > > >> Thanks for your feedback. I have updated the doc [1]. For
>> > > observability
>> > > > >> purposes we can use spring-cloud-sleuth and servers such as
>> Zipkin
>> > and
>> > > > >> those are capable of distributed tracing. For vault operations we
>> > have
>> > > > >> separate audit feature.
>> > > > >>
>> > > > >> [1]
>> > > > >>
>> > > >
>> > >
>> >
>> https://docs.google.com/document/d/1QtUKTfAnIrErvtYnKo3FizCRL2GUiWikEzvBnHPXfWw/edit?usp=sharing
>> > > > >>
>> > > > >> thanks
>> > > > >>
>> > > > >> On Mon, Nov 18, 2019 at 6:47 PM Suresh Marru <sm...@apache.org>
>> > > wrote:
>> > > > >>
>> > > > >>> Hi Isuru,
>> > > > >>>
>> > > > >>> Looks like your embedded architecture diagram did not make it
>> with
>> > > the
>> > > > >>> email. I will request a wiki space for Custos so you can upload
>> > there
>> > > > for
>> > > > >>> future discussions. For now, can you add it to the google doc?
>> > > > >>>
>> > > > >>> Your overall architecture proposal seems to be good. Can you
>> also
>> > > > >>> consider adding logging and detailed instrumentation as part of
>> the
>> > > > core
>> > > > >>> architecture? This will help to have a detailed audit of each
>> and
>> > > > every
>> > > > >>> action on who did what and when.
>> > > > >>>
>> > > > >>> Cheers,
>> > > > >>> Suresh
>> > > > >>>
>> > > > >>> > On Nov 18, 2019, at 3:41 PM, Isuru Ranawaka <
>> irjanith@gmail.com>
>> > > > >>> wrote:
>> > > > >>> >
>> > > > >>> > Hi all,
>> > > > >>> >
>> > > > >>> > I am initiating this thread to provide Custos initial version
>> > > > >>> > implementation details. Since we are moving into cloud native
>> > > > >>> development
>> > > > >>> > of Custos. I have come up with the following structure for
>> micro
>> > > > >>> services
>> > > > >>> > based development of Custos.
>> > > > >>> >
>> > > > >>> > Mainly codebase composed of the atomic services and
>> integration
>> > > > >>> services.
>> > > > >>> >
>> > > > >>> > Atomic services are independent services with minimal network
>> > > > >>> communication
>> > > > >>> > and they have their own database with local data.
>> > > > >>> >
>> > > > >>> > Integration services represents the actual business use case
>> and
>> > it
>> > > > >>> acts as
>> > > > >>> > an orchestrator between core services and external services.
>> This
>> > > is
>> > > > >>> > exposed as a rest api via API Gateway. Integration services
>> talk
>> > to
>> > > > >>> atomic
>> > > > >>> > services via gRPC protocol. Integration services includes gRPC
>> > stub
>> > > > of
>> > > > >>> > core services. Each integration service and core service is
>> > > deployed
>> > > > to
>> > > > >>> > Kubernetes cluster as services and have their own service
>> > > addresses.
>> > > > >>> >
>> > > > >>> > Main problem with this architecture is transaction management
>> > and
>> > > it
>> > > > >>> is
>> > > > >>> > planned to handle with Saga using Axon Server or Camunda like
>> > > > >>> framework.
>> > > > >>> > All the services are written using Spring Boot and Spring
>> Cloud.
>> > > > >>> >
>> > > > >>> > All the Integration services are registered with the service
>> > > registry
>> > > > >>> such
>> > > > >>> > as Consul. API Gateway is used for API Management capabilities
>> > such
>> > > > as
>> > > > >>> SSL
>> > > > >>> > Termination, throttling..etc. There are Open Source API
>> Gateways
>> > > such
>> > > > >>> as
>> > > > >>> > Ambassador, Envoy which are compatible with Kubernetes and
>> > service
>> > > > >>> > registries. API Gateway is responsible for service discovery.
>> > > > >>> >
>> > > > >>> > Vault and Keycloak are deployed in Kubernetes with HA[1].
>> > > > >>> >
>> > > > >>> >
>> > > > >>> > Let me know your thoughts.
>> > > > >>> >
>> > > > >>> >
>> > > > >>> >
>> > > > >>> > [1]
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>> https://docs.google.com/document/d/1zGXlstt9w1ge8oRWjiUv3LdQETcSYtwDngm9grB5sbk/edit?usp=sharing
>> > > > >>> >
>> > > > >>> >
>> > > > >>> > Thanks
>> > > > >>> >
>> > > > >>> > Isuru
>> > > > >>>
>> > > > >>>
>> > > > >>
>> > > > >> --
>> > > > >> Senior Engineer
>> > > > >> WSO2 Lanka (pvt) Ltd
>> > > > >> Blog:- http://isurur.blogspot.com/
>> > > > >> Phone:- +94714629880
>> > > > >>
>> > > > >
>> > > > >
>> > > > > --
>> > > > > Senior Engineer
>> > > > > WSO2 Lanka (pvt) Ltd
>> > > > > Blog:- http://isurur.blogspot.com/
>> > > > > Phone:- +94714629880
>> > > > >
>> > > >
>> > > >
>> > > > --
>> > > > Senior Engineer
>> > > > WSO2 Lanka (pvt) Ltd
>> > > > Blog:- http://isurur.blogspot.com/
>> > > > Phone:- +94714629880
>> > > >
>> > >
>> >
>> >
>> > --
>> > Senior Engineer
>> > WSO2 Lanka (pvt) Ltd
>> > Blog:- http://isurur.blogspot.com/
>> > Phone:- +94714629880
>> >
>>
>>
>> --
>> <https://github.com/thejkane/agm>
>>
>
>
> --
> Senior Engineer
> WSO2 Lanka (pvt) Ltd
> Blog:- http://isurur.blogspot.com/
> Phone:- +94714629880
>
--
Senior Engineer
WSO2 Lanka (pvt) Ltd
Blog:- http://isurur.blogspot.com/
Phone:- +94714629880
Re: Microservices Based Custos Implementation
Posted by Isuru Ranawaka <ir...@gmail.com>.
Hi all,
Tenant creation api is available for testing.
Request : POST
https://custos.scigap.org:32036/tenant-management/v1.0.0/tenant
Body :
{
"tenantName":"TenantSampleTwo",
"requesterEmail":"irjanith@gmail.com",
"requesterUsername":"isjarana",
"adminFirstName":"Jhon",
"adminLastName":"Smith",
"adminEmail":"isjarana@iu.edu",
"contacts":["+1812315366"],
"redirectURIs":["http://tenantsample.com/callback"],
"tenantURI":"http://tenantsample.com",
"scope":"email",
"domain":"tenantsample.com",
"adminPassword":"1234",
"adminUsername":"admin@custos.iu.edu"
}
Response :
{
"tenantId": "10000000",
"clientId": "custos/WYYUpkn9MLArRT0izBHI/10000000",
"clientSecret": "a1CE5mWVohyr3Xwa46utlaPleNcP3DJZAaQ8tse0",
"msg": "Use Base64 encoded clientId:clientSecret as auth token for
authorization, Credentials are activated after admin approval"
}
Once tenant is approved, tenant credentials can be retrieved via
credentials API
Request : GET
https://custos.scigap.org:32036/tenant-management/v1.0.0/credentials
Headers : "Authorization: Bearer
base64(clientId:clentSecret)"
Response :
{
"iamClientId": "pga",
"iamClientSecret": "ec717de7-164a-45f0-9c12",
"ciLogonClientId": "cilogon:/client_id/1b5466978566",
"ciLogonClientSecret": "xnLlQWpPz1LVX5t2O9lG2Gb8DSX7-LkTEnS0Nx0Eob6gz"
}
All credentials are stored in vault. Every state changing events are
captured and can be queried if required. Detailed message flow is
available at the document.
@Amila
Please find my inline comments. Some points are still not finalized and I
am struggling to figure them out.
On Wed, Dec 18, 2019 at 1:32 PM Thejaka Amila J Kanewala <
thejaka.amila@gmail.com> wrote:
> Hi Isuru,
>
> I am sorry for the late reply (was busy traveling). Some
> comments/questions/suggestions inline.
>
> Thanks
> Thejaka
>
> On Sat, Nov 30, 2019 at 5:13 PM Isuru Ranawaka <ir...@gmail.com> wrote:
>
> > Hi Amila,
> >
> > Thank you very much for your comments and really helpful for further
> > enhancements. please check my inline comments. Let me know your thoughts.
> >
> > On Sat, Nov 30, 2019 at 4:33 PM Thejaka Amila J Kanewala <
> > thejaka.amila@gmail.com> wrote:
> >
> > > Hi Isuru,
> > >
> > > First, sorry for my lack of understanding about some of the bus-words
> > used
> > > in this design/discussion (e.g., gRPC, Saga, Axon, Camunda etc.)
> > > Few questions and comments.
> > >
> > > 1. As per the design, we have two services (integration and atomic) and
> > > atomic services have their own database. My assumption is you will have
> > all
> > > tenant information in the tenant db and all user information in the iam
> > db.
> > > When you say two dbs, did you mean two MySQL instances or two databases
> > in
> > > a single MySQL instance? Why we need two databases (even if it is
> running
> > > in a single MySQL instance)?
> > >
> >
> > The atomic services represents a set of functions which can be grouped
> > together to do some independent work . Tenant related functions are
> > grouped to
> > tenant service and IAM related functions are grouped to IAM service. Data
> > related to tenant service are located in tenant db and all the db
> > operations of tenant db is only done through tenant service.
> > If tenant services requires an information related to a tenant which
> > resides in iam db, should call to IAM service and fetch. Tenant service
> is
> > not allowed to call to iam db directly. The main reasons for this kind
> of
> > decoupling is for independent development, deployment and release of
> each
> > versions of services. For instance, if tenant schema is changed and
> want
> > to do some change in tenant db, data migration or rollback it should only
> > affect tenant db. it should not affect iam db. I think we can use single
> > MySQL instance. Moreover, we can use different database technologies for
> > different services, for e.g if tenant service only has direct insertions
> > and retrievals of data with less interdependencies among data we can use
> > mongoDB over mySQL. If iam service deal with data having
> interdependencies
> > and transactions we can use MySQL over mongoDB. This may help agile
> > development as well like adding new services and removing services
> easily.
> >
> >
> In summary, the objective of decoupling services is for the easiness of
> agile development -- I do not find this a strong argument to decouple IAM
> and tenant service. This approach may make the logic more complex when it
> comes to handling crash consistency scenarios (discussed in the next
> point). Further, in general, security data is well structured and well maps
> to a relational model and also I don't think there will be a "huge" amount
> of data in those two databases.
> However, there is one scenario where decoupling make sense -- that is if
> there is a tenant who already has a user store and we need to delegate
> authentication/authorization to their user store without migrating (In this
> case IAM service has to communicate with the tenant's original user store).
> In the design, please address how delegation is handled.
>
Basically IAM is act as broker for keycloak server. My thought is
integration service is responsible for intercept requests and extract out
security metadata and delegate them to authentication and authorization
endpoints. Once authorized request is forwarded to core services .
>
> >
> >
> > > 2. Can you conceptually explain how transaction management works (Saga,
> > > Axon, Camunda) in this solution? Preferably using some examples (e.g.,
> > > tenant registration, user registration, use deletion, etc.)
> > >
> >
> > Let's consider tenant registration message flow, firstly, tenant
> > registration integration service will talk to tenant registration core
> > service to create tenant id and save tenant profile information in
> tenant
> > db and subsequently tenant registration integration service will call to
> > iam service with tenant id to create keycloak realm and save
> credentials.
> > Let's say call to iam service is failed. Then we have two options to do
> > roll back tenant info from tenant db or updated tenant info with tenant
> > creation failed status. As in monolithic architecture we cannot perform
> two
> > phase commits here. We can only achieved eventual consistency. Saga is
> > architectural pattern which keeps track of all passed service calls and
> > failed service calls. So we can rollback passed service calls using saga
> > logs. Axon is a framework which supports saga pattern.
> >
>
> "we cannot perform two-phase commits here" -- I don't quite understand why.
> You still have an orchestrator component/service that talks to IAM and
> tenant services and in fact there are variations of 2-phase commit protocol
> that are more distributed and have more complex group communication.
>
> I am not a fan of "eventual consistency" to handle security data (basically
> I believe we need strong consistency when it comes to security data -- just
> like transactions). For example, consider the use-case you explained. i.e.
> add tenant "X" with the tenant admin "Y". Suppose adding "X" is successful
> in "Tenant Core Service" but adding "Y" failed in IAM service. Now suppose,
> we have another concurrent request to add user "Z" to tenant "X" and this
> call succeeds in IAM service. Now we have a state where a normal user is
> added X but there is not tenant admin for the tenant.
>
> If both user data and tenant information are in a single DB, you can move
> consistency handling to the DB levell.
>
This is handled with states of tenant. Integration service is responsible
for updating tenant with states. Integration service is not allowing
to add users to partially completed tenants.
> >
> >
> > > 3. At first, I thought "API gateway" refers to "Science Gateway" in
> your
> > > document, apparently that is not the case (as per description API
> gateway
> > > is for API management capabilities....). In that case, what is
> "Consumer"
> > > in your diagram? Is that the actual "science gateway" (e.g., SeaGrid)
> or
> > > "science gateway middleware" (e.g., Airavata)?
> > >
> > > Actual consumers would be integration services. API gateway is not
> doing
> > any data related operation. It is just for handle SSL termination,
> > service discovery, throttling, ..etc.
> >
>
> Can you include "science gateway" (e.g., SeaGrid) and the middleware (e.g.,
> Airvata) also into the design? This will help us to better understand the
> high-level design and how different components interact. Further, consider
> how execution taking place for other gateway related software (e.g.,
> different deployment variations of Galaxy, Hub0 etc.).
>
>
> >
> > 4. Is this design only for provisioning? In the sense, are users also
> > > authenticated/authorized through this architecture?
> > >
> >
>
> It is not very clear how this design handles delegated
> authentication/authorization and supports protocols like OAuth2 (browser
> profile) -- These are important as some users don't want us to store their
> credentials. Will be great if you could explain how these are handled in
> the design.
>
We are not storing user credentials in our own databases. We are using
hashicorp vault as the credential store. My thought is every tenant/user
registering with the custos are issued a clientId and a client secret.
Clients can use those credentials to authenticate for custos APIs. we
should have authentication and authorization service to map access levels
between APIs and clients. Probably we may need token and refresh endpoints
as well.
>
> >
> > Users are authenticated and authorized. Actually, issuing access tokens
> or
> > client credentials are done through custos services.
> >
> >
> > > 5. Regarding the asynchronous model: You probably need to consider a
> way
> > to
> > > avoid duplicate requests and should have an API call to check the
> status
> > of
> > > an asynchronous request.
> > >
> >
> > Yes we need to handle it. May be we may need some background tasks to
> retry
> > for failure and scenarios and update status for each task.
> >
> >
> > > I have a few more questions but let me wait for answers to the above
> > > questions.
> > >
> > > Thank you.
> > > Best Regards,
> > > Thejaka Amila Kanewala, PhD
> > > https://github.com/thejkane/agm
> > >
> > > On Wed, Nov 27, 2019 at 4:36 PM Isuru Ranawaka <ir...@gmail.com>
> > wrote:
> > >
> > > > Hi Dimuthu,
> > > >
> > > > For instance, in tenant registration when client requests to
> register a
> > > > tenant, the api should return with 200 OK saying tenant requested and
> > > > subsequently calling to keycloak, vault and iam services should be
> run
> > in
> > > > background and update the status.
> > > >
> > > > thanks
> > > >
> > > >
> > > > On Wed, Nov 27, 2019 at 4:07 PM DImuthu Upeksha <
> > > > dimuthu.upeksha2@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi Isuru,
> > > > >
> > > > > What are the use cases for asynchronous communication?
> > > > >
> > > > > Thanks
> > > > > Dimuthu
> > > > >
> > > > > On Wed, Nov 27, 2019 at 3:39 PM Isuru Ranawaka <irjanith@gmail.com
> >
> > > > wrote:
> > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > Just to give an update on the project,
> > > > > >
> > > > > > As per the discussion had with Suresh, Custos has synchronous
> > > > > communication
> > > > > > as well as asynchronous communication. For synchronous, there is
> > one
> > > to
> > > > > one
> > > > > > mapping between integration service endpoint to core service
> > > endpoint.
> > > > > For
> > > > > > asynchronous communication there are many services communicate
> > > > > > asynchronously and get work done.
> > > > > >
> > > > > > I have modeled and implemented asynchronous communication as a
> > > service
> > > > > > chain and each core service is invoked via service task and
> > > integration
> > > > > > service builds a service task pipeline which is modeled as
> service
> > > > chain.
> > > > > > Responses are communicated back via callbacks.
> > > > > >
> > > > > > Moreover, gRPC zipkin interceptors for tracing and Prometheus
> > related
> > > > > > configurations for metrics are added. Docker images for each
> > service
> > > is
> > > > > > built and published using docker maven plugin and helm charts are
> > > > > packaged
> > > > > > using helm maven plugin.
> > > > > >
> > > > > > thanks
> > > > > >
> > > > > > Isuru
> > > > > >
> > > > > >
> > > > > > On Wed, Nov 20, 2019 at 3:12 PM Isuru Ranawaka <
> irjanith@gmail.com
> > >
> > > > > wrote:
> > > > > >
> > > > > > > Hi all,
> > > > > > >
> > > > > > > I have implemented a basic prototype [1] according to the
> > > > architecture
> > > > > > > mentioned above.
> > > > > > >
> > > > > > > custos
> > > > > > > |
> > > > > > > |_ custos-core-services
> > > > > > > | |
> > > > > > > | | _ tenant-profile-core-service
> > > > > > > |
> > > > > > > |_ java
> > > > > > > |_proto
> > > > > > > |_ resources
> > > > > > > |
> > > > > > > |_custos-core-services-client-stubs
> > > > > > > |
> > > > > > > |_ tenant-profile-core-service-client-stub
> > > > > > > |
> > > > > > > |_custos-integration-services
> > > > > > > |
> > > > > > > | _ tenant-registration-service
> > > > > > >
> > > > > > >
> > > > > > > Basically core services are gRPC services and their stubs and
> > > clients
> > > > > are
> > > > > > > generated in client-stubs module. Integration services use
> those
> > > > stubs
> > > > > to
> > > > > > > talk to core services. This is currently working locally and as
> > > next
> > > > > > step,
> > > > > > > need to develop docker images, k8s artifacts and run on K8s
> > cluster
> > > > to
> > > > > > > check whether integration services resolves core services
> > > addresses
> > > > > > > seamlessly.
> > > > > > >
> > > > > > > [1]https://github.com/apache/airavata-custos/pull/14
> > > > > > >
> > > > > > > thanks
> > > > > > > Isuru
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Mon, Nov 18, 2019 at 9:49 PM Isuru Ranawaka <
> > irjanith@gmail.com
> > > >
> > > > > > wrote:
> > > > > > >
> > > > > > >> Hi Suresh,
> > > > > > >>
> > > > > > >> Thanks for your feedback. I have updated the doc [1]. For
> > > > > observability
> > > > > > >> purposes we can use spring-cloud-sleuth and servers such as
> > Zipkin
> > > > and
> > > > > > >> those are capable of distributed tracing. For vault operations
> > we
> > > > have
> > > > > > >> separate audit feature.
> > > > > > >>
> > > > > > >> [1]
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://docs.google.com/document/d/1QtUKTfAnIrErvtYnKo3FizCRL2GUiWikEzvBnHPXfWw/edit?usp=sharing
> > > > > > >>
> > > > > > >> thanks
> > > > > > >>
> > > > > > >> On Mon, Nov 18, 2019 at 6:47 PM Suresh Marru <
> smarru@apache.org
> > >
> > > > > wrote:
> > > > > > >>
> > > > > > >>> Hi Isuru,
> > > > > > >>>
> > > > > > >>> Looks like your embedded architecture diagram did not make it
> > > with
> > > > > the
> > > > > > >>> email. I will request a wiki space for Custos so you can
> upload
> > > > there
> > > > > > for
> > > > > > >>> future discussions. For now, can you add it to the google
> doc?
> > > > > > >>>
> > > > > > >>> Your overall architecture proposal seems to be good. Can you
> > also
> > > > > > >>> consider adding logging and detailed instrumentation as part
> of
> > > the
> > > > > > core
> > > > > > >>> architecture? This will help to have a detailed audit of
> each
> > > and
> > > > > > every
> > > > > > >>> action on who did what and when.
> > > > > > >>>
> > > > > > >>> Cheers,
> > > > > > >>> Suresh
> > > > > > >>>
> > > > > > >>> > On Nov 18, 2019, at 3:41 PM, Isuru Ranawaka <
> > > irjanith@gmail.com>
> > > > > > >>> wrote:
> > > > > > >>> >
> > > > > > >>> > Hi all,
> > > > > > >>> >
> > > > > > >>> > I am initiating this thread to provide Custos initial
> version
> > > > > > >>> > implementation details. Since we are moving into cloud
> native
> > > > > > >>> development
> > > > > > >>> > of Custos. I have come up with the following structure for
> > > micro
> > > > > > >>> services
> > > > > > >>> > based development of Custos.
> > > > > > >>> >
> > > > > > >>> > Mainly codebase composed of the atomic services and
> > > integration
> > > > > > >>> services.
> > > > > > >>> >
> > > > > > >>> > Atomic services are independent services with minimal
> network
> > > > > > >>> communication
> > > > > > >>> > and they have their own database with local data.
> > > > > > >>> >
> > > > > > >>> > Integration services represents the actual business use
> case
> > > and
> > > > it
> > > > > > >>> acts as
> > > > > > >>> > an orchestrator between core services and external
> services.
> > > This
> > > > > is
> > > > > > >>> > exposed as a rest api via API Gateway. Integration services
> > > talk
> > > > to
> > > > > > >>> atomic
> > > > > > >>> > services via gRPC protocol. Integration services includes
> > gRPC
> > > > stub
> > > > > > of
> > > > > > >>> > core services. Each integration service and core service is
> > > > > deployed
> > > > > > to
> > > > > > >>> > Kubernetes cluster as services and have their own service
> > > > > addresses.
> > > > > > >>> >
> > > > > > >>> > Main problem with this architecture is transaction
> > management
> > > > and
> > > > > it
> > > > > > >>> is
> > > > > > >>> > planned to handle with Saga using Axon Server or Camunda
> like
> > > > > > >>> framework.
> > > > > > >>> > All the services are written using Spring Boot and Spring
> > > Cloud.
> > > > > > >>> >
> > > > > > >>> > All the Integration services are registered with the
> service
> > > > > registry
> > > > > > >>> such
> > > > > > >>> > as Consul. API Gateway is used for API Management
> > capabilities
> > > > such
> > > > > > as
> > > > > > >>> SSL
> > > > > > >>> > Termination, throttling..etc. There are Open Source API
> > > Gateways
> > > > > such
> > > > > > >>> as
> > > > > > >>> > Ambassador, Envoy which are compatible with Kubernetes and
> > > > service
> > > > > > >>> > registries. API Gateway is responsible for service
> discovery.
> > > > > > >>> >
> > > > > > >>> > Vault and Keycloak are deployed in Kubernetes with HA[1].
> > > > > > >>> >
> > > > > > >>> >
> > > > > > >>> > Let me know your thoughts.
> > > > > > >>> >
> > > > > > >>> >
> > > > > > >>> >
> > > > > > >>> > [1]
> > > > > > >>> >
> > > > > > >>>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://docs.google.com/document/d/1zGXlstt9w1ge8oRWjiUv3LdQETcSYtwDngm9grB5sbk/edit?usp=sharing
> > > > > > >>> >
> > > > > > >>> >
> > > > > > >>> > Thanks
> > > > > > >>> >
> > > > > > >>> > Isuru
> > > > > > >>>
> > > > > > >>>
> > > > > > >>
> > > > > > >> --
> > > > > > >> Senior Engineer
> > > > > > >> WSO2 Lanka (pvt) Ltd
> > > > > > >> Blog:- http://isurur.blogspot.com/
> > > > > > >> Phone:- +94714629880
> > > > > > >>
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Senior Engineer
> > > > > > > WSO2 Lanka (pvt) Ltd
> > > > > > > Blog:- http://isurur.blogspot.com/
> > > > > > > Phone:- +94714629880
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Senior Engineer
> > > > > > WSO2 Lanka (pvt) Ltd
> > > > > > Blog:- http://isurur.blogspot.com/
> > > > > > Phone:- +94714629880
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Senior Engineer
> > > > WSO2 Lanka (pvt) Ltd
> > > > Blog:- http://isurur.blogspot.com/
> > > > Phone:- +94714629880
> > > >
> > >
> > >
> > > --
> > > <https://github.com/thejkane/agm>
> > >
> >
> >
> > --
> > Senior Engineer
> > WSO2 Lanka (pvt) Ltd
> > Blog:- http://isurur.blogspot.com/
> > Phone:- +94714629880
> >
>
>
> --
> Best Regards,
> Thejaka Amila Kanewala, PhD
> https://github.com/thejkane/agm
>
--
Senior Engineer
WSO2 Lanka (pvt) Ltd
Blog:- http://isurur.blogspot.com/
Phone:- +94714629880
Re: Microservices Based Custos Implementation
Posted by Thejaka Amila J Kanewala <th...@gmail.com>.
Hi Isuru,
I am sorry for the late reply (was busy traveling). Some
comments/questions/suggestions inline.
Thanks
Thejaka
On Sat, Nov 30, 2019 at 5:13 PM Isuru Ranawaka <ir...@gmail.com> wrote:
> Hi Amila,
>
> Thank you very much for your comments and really helpful for further
> enhancements. please check my inline comments. Let me know your thoughts.
>
> On Sat, Nov 30, 2019 at 4:33 PM Thejaka Amila J Kanewala <
> thejaka.amila@gmail.com> wrote:
>
> > Hi Isuru,
> >
> > First, sorry for my lack of understanding about some of the bus-words
> used
> > in this design/discussion (e.g., gRPC, Saga, Axon, Camunda etc.)
> > Few questions and comments.
> >
> > 1. As per the design, we have two services (integration and atomic) and
> > atomic services have their own database. My assumption is you will have
> all
> > tenant information in the tenant db and all user information in the iam
> db.
> > When you say two dbs, did you mean two MySQL instances or two databases
> in
> > a single MySQL instance? Why we need two databases (even if it is running
> > in a single MySQL instance)?
> >
>
> The atomic services represents a set of functions which can be grouped
> together to do some independent work . Tenant related functions are
> grouped to
> tenant service and IAM related functions are grouped to IAM service. Data
> related to tenant service are located in tenant db and all the db
> operations of tenant db is only done through tenant service.
> If tenant services requires an information related to a tenant which
> resides in iam db, should call to IAM service and fetch. Tenant service is
> not allowed to call to iam db directly. The main reasons for this kind of
> decoupling is for independent development, deployment and release of each
> versions of services. For instance, if tenant schema is changed and want
> to do some change in tenant db, data migration or rollback it should only
> affect tenant db. it should not affect iam db. I think we can use single
> MySQL instance. Moreover, we can use different database technologies for
> different services, for e.g if tenant service only has direct insertions
> and retrievals of data with less interdependencies among data we can use
> mongoDB over mySQL. If iam service deal with data having interdependencies
> and transactions we can use MySQL over mongoDB. This may help agile
> development as well like adding new services and removing services easily.
>
>
In summary, the objective of decoupling services is for the easiness of
agile development -- I do not find this a strong argument to decouple IAM
and tenant service. This approach may make the logic more complex when it
comes to handling crash consistency scenarios (discussed in the next
point). Further, in general, security data is well structured and well maps
to a relational model and also I don't think there will be a "huge" amount
of data in those two databases.
However, there is one scenario where decoupling make sense -- that is if
there is a tenant who already has a user store and we need to delegate
authentication/authorization to their user store without migrating (In this
case IAM service has to communicate with the tenant's original user store).
In the design, please address how delegation is handled.
>
>
>
> > 2. Can you conceptually explain how transaction management works (Saga,
> > Axon, Camunda) in this solution? Preferably using some examples (e.g.,
> > tenant registration, user registration, use deletion, etc.)
> >
>
> Let's consider tenant registration message flow, firstly, tenant
> registration integration service will talk to tenant registration core
> service to create tenant id and save tenant profile information in tenant
> db and subsequently tenant registration integration service will call to
> iam service with tenant id to create keycloak realm and save credentials.
> Let's say call to iam service is failed. Then we have two options to do
> roll back tenant info from tenant db or updated tenant info with tenant
> creation failed status. As in monolithic architecture we cannot perform two
> phase commits here. We can only achieved eventual consistency. Saga is
> architectural pattern which keeps track of all passed service calls and
> failed service calls. So we can rollback passed service calls using saga
> logs. Axon is a framework which supports saga pattern.
>
"we cannot perform two-phase commits here" -- I don't quite understand why.
You still have an orchestrator component/service that talks to IAM and
tenant services and in fact there are variations of 2-phase commit protocol
that are more distributed and have more complex group communication.
I am not a fan of "eventual consistency" to handle security data (basically
I believe we need strong consistency when it comes to security data -- just
like transactions). For example, consider the use-case you explained. i.e.
add tenant "X" with the tenant admin "Y". Suppose adding "X" is successful
in "Tenant Core Service" but adding "Y" failed in IAM service. Now suppose,
we have another concurrent request to add user "Z" to tenant "X" and this
call succeeds in IAM service. Now we have a state where a normal user is
added X but there is not tenant admin for the tenant.
If both user data and tenant information are in a single DB, you can move
consistency handling to the DB level.
>
>
> > 3. At first, I thought "API gateway" refers to "Science Gateway" in your
> > document, apparently that is not the case (as per description API gateway
> > is for API management capabilities....). In that case, what is "Consumer"
> > in your diagram? Is that the actual "science gateway" (e.g., SeaGrid) or
> > "science gateway middleware" (e.g., Airavata)?
> >
> > Actual consumers would be integration services. API gateway is not doing
> any data related operation. It is just for handle SSL termination,
> service discovery, throttling, ..etc.
>
Can you include "science gateway" (e.g., SeaGrid) and the middleware (e.g.,
Airvata) also into the design? This will help us to better understand the
high-level design and how different components interact. Further, consider
how execution taking place for other gateway related software (e.g.,
different deployment variations of Galaxy, Hub0 etc.).
>
> 4. Is this design only for provisioning? In the sense, are users also
> > authenticated/authorized through this architecture?
> >
>
It is not very clear how this design handles delegated
authentication/authorization and supports protocols like OAuth2 (browser
profile) -- These are important as some users don't want us to store their
credentials. Will be great if you could explain how these are handled in
the design.
>
> Users are authenticated and authorized. Actually, issuing access tokens or
> client credentials are done through custos services.
>
>
> > 5. Regarding the asynchronous model: You probably need to consider a way
> to
> > avoid duplicate requests and should have an API call to check the status
> of
> > an asynchronous request.
> >
>
> Yes we need to handle it. May be we may need some background tasks to retry
> for failure and scenarios and update status for each task.
>
>
> > I have a few more questions but let me wait for answers to the above
> > questions.
> >
> > Thank you.
> > Best Regards,
> > Thejaka Amila Kanewala, PhD
> > https://github.com/thejkane/agm
> >
> > On Wed, Nov 27, 2019 at 4:36 PM Isuru Ranawaka <ir...@gmail.com>
> wrote:
> >
> > > Hi Dimuthu,
> > >
> > > For instance, in tenant registration when client requests to register a
> > > tenant, the api should return with 200 OK saying tenant requested and
> > > subsequently calling to keycloak, vault and iam services should be run
> in
> > > background and update the status.
> > >
> > > thanks
> > >
> > >
> > > On Wed, Nov 27, 2019 at 4:07 PM DImuthu Upeksha <
> > > dimuthu.upeksha2@gmail.com>
> > > wrote:
> > >
> > > > Hi Isuru,
> > > >
> > > > What are the use cases for asynchronous communication?
> > > >
> > > > Thanks
> > > > Dimuthu
> > > >
> > > > On Wed, Nov 27, 2019 at 3:39 PM Isuru Ranawaka <ir...@gmail.com>
> > > wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > Just to give an update on the project,
> > > > >
> > > > > As per the discussion had with Suresh, Custos has synchronous
> > > > communication
> > > > > as well as asynchronous communication. For synchronous, there is
> one
> > to
> > > > one
> > > > > mapping between integration service endpoint to core service
> > endpoint.
> > > > For
> > > > > asynchronous communication there are many services communicate
> > > > > asynchronously and get work done.
> > > > >
> > > > > I have modeled and implemented asynchronous communication as a
> > service
> > > > > chain and each core service is invoked via service task and
> > integration
> > > > > service builds a service task pipeline which is modeled as service
> > > chain.
> > > > > Responses are communicated back via callbacks.
> > > > >
> > > > > Moreover, gRPC zipkin interceptors for tracing and Prometheus
> related
> > > > > configurations for metrics are added. Docker images for each
> service
> > is
> > > > > built and published using docker maven plugin and helm charts are
> > > > packaged
> > > > > using helm maven plugin.
> > > > >
> > > > > thanks
> > > > >
> > > > > Isuru
> > > > >
> > > > >
> > > > > On Wed, Nov 20, 2019 at 3:12 PM Isuru Ranawaka <irjanith@gmail.com
> >
> > > > wrote:
> > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > I have implemented a basic prototype [1] according to the
> > > architecture
> > > > > > mentioned above.
> > > > > >
> > > > > > custos
> > > > > > |
> > > > > > |_ custos-core-services
> > > > > > | |
> > > > > > | | _ tenant-profile-core-service
> > > > > > |
> > > > > > |_ java
> > > > > > |_proto
> > > > > > |_ resources
> > > > > > |
> > > > > > |_custos-core-services-client-stubs
> > > > > > |
> > > > > > |_ tenant-profile-core-service-client-stub
> > > > > > |
> > > > > > |_custos-integration-services
> > > > > > |
> > > > > > | _ tenant-registration-service
> > > > > >
> > > > > >
> > > > > > Basically core services are gRPC services and their stubs and
> > clients
> > > > are
> > > > > > generated in client-stubs module. Integration services use those
> > > stubs
> > > > to
> > > > > > talk to core services. This is currently working locally and as
> > next
> > > > > step,
> > > > > > need to develop docker images, k8s artifacts and run on K8s
> cluster
> > > to
> > > > > > check whether integration services resolves core services
> > addresses
> > > > > > seamlessly.
> > > > > >
> > > > > > [1]https://github.com/apache/airavata-custos/pull/14
> > > > > >
> > > > > > thanks
> > > > > > Isuru
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Mon, Nov 18, 2019 at 9:49 PM Isuru Ranawaka <
> irjanith@gmail.com
> > >
> > > > > wrote:
> > > > > >
> > > > > >> Hi Suresh,
> > > > > >>
> > > > > >> Thanks for your feedback. I have updated the doc [1]. For
> > > > observability
> > > > > >> purposes we can use spring-cloud-sleuth and servers such as
> Zipkin
> > > and
> > > > > >> those are capable of distributed tracing. For vault operations
> we
> > > have
> > > > > >> separate audit feature.
> > > > > >>
> > > > > >> [1]
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://docs.google.com/document/d/1QtUKTfAnIrErvtYnKo3FizCRL2GUiWikEzvBnHPXfWw/edit?usp=sharing
> > > > > >>
> > > > > >> thanks
> > > > > >>
> > > > > >> On Mon, Nov 18, 2019 at 6:47 PM Suresh Marru <smarru@apache.org
> >
> > > > wrote:
> > > > > >>
> > > > > >>> Hi Isuru,
> > > > > >>>
> > > > > >>> Looks like your embedded architecture diagram did not make it
> > with
> > > > the
> > > > > >>> email. I will request a wiki space for Custos so you can upload
> > > there
> > > > > for
> > > > > >>> future discussions. For now, can you add it to the google doc?
> > > > > >>>
> > > > > >>> Your overall architecture proposal seems to be good. Can you
> also
> > > > > >>> consider adding logging and detailed instrumentation as part of
> > the
> > > > > core
> > > > > >>> architecture? This will help to have a detailed audit of each
> > and
> > > > > every
> > > > > >>> action on who did what and when.
> > > > > >>>
> > > > > >>> Cheers,
> > > > > >>> Suresh
> > > > > >>>
> > > > > >>> > On Nov 18, 2019, at 3:41 PM, Isuru Ranawaka <
> > irjanith@gmail.com>
> > > > > >>> wrote:
> > > > > >>> >
> > > > > >>> > Hi all,
> > > > > >>> >
> > > > > >>> > I am initiating this thread to provide Custos initial version
> > > > > >>> > implementation details. Since we are moving into cloud native
> > > > > >>> development
> > > > > >>> > of Custos. I have come up with the following structure for
> > micro
> > > > > >>> services
> > > > > >>> > based development of Custos.
> > > > > >>> >
> > > > > >>> > Mainly codebase composed of the atomic services and
> > integration
> > > > > >>> services.
> > > > > >>> >
> > > > > >>> > Atomic services are independent services with minimal network
> > > > > >>> communication
> > > > > >>> > and they have their own database with local data.
> > > > > >>> >
> > > > > >>> > Integration services represents the actual business use case
> > and
> > > it
> > > > > >>> acts as
> > > > > >>> > an orchestrator between core services and external services.
> > This
> > > > is
> > > > > >>> > exposed as a rest api via API Gateway. Integration services
> > talk
> > > to
> > > > > >>> atomic
> > > > > >>> > services via gRPC protocol. Integration services includes
> gRPC
> > > stub
> > > > > of
> > > > > >>> > core services. Each integration service and core service is
> > > > deployed
> > > > > to
> > > > > >>> > Kubernetes cluster as services and have their own service
> > > > addresses.
> > > > > >>> >
> > > > > >>> > Main problem with this architecture is transaction
> management
> > > and
> > > > it
> > > > > >>> is
> > > > > >>> > planned to handle with Saga using Axon Server or Camunda like
> > > > > >>> framework.
> > > > > >>> > All the services are written using Spring Boot and Spring
> > Cloud.
> > > > > >>> >
> > > > > >>> > All the Integration services are registered with the service
> > > > registry
> > > > > >>> such
> > > > > >>> > as Consul. API Gateway is used for API Management
> capabilities
> > > such
> > > > > as
> > > > > >>> SSL
> > > > > >>> > Termination, throttling..etc. There are Open Source API
> > Gateways
> > > > such
> > > > > >>> as
> > > > > >>> > Ambassador, Envoy which are compatible with Kubernetes and
> > > service
> > > > > >>> > registries. API Gateway is responsible for service discovery.
> > > > > >>> >
> > > > > >>> > Vault and Keycloak are deployed in Kubernetes with HA[1].
> > > > > >>> >
> > > > > >>> >
> > > > > >>> > Let me know your thoughts.
> > > > > >>> >
> > > > > >>> >
> > > > > >>> >
> > > > > >>> > [1]
> > > > > >>> >
> > > > > >>>
> > > > >
> > > >
> > >
> >
> https://docs.google.com/document/d/1zGXlstt9w1ge8oRWjiUv3LdQETcSYtwDngm9grB5sbk/edit?usp=sharing
> > > > > >>> >
> > > > > >>> >
> > > > > >>> > Thanks
> > > > > >>> >
> > > > > >>> > Isuru
> > > > > >>>
> > > > > >>>
> > > > > >>
> > > > > >> --
> > > > > >> Senior Engineer
> > > > > >> WSO2 Lanka (pvt) Ltd
> > > > > >> Blog:- http://isurur.blogspot.com/
> > > > > >> Phone:- +94714629880
> > > > > >>
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Senior Engineer
> > > > > > WSO2 Lanka (pvt) Ltd
> > > > > > Blog:- http://isurur.blogspot.com/
> > > > > > Phone:- +94714629880
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Senior Engineer
> > > > > WSO2 Lanka (pvt) Ltd
> > > > > Blog:- http://isurur.blogspot.com/
> > > > > Phone:- +94714629880
> > > > >
> > > >
> > >
> > >
> > > --
> > > Senior Engineer
> > > WSO2 Lanka (pvt) Ltd
> > > Blog:- http://isurur.blogspot.com/
> > > Phone:- +94714629880
> > >
> >
> >
> > --
> > <https://github.com/thejkane/agm>
> >
>
>
> --
> Senior Engineer
> WSO2 Lanka (pvt) Ltd
> Blog:- http://isurur.blogspot.com/
> Phone:- +94714629880
>
--
Best Regards,
Thejaka Amila Kanewala, PhD
https://github.com/thejkane/agm