You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@synapse.apache.org by Nuwan Jayawardene <ns...@gmail.com> on 2021/10/05 04:45:30 UTC
Re: Synapse Containerized architecture Proposal
[image: image.gif]Hi all,
Took more time than expected how the Medium post is finalized and can be
found here:
https://nuwnjay.medium.com/how-to-containerize-an-esb-26d8a5404ee9
It includes all the design decisions, implementation steps and general
experience we had when working on the project.
Kind regards
On Tue, Aug 31, 2021 at 1:20 PM Nuwan Jayawardene <ns...@gmail.com> wrote:
> Hi[image: image.gif] all,
> I've created a video recording of the demo that showcases the changes
> mentioned above.
> https://www.youtube.com/watch?v=jeqBZOi60DQ
>
> The slides used as supplementary material in the video can be found here
>
> https://docs.google.com/presentation/d/1rZjEwvnET91aa-4kd7tgw6ATapPb8bWZpiARioSM6Uo/edit?usp=sharing
>
> This will be followed with a Medium post detailing the implementation
> decisions within this week.
> Kind regards
>
>
>
> [image: image.gif]
> On Mon, Aug 23, 2021 at 9:47 PM Nuwan Jayawardene <ns...@gmail.com>
> wrote:
>
>> Hi All,
>> this project I was working on was related to GSoC 2021, and with it
>> coming to a conclusion, I'd like to give an update on how it has progressed
>> up until this point.
>> After the initial proposal, on-boarding and preliminary development work
>> we settled on the following main objectives;
>>
>> 1. Containerizing Synapse and running on k8s cluster (via Minikube)
>> 2. Parameter passing to container/pods
>> 3. Abiding to the 12 Factor app spec especially when considering
>> factor 3 (Config)
>>
>> The way I went about achieving the above (and the reasons for certain
>> decisions taken) is as follows;
>>
>> 1. This objective was quite straightforward. I used the OpenJDK 8
>> image <https://hub.docker.com/_/openjdk> as the base and copied a
>> built pack onto it.
>> A Dockerfile was written to achieve this and is linked here:
>> https://github.com/n-jay/gsoc-2021/blob/feature1/synapse/Dockerfile.
>>
>> 2. This is similar to a currently available feature
>> <https://docs.wso2.com/display/EI660/Injecting+Parameters+as+Environment+Variables>
>> in the Micro Intergrator component of WSO2 EI; a fork of Synapse.
>> The goal was to enable parameter passing at runtime such that certain
>> config fields (eg: URIs) can be updated at deployment, instead of hardcoded.
>>
>> As mentioned in the previous mail this was done via the use of a
>> ConfigMap.
>> Endpoint factories within Synapse were amended to extract the
>> injected parameters (visible within the pod as environment variables), and
>> the config file populated thereafter.
>>
>> The ConfigMap used for the demo is linked here:
>> https://github.com/n-jay/gsoc-2021/blob/feature1/synapse-config.yml
>> Code changes done to Synapse is present as 2 commits linked here
>> <https://github.com/n-jay/synapse/commit/d725f3fcaaac5e3e110a40161a984a3a88236f32>
>> and here
>> <https://github.com/n-jay/synapse/commit/a85a439e10a75d75a925e33414c2e75c5f07e1f5> which
>> are currently on my personal fork.
>> They will be compiled into a single PR once a code review has been
>> conducted and unit tests written.
>>
>> 3. Config separation as per 12 Factor apps was done via Kubernetes
>> PersistentVolumes.
>> First a PersistentVolume was created and then allocated to the
>> Synapse deployment via a PersistenVolume Claim.
>> It was created such that the a directory created within the node
>> (with synapse.xml files already within it) gets mounted onto the
>> {SYNAPSE_HOME}/repostory/conf/synape-config directory.
>> PersistentVolume:
>> https://github.com/n-jay/gsoc-2021/blob/feature1/synapse-persistent-volume.yml
>> PersistentVolumeClaim:
>> https://github.com/n-jay/gsoc-2021/blob/feature1/synapse-persistent-volume-claim.yml
>>
>> What this allows is for hot swapping the synapse configuration as
>> needed as opposed to having it baked into the docker image, requiring
>> different images for different configs.
>> This combined with the the parameter injection allows for complete
>> independence as to what variables are passed.
>>
>>
>> Once the above were reached they were demoed against customized versions
>> of config sample 56
>> <https://synapse.apache.org/userguide/samples/sample56.html> and 800
>> <https://synapse.apache.org/userguide/samples/sample800.html>.
>> The implementation arrangement consisted of containerized Apache Axis2
>> server and client along with Synapse in an Ambassador pattern arrangement.
>> The 3 deployments were connected together using 2 k8s services.
>>
>> All deployment files are available in the repository linked here:
>> https://github.com/n-jay/gsoc-2021/tree/feature1.
>> The README has been updated to include directory and file details, it
>> will be further amended with deployment instructions.
>>
>> A medium blog post is also being written and will me attached to this
>> thread upon completion along with a screencapped video of the impementation.
>> Kind regards
>>
>> [image: image.gif]
>> On Tue, Aug 10, 2021 at 11:46 AM Nuwan Jayawardene <ns...@gmail.com>
>> wrote:
>>
>>> [image: image.gif]
>>> Hi all,
>>>
>>>>
>>>> 1. Inject configuration parameters to Synapse at runtime
>>>> This would include parameters such as URI endpoints
>>>>
>>>> I've been working on the above function and as for progress so far;
>>>
>>> 1. Created a ConfigMap setup in Kubernetes and connected it with the
>>> Synapse deployment to pass environment variables.
>>> This is to make the image completely environment independent, in
>>> keeping with the 12 Factors.
>>> 2. Took inspiration from WSO2 EI
>>> <https://ei.docs.wso2.com/en/latest/> and which already has similar
>>> functionality
>>> <https://docs.wso2.com/display/EI660/Injecting+Parameters+as+Environment+Variables>,
>>> and ported it to Synapse.
>>> The first pull request including the relevant classes can be found
>>> in linked PR to my local fork:
>>> https://github.com/n-jay/synapse/pull/1.
>>> Haven't merged it yet due to CD build failure because of a version
>>> mismatch.
>>>
>>> I'm currently debugging this implementation.
>>>
>>> Would appreciate some feedback on this.
>>> Kind regards
>>>
>>>
>>>
>>> [image: image.gif]
>>>
>>> On Fri, Aug 6, 2021 at 3:27 PM Nuwan Jayawardene <ns...@gmail.com>
>>> wrote:
>>>
>>>> Hi All,
>>>> I’m currently working on a project containerizing Synapse, taking it
>>>> into a cloud-native form.
>>>> The work done so far is as follows;
>>>>
>>>> - Dockerize Synapse using Dockerfile
>>>> - Get Synapse to run within Kubernetes environment in a deployment
>>>> - Setup a Kubernetes cluster in the Ambassador pattern with Synapse
>>>> acting as a proxy between a dockerized client and server (for testing
>>>> purposes).
>>>>
>>>> Moving forward I’m working on developing the following;
>>>>
>>>> 1. Inject configuration parameters to Synapse at runtime
>>>> This would include parameters such as URI endpoints
>>>> 2. Reduce Synapse image size by making runtime more lean.
>>>> By measuring performance metrics and reducing
>>>> legacy/redundant/superfluous components
>>>>
>>>> Would greatly appreciate your thoughts and feedback on this.
>>>> Regards
>>>>
>>>> --
>>>> *Nuwan Jayawardene*
>>>> Assistant Lecturer, Informatics Institute of Technology (IIT).
>>>> Research Assistant, University of Moratuwa
>>>> Tel - +94 71 181 3299
>>>> https://www.linkedin.com/m/in/nuwanjaya
>>>>
>>>>
>>>> [image: image.gif]
>>>> [image: image.gif]
>>>>
>>>
>>>
>>> --
>>> *Nuwan Jayawardene*
>>> Assistant Lecturer, Informatics Institute of Technology (IIT).
>>> Research Assistant, University of Moratuwa
>>> Tel - +94 71 181 3299
>>> https://www.linkedin.com/m/in/nuwanjaya
>>>
>>>
>>
>> --
>> *Nuwan Jayawardene*
>> Assistant Lecturer, Informatics Institute of Technology (IIT).
>> Research Assistant, University of Moratuwa
>> Tel - +94 71 181 3299
>> https://www.linkedin.com/m/in/nuwanjaya
>>
>>
>
> --
> *Nuwan Jayawardene*
> Assistant Lecturer, Informatics Institute of Technology (IIT).
> Research Assistant, University of Moratuwa
> Tel - +94 71 181 3299
> https://www.linkedin.com/m/in/nuwanjaya
>
>
--
*Nuwan Jayawardene*
Assistant Lecturer, Informatics Institute of Technology (IIT).
Research Assistant, University of Moratuwa
https://www.linkedin.com/m/in/nuwanjaya
Re: Synapse Containerized architecture Proposal
Posted by Nuwan Jayawardene <ns...@gmail.com>.
Hi all,
I'm currently continuing development on the Synapse containerization
project.
The project progress can be tracked here:
https://github.com/users/n-jay/projects/1/views/1?visibleFields=%5B29200798%2C%22Title%22%2C%22Assignees%22%2C%22Status%22%2C%22Linked+pull+requests%22%5D&layout=table
The linked project tracker currently exists in my personal Synapse fork.
I'm hoping to recreate this in the Synapse main repo once write access has
been granted with the Apache committership.
This will allow me to link PRs directly with the individual tasks.
Kind regards
On Thu, Sep 15, 2022 at 11:35 AM Nuwan Jayawardene <ns...@gmail.com>
wrote:
> Hi all,
>
>> The next step is to complete WSDL endpoint injection and finalize the PR.
>>
> WSDL endpoint parameter injection has also been finalized. PR is linked
> herewith.
> https://github.com/apache/synapse/pull/55
>
> Kind regards
>
> On Tue, Sep 13, 2022 at 8:58 PM Nuwan Jayawardene <ns...@gmail.com>
> wrote:
>
>> Hi all,
>> been making progress on the development of this with support from @Isuru
>> Udana <is...@gmail.com>.
>> After submitting the initial PR and undergoing code reviews, the current
>> state includes the finalization of SOAP endpoint parameter injection.
>> PR is linked herewith: https://github.com/apache/synapse/pull/55
>>
>> Our initial attempts were to try and generalize the parameter injection
>> code to make it uniform across as many endpoints as possible.
>> However, this proved to be problematic given the variations of different
>> implementations.
>>
>> The next step is to complete WSDL endpoint injection and finalize the PR.
>> Will be moving onto subsequent features thereafter:
>> https://docs.google.com/spreadsheets/d/1L8lZ1ZLk6UxwuA0LijiZxsHXiNdVwG_bYUhccjfaB2g/edit?usp=sharing
>>
>> Kind regards
>>
>>
>> On Tue, Apr 12, 2022 at 12:36 PM Nuwan Jayawardene <ns...@gmail.com>
>> wrote:
>>
>>> Hi all,
>>> after the PR was submitted with the changes
>>> <https://github.com/apache/synapse/pull/55> I was working on earlier, I
>>> decided to move into the next stage of the containerization project and
>>> begin adding the rest of the planned features.
>>> I've compiled an open Google Sheet linked herewith with the state of
>>> work thus far.
>>>
>>> https://docs.google.com/spreadsheets/d/1L8lZ1ZLk6UxwuA0LijiZxsHXiNdVwG_bYUhccjfaB2g/edit?usp=sharing
>>>
>>> As it can be seen, parameter injection via environment variables is the
>>> priority atm.
>>> I've used the WSO2 Micro Integrator as the closest analog to Synapse,
>>> imo and based this feature list off of that.
>>>
>>> Would appreciate any and all feedback on this.
>>> Kind regards
>>>
>>> On Tue, Oct 5, 2021 at 10:15 AM Nuwan Jayawardene <ns...@gmail.com>
>>> wrote:
>>>
>>>> [image: image.gif]Hi all,
>>>> Took more time than expected how the Medium post is finalized and can
>>>> be found here:
>>>> https://nuwnjay.medium.com/how-to-containerize-an-esb-26d8a5404ee9
>>>>
>>>> It includes all the design decisions, implementation steps and general
>>>> experience we had when working on the project.
>>>> Kind regards
>>>>
>>>> On Tue, Aug 31, 2021 at 1:20 PM Nuwan Jayawardene <ns...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi[image: image.gif] all,
>>>>> I've created a video recording of the demo that showcases the changes
>>>>> mentioned above.
>>>>> https://www.youtube.com/watch?v=jeqBZOi60DQ
>>>>>
>>>>> The slides used as supplementary material in the video can be found
>>>>> here
>>>>>
>>>>> https://docs.google.com/presentation/d/1rZjEwvnET91aa-4kd7tgw6ATapPb8bWZpiARioSM6Uo/edit?usp=sharing
>>>>>
>>>>> This will be followed with a Medium post detailing the implementation
>>>>> decisions within this week.
>>>>> Kind regards
>>>>>
>>>>>
>>>>>
>>>>> [image: image.gif]
>>>>> On Mon, Aug 23, 2021 at 9:47 PM Nuwan Jayawardene <ns...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi All,
>>>>>> this project I was working on was related to GSoC 2021, and with it
>>>>>> coming to a conclusion, I'd like to give an update on how it has progressed
>>>>>> up until this point.
>>>>>> After the initial proposal, on-boarding and preliminary development
>>>>>> work we settled on the following main objectives;
>>>>>>
>>>>>> 1. Containerizing Synapse and running on k8s cluster (via
>>>>>> Minikube)
>>>>>> 2. Parameter passing to container/pods
>>>>>> 3. Abiding to the 12 Factor app spec especially when considering
>>>>>> factor 3 (Config)
>>>>>>
>>>>>> The way I went about achieving the above (and the reasons for certain
>>>>>> decisions taken) is as follows;
>>>>>>
>>>>>> 1. This objective was quite straightforward. I used the OpenJDK 8
>>>>>> image <https://hub.docker.com/_/openjdk> as the base and copied a
>>>>>> built pack onto it.
>>>>>> A Dockerfile was written to achieve this and is linked here:
>>>>>> https://github.com/n-jay/gsoc-2021/blob/feature1/synapse/Dockerfile
>>>>>> .
>>>>>>
>>>>>> 2. This is similar to a currently available feature
>>>>>> <https://docs.wso2.com/display/EI660/Injecting+Parameters+as+Environment+Variables>
>>>>>> in the Micro Intergrator component of WSO2 EI; a fork of Synapse.
>>>>>> The goal was to enable parameter passing at runtime such that
>>>>>> certain config fields (eg: URIs) can be updated at deployment, instead of
>>>>>> hardcoded.
>>>>>>
>>>>>> As mentioned in the previous mail this was done via the use of a
>>>>>> ConfigMap.
>>>>>> Endpoint factories within Synapse were amended to extract the
>>>>>> injected parameters (visible within the pod as environment variables), and
>>>>>> the config file populated thereafter.
>>>>>>
>>>>>> The ConfigMap used for the demo is linked here:
>>>>>> https://github.com/n-jay/gsoc-2021/blob/feature1/synapse-config.yml
>>>>>> Code changes done to Synapse is present as 2 commits linked here
>>>>>> <https://github.com/n-jay/synapse/commit/d725f3fcaaac5e3e110a40161a984a3a88236f32>
>>>>>> and here
>>>>>> <https://github.com/n-jay/synapse/commit/a85a439e10a75d75a925e33414c2e75c5f07e1f5> which
>>>>>> are currently on my personal fork.
>>>>>> They will be compiled into a single PR once a code review has
>>>>>> been conducted and unit tests written.
>>>>>>
>>>>>> 3. Config separation as per 12 Factor apps was done via
>>>>>> Kubernetes PersistentVolumes.
>>>>>> First a PersistentVolume was created and then allocated to the
>>>>>> Synapse deployment via a PersistenVolume Claim.
>>>>>> It was created such that the a directory created within the node
>>>>>> (with synapse.xml files already within it) gets mounted onto the
>>>>>> {SYNAPSE_HOME}/repostory/conf/synape-config directory.
>>>>>> PersistentVolume:
>>>>>> https://github.com/n-jay/gsoc-2021/blob/feature1/synapse-persistent-volume.yml
>>>>>> PersistentVolumeClaim:
>>>>>> https://github.com/n-jay/gsoc-2021/blob/feature1/synapse-persistent-volume-claim.yml
>>>>>>
>>>>>> What this allows is for hot swapping the synapse configuration as
>>>>>> needed as opposed to having it baked into the docker image, requiring
>>>>>> different images for different configs.
>>>>>> This combined with the the parameter injection allows for
>>>>>> complete independence as to what variables are passed.
>>>>>>
>>>>>>
>>>>>> Once the above were reached they were demoed against customized
>>>>>> versions of config sample 56
>>>>>> <https://synapse.apache.org/userguide/samples/sample56.html> and 800
>>>>>> <https://synapse.apache.org/userguide/samples/sample800.html>.
>>>>>> The implementation arrangement consisted of containerized Apache
>>>>>> Axis2 server and client along with Synapse in an Ambassador pattern
>>>>>> arrangement.
>>>>>> The 3 deployments were connected together using 2 k8s services.
>>>>>>
>>>>>> All deployment files are available in the repository linked here:
>>>>>> https://github.com/n-jay/gsoc-2021/tree/feature1.
>>>>>> The README has been updated to include directory and file details, it
>>>>>> will be further amended with deployment instructions.
>>>>>>
>>>>>> A medium blog post is also being written and will me attached to this
>>>>>> thread upon completion along with a screencapped video of the impementation.
>>>>>> Kind regards
>>>>>>
>>>>>> [image: image.gif]
>>>>>> On Tue, Aug 10, 2021 at 11:46 AM Nuwan Jayawardene <ns...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> [image: image.gif]
>>>>>>> Hi all,
>>>>>>>
>>>>>>>>
>>>>>>>> 1. Inject configuration parameters to Synapse at runtime
>>>>>>>> This would include parameters such as URI endpoints
>>>>>>>>
>>>>>>>> I've been working on the above function and as for progress so far;
>>>>>>>
>>>>>>> 1. Created a ConfigMap setup in Kubernetes and connected it with
>>>>>>> the Synapse deployment to pass environment variables.
>>>>>>> This is to make the image completely environment independent, in
>>>>>>> keeping with the 12 Factors.
>>>>>>> 2. Took inspiration from WSO2 EI
>>>>>>> <https://ei.docs.wso2.com/en/latest/> and which already has similar
>>>>>>> functionality
>>>>>>> <https://docs.wso2.com/display/EI660/Injecting+Parameters+as+Environment+Variables>,
>>>>>>> and ported it to Synapse.
>>>>>>> The first pull request including the relevant classes can be
>>>>>>> found in linked PR to my local fork:
>>>>>>> https://github.com/n-jay/synapse/pull/1.
>>>>>>> Haven't merged it yet due to CD build failure because of a
>>>>>>> version mismatch.
>>>>>>>
>>>>>>> I'm currently debugging this implementation.
>>>>>>>
>>>>>>> Would appreciate some feedback on this.
>>>>>>> Kind regards
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> [image: image.gif]
>>>>>>>
>>>>>>> On Fri, Aug 6, 2021 at 3:27 PM Nuwan Jayawardene <ns...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi All,
>>>>>>>> I’m currently working on a project containerizing Synapse, taking
>>>>>>>> it into a cloud-native form.
>>>>>>>> The work done so far is as follows;
>>>>>>>>
>>>>>>>> - Dockerize Synapse using Dockerfile
>>>>>>>> - Get Synapse to run within Kubernetes environment in a
>>>>>>>> deployment
>>>>>>>> - Setup a Kubernetes cluster in the Ambassador pattern with
>>>>>>>> Synapse acting as a proxy between a dockerized client and server (for
>>>>>>>> testing purposes).
>>>>>>>>
>>>>>>>> Moving forward I’m working on developing the following;
>>>>>>>>
>>>>>>>> 1. Inject configuration parameters to Synapse at runtime
>>>>>>>> This would include parameters such as URI endpoints
>>>>>>>> 2. Reduce Synapse image size by making runtime more lean.
>>>>>>>> By measuring performance metrics and reducing
>>>>>>>> legacy/redundant/superfluous components
>>>>>>>>
>>>>>>>> Would greatly appreciate your thoughts and feedback on this.
>>>>>>>> Regards
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Nuwan Jayawardene*
>>>>>>>> Assistant Lecturer, Informatics Institute of Technology (IIT).
>>>>>>>> Research Assistant, University of Moratuwa
>>>>>>>> Tel - +94 71 181 3299
>>>>>>>> https://www.linkedin.com/m/in/nuwanjaya
>>>>>>>>
>>>>>>>>
>>>>>>>> [image: image.gif]
>>>>>>>> [image: image.gif]
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Nuwan Jayawardene*
>>>>>>> Assistant Lecturer, Informatics Institute of Technology (IIT).
>>>>>>> Research Assistant, University of Moratuwa
>>>>>>> Tel - +94 71 181 3299
>>>>>>> https://www.linkedin.com/m/in/nuwanjaya
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Nuwan Jayawardene*
>>>>>> Assistant Lecturer, Informatics Institute of Technology (IIT).
>>>>>> Research Assistant, University of Moratuwa
>>>>>> Tel - +94 71 181 3299
>>>>>> https://www.linkedin.com/m/in/nuwanjaya
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> *Nuwan Jayawardene*
>>>>> Assistant Lecturer, Informatics Institute of Technology (IIT).
>>>>> Research Assistant, University of Moratuwa
>>>>> Tel - +94 71 181 3299
>>>>> https://www.linkedin.com/m/in/nuwanjaya
>>>>>
>>>>>
>>>>
>>>> --
>>>> *Nuwan Jayawardene*
>>>> Assistant Lecturer, Informatics Institute of Technology (IIT).
>>>> Research Assistant, University of Moratuwa
>>>>
>>>> https://www.linkedin.com/m/in/nuwanjaya
>>>>
>>>>
>>>
>>> --
>>> *Nuwan Jayawardene*
>>> Assistant Lecturer, Informatics Institute of Technology (IIT).
>>> Research Assistant, University of Moratuwa
>>>
>>> https://www.linkedin.com/m/in/nuwanjaya
>>>
>>>
>>
>> --
>> *Nuwan Jayawardene*
>> Assistant Lecturer, Informatics Institute of Technology (IIT).
>> Research Assistant, University of Moratuwa
>>
>> https://www.linkedin.com/m/in/nuwanjaya
>>
>>
>
> --
> *Nuwan Jayawardene*
> Assistant Lecturer, Informatics Institute of Technology (IIT).
> Research Assistant, University of Moratuwa
>
> https://www.linkedin.com/m/in/nuwanjaya
>
>
--
*Nuwan Jayawardene*
https://www.linkedin.com/m/in/nuwanjaya
Re: Synapse Containerized architecture Proposal
Posted by Nuwan Jayawardene <ns...@gmail.com>.
Hi all,
> The next step is to complete WSDL endpoint injection and finalize the PR.
>
WSDL endpoint parameter injection has also been finalized. PR is linked
herewith.
https://github.com/apache/synapse/pull/55
Kind regards
On Tue, Sep 13, 2022 at 8:58 PM Nuwan Jayawardene <ns...@gmail.com> wrote:
> Hi all,
> been making progress on the development of this with support from @Isuru
> Udana <is...@gmail.com>.
> After submitting the initial PR and undergoing code reviews, the current
> state includes the finalization of SOAP endpoint parameter injection.
> PR is linked herewith: https://github.com/apache/synapse/pull/55
>
> Our initial attempts were to try and generalize the parameter injection
> code to make it uniform across as many endpoints as possible.
> However, this proved to be problematic given the variations of different
> implementations.
>
> The next step is to complete WSDL endpoint injection and finalize the PR.
> Will be moving onto subsequent features thereafter:
> https://docs.google.com/spreadsheets/d/1L8lZ1ZLk6UxwuA0LijiZxsHXiNdVwG_bYUhccjfaB2g/edit?usp=sharing
>
> Kind regards
>
>
> On Tue, Apr 12, 2022 at 12:36 PM Nuwan Jayawardene <ns...@gmail.com>
> wrote:
>
>> Hi all,
>> after the PR was submitted with the changes
>> <https://github.com/apache/synapse/pull/55> I was working on earlier, I
>> decided to move into the next stage of the containerization project and
>> begin adding the rest of the planned features.
>> I've compiled an open Google Sheet linked herewith with the state of work
>> thus far.
>>
>> https://docs.google.com/spreadsheets/d/1L8lZ1ZLk6UxwuA0LijiZxsHXiNdVwG_bYUhccjfaB2g/edit?usp=sharing
>>
>> As it can be seen, parameter injection via environment variables is the
>> priority atm.
>> I've used the WSO2 Micro Integrator as the closest analog to Synapse, imo
>> and based this feature list off of that.
>>
>> Would appreciate any and all feedback on this.
>> Kind regards
>>
>> On Tue, Oct 5, 2021 at 10:15 AM Nuwan Jayawardene <ns...@gmail.com>
>> wrote:
>>
>>> [image: image.gif]Hi all,
>>> Took more time than expected how the Medium post is finalized and can be
>>> found here:
>>> https://nuwnjay.medium.com/how-to-containerize-an-esb-26d8a5404ee9
>>>
>>> It includes all the design decisions, implementation steps and general
>>> experience we had when working on the project.
>>> Kind regards
>>>
>>> On Tue, Aug 31, 2021 at 1:20 PM Nuwan Jayawardene <ns...@gmail.com>
>>> wrote:
>>>
>>>> Hi[image: image.gif] all,
>>>> I've created a video recording of the demo that showcases the changes
>>>> mentioned above.
>>>> https://www.youtube.com/watch?v=jeqBZOi60DQ
>>>>
>>>> The slides used as supplementary material in the video can be found here
>>>>
>>>> https://docs.google.com/presentation/d/1rZjEwvnET91aa-4kd7tgw6ATapPb8bWZpiARioSM6Uo/edit?usp=sharing
>>>>
>>>> This will be followed with a Medium post detailing the implementation
>>>> decisions within this week.
>>>> Kind regards
>>>>
>>>>
>>>>
>>>> [image: image.gif]
>>>> On Mon, Aug 23, 2021 at 9:47 PM Nuwan Jayawardene <ns...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi All,
>>>>> this project I was working on was related to GSoC 2021, and with it
>>>>> coming to a conclusion, I'd like to give an update on how it has progressed
>>>>> up until this point.
>>>>> After the initial proposal, on-boarding and preliminary development
>>>>> work we settled on the following main objectives;
>>>>>
>>>>> 1. Containerizing Synapse and running on k8s cluster (via Minikube)
>>>>> 2. Parameter passing to container/pods
>>>>> 3. Abiding to the 12 Factor app spec especially when considering
>>>>> factor 3 (Config)
>>>>>
>>>>> The way I went about achieving the above (and the reasons for certain
>>>>> decisions taken) is as follows;
>>>>>
>>>>> 1. This objective was quite straightforward. I used the OpenJDK 8
>>>>> image <https://hub.docker.com/_/openjdk> as the base and copied a
>>>>> built pack onto it.
>>>>> A Dockerfile was written to achieve this and is linked here:
>>>>> https://github.com/n-jay/gsoc-2021/blob/feature1/synapse/Dockerfile
>>>>> .
>>>>>
>>>>> 2. This is similar to a currently available feature
>>>>> <https://docs.wso2.com/display/EI660/Injecting+Parameters+as+Environment+Variables>
>>>>> in the Micro Intergrator component of WSO2 EI; a fork of Synapse.
>>>>> The goal was to enable parameter passing at runtime such that
>>>>> certain config fields (eg: URIs) can be updated at deployment, instead of
>>>>> hardcoded.
>>>>>
>>>>> As mentioned in the previous mail this was done via the use of a
>>>>> ConfigMap.
>>>>> Endpoint factories within Synapse were amended to extract the
>>>>> injected parameters (visible within the pod as environment variables), and
>>>>> the config file populated thereafter.
>>>>>
>>>>> The ConfigMap used for the demo is linked here:
>>>>> https://github.com/n-jay/gsoc-2021/blob/feature1/synapse-config.yml
>>>>> Code changes done to Synapse is present as 2 commits linked here
>>>>> <https://github.com/n-jay/synapse/commit/d725f3fcaaac5e3e110a40161a984a3a88236f32>
>>>>> and here
>>>>> <https://github.com/n-jay/synapse/commit/a85a439e10a75d75a925e33414c2e75c5f07e1f5> which
>>>>> are currently on my personal fork.
>>>>> They will be compiled into a single PR once a code review has been
>>>>> conducted and unit tests written.
>>>>>
>>>>> 3. Config separation as per 12 Factor apps was done via Kubernetes
>>>>> PersistentVolumes.
>>>>> First a PersistentVolume was created and then allocated to the
>>>>> Synapse deployment via a PersistenVolume Claim.
>>>>> It was created such that the a directory created within the node
>>>>> (with synapse.xml files already within it) gets mounted onto the
>>>>> {SYNAPSE_HOME}/repostory/conf/synape-config directory.
>>>>> PersistentVolume:
>>>>> https://github.com/n-jay/gsoc-2021/blob/feature1/synapse-persistent-volume.yml
>>>>> PersistentVolumeClaim:
>>>>> https://github.com/n-jay/gsoc-2021/blob/feature1/synapse-persistent-volume-claim.yml
>>>>>
>>>>> What this allows is for hot swapping the synapse configuration as
>>>>> needed as opposed to having it baked into the docker image, requiring
>>>>> different images for different configs.
>>>>> This combined with the the parameter injection allows for complete
>>>>> independence as to what variables are passed.
>>>>>
>>>>>
>>>>> Once the above were reached they were demoed against customized
>>>>> versions of config sample 56
>>>>> <https://synapse.apache.org/userguide/samples/sample56.html> and 800
>>>>> <https://synapse.apache.org/userguide/samples/sample800.html>.
>>>>> The implementation arrangement consisted of containerized Apache Axis2
>>>>> server and client along with Synapse in an Ambassador pattern arrangement.
>>>>> The 3 deployments were connected together using 2 k8s services.
>>>>>
>>>>> All deployment files are available in the repository linked here:
>>>>> https://github.com/n-jay/gsoc-2021/tree/feature1.
>>>>> The README has been updated to include directory and file details, it
>>>>> will be further amended with deployment instructions.
>>>>>
>>>>> A medium blog post is also being written and will me attached to this
>>>>> thread upon completion along with a screencapped video of the impementation.
>>>>> Kind regards
>>>>>
>>>>> [image: image.gif]
>>>>> On Tue, Aug 10, 2021 at 11:46 AM Nuwan Jayawardene <ns...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> [image: image.gif]
>>>>>> Hi all,
>>>>>>
>>>>>>>
>>>>>>> 1. Inject configuration parameters to Synapse at runtime
>>>>>>> This would include parameters such as URI endpoints
>>>>>>>
>>>>>>> I've been working on the above function and as for progress so far;
>>>>>>
>>>>>> 1. Created a ConfigMap setup in Kubernetes and connected it with
>>>>>> the Synapse deployment to pass environment variables.
>>>>>> This is to make the image completely environment independent, in
>>>>>> keeping with the 12 Factors.
>>>>>> 2. Took inspiration from WSO2 EI
>>>>>> <https://ei.docs.wso2.com/en/latest/> and which already has similar
>>>>>> functionality
>>>>>> <https://docs.wso2.com/display/EI660/Injecting+Parameters+as+Environment+Variables>,
>>>>>> and ported it to Synapse.
>>>>>> The first pull request including the relevant classes can be
>>>>>> found in linked PR to my local fork:
>>>>>> https://github.com/n-jay/synapse/pull/1.
>>>>>> Haven't merged it yet due to CD build failure because of a
>>>>>> version mismatch.
>>>>>>
>>>>>> I'm currently debugging this implementation.
>>>>>>
>>>>>> Would appreciate some feedback on this.
>>>>>> Kind regards
>>>>>>
>>>>>>
>>>>>>
>>>>>> [image: image.gif]
>>>>>>
>>>>>> On Fri, Aug 6, 2021 at 3:27 PM Nuwan Jayawardene <ns...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi All,
>>>>>>> I’m currently working on a project containerizing Synapse, taking it
>>>>>>> into a cloud-native form.
>>>>>>> The work done so far is as follows;
>>>>>>>
>>>>>>> - Dockerize Synapse using Dockerfile
>>>>>>> - Get Synapse to run within Kubernetes environment in a
>>>>>>> deployment
>>>>>>> - Setup a Kubernetes cluster in the Ambassador pattern with
>>>>>>> Synapse acting as a proxy between a dockerized client and server (for
>>>>>>> testing purposes).
>>>>>>>
>>>>>>> Moving forward I’m working on developing the following;
>>>>>>>
>>>>>>> 1. Inject configuration parameters to Synapse at runtime
>>>>>>> This would include parameters such as URI endpoints
>>>>>>> 2. Reduce Synapse image size by making runtime more lean.
>>>>>>> By measuring performance metrics and reducing
>>>>>>> legacy/redundant/superfluous components
>>>>>>>
>>>>>>> Would greatly appreciate your thoughts and feedback on this.
>>>>>>> Regards
>>>>>>>
>>>>>>> --
>>>>>>> *Nuwan Jayawardene*
>>>>>>> Assistant Lecturer, Informatics Institute of Technology (IIT).
>>>>>>> Research Assistant, University of Moratuwa
>>>>>>> Tel - +94 71 181 3299
>>>>>>> https://www.linkedin.com/m/in/nuwanjaya
>>>>>>>
>>>>>>>
>>>>>>> [image: image.gif]
>>>>>>> [image: image.gif]
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Nuwan Jayawardene*
>>>>>> Assistant Lecturer, Informatics Institute of Technology (IIT).
>>>>>> Research Assistant, University of Moratuwa
>>>>>> Tel - +94 71 181 3299
>>>>>> https://www.linkedin.com/m/in/nuwanjaya
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> *Nuwan Jayawardene*
>>>>> Assistant Lecturer, Informatics Institute of Technology (IIT).
>>>>> Research Assistant, University of Moratuwa
>>>>> Tel - +94 71 181 3299
>>>>> https://www.linkedin.com/m/in/nuwanjaya
>>>>>
>>>>>
>>>>
>>>> --
>>>> *Nuwan Jayawardene*
>>>> Assistant Lecturer, Informatics Institute of Technology (IIT).
>>>> Research Assistant, University of Moratuwa
>>>> Tel - +94 71 181 3299
>>>> https://www.linkedin.com/m/in/nuwanjaya
>>>>
>>>>
>>>
>>> --
>>> *Nuwan Jayawardene*
>>> Assistant Lecturer, Informatics Institute of Technology (IIT).
>>> Research Assistant, University of Moratuwa
>>>
>>> https://www.linkedin.com/m/in/nuwanjaya
>>>
>>>
>>
>> --
>> *Nuwan Jayawardene*
>> Assistant Lecturer, Informatics Institute of Technology (IIT).
>> Research Assistant, University of Moratuwa
>>
>> https://www.linkedin.com/m/in/nuwanjaya
>>
>>
>
> --
> *Nuwan Jayawardene*
> Assistant Lecturer, Informatics Institute of Technology (IIT).
> Research Assistant, University of Moratuwa
>
> https://www.linkedin.com/m/in/nuwanjaya
>
>
--
*Nuwan Jayawardene*
Assistant Lecturer, Informatics Institute of Technology (IIT).
Research Assistant, University of Moratuwa
https://www.linkedin.com/m/in/nuwanjaya
Re: Synapse Containerized architecture Proposal
Posted by Nuwan Jayawardene <ns...@gmail.com>.
Hi all,
been making progress on the development of this with support from @Isuru
Udana <is...@gmail.com>.
After submitting the initial PR and undergoing code reviews, the current
state includes the finalization of SOAP endpoint parameter injection.
PR is linked herewith: https://github.com/apache/synapse/pull/55
Our initial attempts were to try and generalize the parameter injection
code to make it uniform across as many endpoints as possible.
However, this proved to be problematic given the variations of different
implementations.
The next step is to complete WSDL endpoint injection and finalize the PR.
Will be moving onto subsequent features thereafter:
https://docs.google.com/spreadsheets/d/1L8lZ1ZLk6UxwuA0LijiZxsHXiNdVwG_bYUhccjfaB2g/edit?usp=sharing
Kind regards
On Tue, Apr 12, 2022 at 12:36 PM Nuwan Jayawardene <ns...@gmail.com>
wrote:
> Hi all,
> after the PR was submitted with the changes
> <https://github.com/apache/synapse/pull/55> I was working on earlier, I
> decided to move into the next stage of the containerization project and
> begin adding the rest of the planned features.
> I've compiled an open Google Sheet linked herewith with the state of work
> thus far.
>
> https://docs.google.com/spreadsheets/d/1L8lZ1ZLk6UxwuA0LijiZxsHXiNdVwG_bYUhccjfaB2g/edit?usp=sharing
>
> As it can be seen, parameter injection via environment variables is the
> priority atm.
> I've used the WSO2 Micro Integrator as the closest analog to Synapse, imo
> and based this feature list off of that.
>
> Would appreciate any and all feedback on this.
> Kind regards
>
> On Tue, Oct 5, 2021 at 10:15 AM Nuwan Jayawardene <ns...@gmail.com>
> wrote:
>
>> [image: image.gif]Hi all,
>> Took more time than expected how the Medium post is finalized and can be
>> found here:
>> https://nuwnjay.medium.com/how-to-containerize-an-esb-26d8a5404ee9
>>
>> It includes all the design decisions, implementation steps and general
>> experience we had when working on the project.
>> Kind regards
>>
>> On Tue, Aug 31, 2021 at 1:20 PM Nuwan Jayawardene <ns...@gmail.com>
>> wrote:
>>
>>> Hi[image: image.gif] all,
>>> I've created a video recording of the demo that showcases the changes
>>> mentioned above.
>>> https://www.youtube.com/watch?v=jeqBZOi60DQ
>>>
>>> The slides used as supplementary material in the video can be found here
>>>
>>> https://docs.google.com/presentation/d/1rZjEwvnET91aa-4kd7tgw6ATapPb8bWZpiARioSM6Uo/edit?usp=sharing
>>>
>>> This will be followed with a Medium post detailing the implementation
>>> decisions within this week.
>>> Kind regards
>>>
>>>
>>>
>>> [image: image.gif]
>>> On Mon, Aug 23, 2021 at 9:47 PM Nuwan Jayawardene <ns...@gmail.com>
>>> wrote:
>>>
>>>> Hi All,
>>>> this project I was working on was related to GSoC 2021, and with it
>>>> coming to a conclusion, I'd like to give an update on how it has progressed
>>>> up until this point.
>>>> After the initial proposal, on-boarding and preliminary development
>>>> work we settled on the following main objectives;
>>>>
>>>> 1. Containerizing Synapse and running on k8s cluster (via Minikube)
>>>> 2. Parameter passing to container/pods
>>>> 3. Abiding to the 12 Factor app spec especially when considering
>>>> factor 3 (Config)
>>>>
>>>> The way I went about achieving the above (and the reasons for certain
>>>> decisions taken) is as follows;
>>>>
>>>> 1. This objective was quite straightforward. I used the OpenJDK 8
>>>> image <https://hub.docker.com/_/openjdk> as the base and copied a
>>>> built pack onto it.
>>>> A Dockerfile was written to achieve this and is linked here:
>>>> https://github.com/n-jay/gsoc-2021/blob/feature1/synapse/Dockerfile.
>>>>
>>>> 2. This is similar to a currently available feature
>>>> <https://docs.wso2.com/display/EI660/Injecting+Parameters+as+Environment+Variables>
>>>> in the Micro Intergrator component of WSO2 EI; a fork of Synapse.
>>>> The goal was to enable parameter passing at runtime such that
>>>> certain config fields (eg: URIs) can be updated at deployment, instead of
>>>> hardcoded.
>>>>
>>>> As mentioned in the previous mail this was done via the use of a
>>>> ConfigMap.
>>>> Endpoint factories within Synapse were amended to extract the
>>>> injected parameters (visible within the pod as environment variables), and
>>>> the config file populated thereafter.
>>>>
>>>> The ConfigMap used for the demo is linked here:
>>>> https://github.com/n-jay/gsoc-2021/blob/feature1/synapse-config.yml
>>>> Code changes done to Synapse is present as 2 commits linked here
>>>> <https://github.com/n-jay/synapse/commit/d725f3fcaaac5e3e110a40161a984a3a88236f32>
>>>> and here
>>>> <https://github.com/n-jay/synapse/commit/a85a439e10a75d75a925e33414c2e75c5f07e1f5> which
>>>> are currently on my personal fork.
>>>> They will be compiled into a single PR once a code review has been
>>>> conducted and unit tests written.
>>>>
>>>> 3. Config separation as per 12 Factor apps was done via Kubernetes
>>>> PersistentVolumes.
>>>> First a PersistentVolume was created and then allocated to the
>>>> Synapse deployment via a PersistenVolume Claim.
>>>> It was created such that the a directory created within the node
>>>> (with synapse.xml files already within it) gets mounted onto the
>>>> {SYNAPSE_HOME}/repostory/conf/synape-config directory.
>>>> PersistentVolume:
>>>> https://github.com/n-jay/gsoc-2021/blob/feature1/synapse-persistent-volume.yml
>>>> PersistentVolumeClaim:
>>>> https://github.com/n-jay/gsoc-2021/blob/feature1/synapse-persistent-volume-claim.yml
>>>>
>>>> What this allows is for hot swapping the synapse configuration as
>>>> needed as opposed to having it baked into the docker image, requiring
>>>> different images for different configs.
>>>> This combined with the the parameter injection allows for complete
>>>> independence as to what variables are passed.
>>>>
>>>>
>>>> Once the above were reached they were demoed against customized
>>>> versions of config sample 56
>>>> <https://synapse.apache.org/userguide/samples/sample56.html> and 800
>>>> <https://synapse.apache.org/userguide/samples/sample800.html>.
>>>> The implementation arrangement consisted of containerized Apache Axis2
>>>> server and client along with Synapse in an Ambassador pattern arrangement.
>>>> The 3 deployments were connected together using 2 k8s services.
>>>>
>>>> All deployment files are available in the repository linked here:
>>>> https://github.com/n-jay/gsoc-2021/tree/feature1.
>>>> The README has been updated to include directory and file details, it
>>>> will be further amended with deployment instructions.
>>>>
>>>> A medium blog post is also being written and will me attached to this
>>>> thread upon completion along with a screencapped video of the impementation.
>>>> Kind regards
>>>>
>>>> [image: image.gif]
>>>> On Tue, Aug 10, 2021 at 11:46 AM Nuwan Jayawardene <ns...@gmail.com>
>>>> wrote:
>>>>
>>>>> [image: image.gif]
>>>>> Hi all,
>>>>>
>>>>>>
>>>>>> 1. Inject configuration parameters to Synapse at runtime
>>>>>> This would include parameters such as URI endpoints
>>>>>>
>>>>>> I've been working on the above function and as for progress so far;
>>>>>
>>>>> 1. Created a ConfigMap setup in Kubernetes and connected it with
>>>>> the Synapse deployment to pass environment variables.
>>>>> This is to make the image completely environment independent, in
>>>>> keeping with the 12 Factors.
>>>>> 2. Took inspiration from WSO2 EI
>>>>> <https://ei.docs.wso2.com/en/latest/> and which already has similar
>>>>> functionality
>>>>> <https://docs.wso2.com/display/EI660/Injecting+Parameters+as+Environment+Variables>,
>>>>> and ported it to Synapse.
>>>>> The first pull request including the relevant classes can be found
>>>>> in linked PR to my local fork:
>>>>> https://github.com/n-jay/synapse/pull/1.
>>>>> Haven't merged it yet due to CD build failure because of a version
>>>>> mismatch.
>>>>>
>>>>> I'm currently debugging this implementation.
>>>>>
>>>>> Would appreciate some feedback on this.
>>>>> Kind regards
>>>>>
>>>>>
>>>>>
>>>>> [image: image.gif]
>>>>>
>>>>> On Fri, Aug 6, 2021 at 3:27 PM Nuwan Jayawardene <ns...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi All,
>>>>>> I’m currently working on a project containerizing Synapse, taking it
>>>>>> into a cloud-native form.
>>>>>> The work done so far is as follows;
>>>>>>
>>>>>> - Dockerize Synapse using Dockerfile
>>>>>> - Get Synapse to run within Kubernetes environment in a deployment
>>>>>> - Setup a Kubernetes cluster in the Ambassador pattern with
>>>>>> Synapse acting as a proxy between a dockerized client and server (for
>>>>>> testing purposes).
>>>>>>
>>>>>> Moving forward I’m working on developing the following;
>>>>>>
>>>>>> 1. Inject configuration parameters to Synapse at runtime
>>>>>> This would include parameters such as URI endpoints
>>>>>> 2. Reduce Synapse image size by making runtime more lean.
>>>>>> By measuring performance metrics and reducing
>>>>>> legacy/redundant/superfluous components
>>>>>>
>>>>>> Would greatly appreciate your thoughts and feedback on this.
>>>>>> Regards
>>>>>>
>>>>>> --
>>>>>> *Nuwan Jayawardene*
>>>>>> Assistant Lecturer, Informatics Institute of Technology (IIT).
>>>>>> Research Assistant, University of Moratuwa
>>>>>> Tel - +94 71 181 3299
>>>>>> https://www.linkedin.com/m/in/nuwanjaya
>>>>>>
>>>>>>
>>>>>> [image: image.gif]
>>>>>> [image: image.gif]
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Nuwan Jayawardene*
>>>>> Assistant Lecturer, Informatics Institute of Technology (IIT).
>>>>> Research Assistant, University of Moratuwa
>>>>> Tel - +94 71 181 3299
>>>>> https://www.linkedin.com/m/in/nuwanjaya
>>>>>
>>>>>
>>>>
>>>> --
>>>> *Nuwan Jayawardene*
>>>> Assistant Lecturer, Informatics Institute of Technology (IIT).
>>>> Research Assistant, University of Moratuwa
>>>> Tel - +94 71 181 3299
>>>> https://www.linkedin.com/m/in/nuwanjaya
>>>>
>>>>
>>>
>>> --
>>> *Nuwan Jayawardene*
>>> Assistant Lecturer, Informatics Institute of Technology (IIT).
>>> Research Assistant, University of Moratuwa
>>> Tel - +94 71 181 3299
>>> https://www.linkedin.com/m/in/nuwanjaya
>>>
>>>
>>
>> --
>> *Nuwan Jayawardene*
>> Assistant Lecturer, Informatics Institute of Technology (IIT).
>> Research Assistant, University of Moratuwa
>>
>> https://www.linkedin.com/m/in/nuwanjaya
>>
>>
>
> --
> *Nuwan Jayawardene*
> Assistant Lecturer, Informatics Institute of Technology (IIT).
> Research Assistant, University of Moratuwa
>
> https://www.linkedin.com/m/in/nuwanjaya
>
>
--
*Nuwan Jayawardene*
Assistant Lecturer, Informatics Institute of Technology (IIT).
Research Assistant, University of Moratuwa
https://www.linkedin.com/m/in/nuwanjaya
Re: Synapse Containerized architecture Proposal
Posted by Nuwan Jayawardene <ns...@gmail.com>.
Hi all,
after the PR was submitted with the changes
<https://github.com/apache/synapse/pull/55> I was working on earlier, I
decided to move into the next stage of the containerization project and
begin adding the rest of the planned features.
I've compiled an open Google Sheet linked herewith with the state of work
thus far.
https://docs.google.com/spreadsheets/d/1L8lZ1ZLk6UxwuA0LijiZxsHXiNdVwG_bYUhccjfaB2g/edit?usp=sharing
As it can be seen, parameter injection via environment variables is the
priority atm.
I've used the WSO2 Micro Integrator as the closest analog to Synapse, imo
and based this feature list off of that.
Would appreciate any and all feedback on this.
Kind regards
On Tue, Oct 5, 2021 at 10:15 AM Nuwan Jayawardene <ns...@gmail.com> wrote:
> [image: image.gif]Hi all,
> Took more time than expected how the Medium post is finalized and can be
> found here:
> https://nuwnjay.medium.com/how-to-containerize-an-esb-26d8a5404ee9
>
> It includes all the design decisions, implementation steps and general
> experience we had when working on the project.
> Kind regards
>
> On Tue, Aug 31, 2021 at 1:20 PM Nuwan Jayawardene <ns...@gmail.com>
> wrote:
>
>> Hi[image: image.gif] all,
>> I've created a video recording of the demo that showcases the changes
>> mentioned above.
>> https://www.youtube.com/watch?v=jeqBZOi60DQ
>>
>> The slides used as supplementary material in the video can be found here
>>
>> https://docs.google.com/presentation/d/1rZjEwvnET91aa-4kd7tgw6ATapPb8bWZpiARioSM6Uo/edit?usp=sharing
>>
>> This will be followed with a Medium post detailing the implementation
>> decisions within this week.
>> Kind regards
>>
>>
>>
>> [image: image.gif]
>> On Mon, Aug 23, 2021 at 9:47 PM Nuwan Jayawardene <ns...@gmail.com>
>> wrote:
>>
>>> Hi All,
>>> this project I was working on was related to GSoC 2021, and with it
>>> coming to a conclusion, I'd like to give an update on how it has progressed
>>> up until this point.
>>> After the initial proposal, on-boarding and preliminary development work
>>> we settled on the following main objectives;
>>>
>>> 1. Containerizing Synapse and running on k8s cluster (via Minikube)
>>> 2. Parameter passing to container/pods
>>> 3. Abiding to the 12 Factor app spec especially when considering
>>> factor 3 (Config)
>>>
>>> The way I went about achieving the above (and the reasons for certain
>>> decisions taken) is as follows;
>>>
>>> 1. This objective was quite straightforward. I used the OpenJDK 8
>>> image <https://hub.docker.com/_/openjdk> as the base and copied a
>>> built pack onto it.
>>> A Dockerfile was written to achieve this and is linked here:
>>> https://github.com/n-jay/gsoc-2021/blob/feature1/synapse/Dockerfile.
>>>
>>> 2. This is similar to a currently available feature
>>> <https://docs.wso2.com/display/EI660/Injecting+Parameters+as+Environment+Variables>
>>> in the Micro Intergrator component of WSO2 EI; a fork of Synapse.
>>> The goal was to enable parameter passing at runtime such that
>>> certain config fields (eg: URIs) can be updated at deployment, instead of
>>> hardcoded.
>>>
>>> As mentioned in the previous mail this was done via the use of a
>>> ConfigMap.
>>> Endpoint factories within Synapse were amended to extract the
>>> injected parameters (visible within the pod as environment variables), and
>>> the config file populated thereafter.
>>>
>>> The ConfigMap used for the demo is linked here:
>>> https://github.com/n-jay/gsoc-2021/blob/feature1/synapse-config.yml
>>> Code changes done to Synapse is present as 2 commits linked here
>>> <https://github.com/n-jay/synapse/commit/d725f3fcaaac5e3e110a40161a984a3a88236f32>
>>> and here
>>> <https://github.com/n-jay/synapse/commit/a85a439e10a75d75a925e33414c2e75c5f07e1f5> which
>>> are currently on my personal fork.
>>> They will be compiled into a single PR once a code review has been
>>> conducted and unit tests written.
>>>
>>> 3. Config separation as per 12 Factor apps was done via Kubernetes
>>> PersistentVolumes.
>>> First a PersistentVolume was created and then allocated to the
>>> Synapse deployment via a PersistenVolume Claim.
>>> It was created such that the a directory created within the node
>>> (with synapse.xml files already within it) gets mounted onto the
>>> {SYNAPSE_HOME}/repostory/conf/synape-config directory.
>>> PersistentVolume:
>>> https://github.com/n-jay/gsoc-2021/blob/feature1/synapse-persistent-volume.yml
>>> PersistentVolumeClaim:
>>> https://github.com/n-jay/gsoc-2021/blob/feature1/synapse-persistent-volume-claim.yml
>>>
>>> What this allows is for hot swapping the synapse configuration as
>>> needed as opposed to having it baked into the docker image, requiring
>>> different images for different configs.
>>> This combined with the the parameter injection allows for complete
>>> independence as to what variables are passed.
>>>
>>>
>>> Once the above were reached they were demoed against customized versions
>>> of config sample 56
>>> <https://synapse.apache.org/userguide/samples/sample56.html> and 800
>>> <https://synapse.apache.org/userguide/samples/sample800.html>.
>>> The implementation arrangement consisted of containerized Apache Axis2
>>> server and client along with Synapse in an Ambassador pattern arrangement.
>>> The 3 deployments were connected together using 2 k8s services.
>>>
>>> All deployment files are available in the repository linked here:
>>> https://github.com/n-jay/gsoc-2021/tree/feature1.
>>> The README has been updated to include directory and file details, it
>>> will be further amended with deployment instructions.
>>>
>>> A medium blog post is also being written and will me attached to this
>>> thread upon completion along with a screencapped video of the impementation.
>>> Kind regards
>>>
>>> [image: image.gif]
>>> On Tue, Aug 10, 2021 at 11:46 AM Nuwan Jayawardene <ns...@gmail.com>
>>> wrote:
>>>
>>>> [image: image.gif]
>>>> Hi all,
>>>>
>>>>>
>>>>> 1. Inject configuration parameters to Synapse at runtime
>>>>> This would include parameters such as URI endpoints
>>>>>
>>>>> I've been working on the above function and as for progress so far;
>>>>
>>>> 1. Created a ConfigMap setup in Kubernetes and connected it with
>>>> the Synapse deployment to pass environment variables.
>>>> This is to make the image completely environment independent, in
>>>> keeping with the 12 Factors.
>>>> 2. Took inspiration from WSO2 EI
>>>> <https://ei.docs.wso2.com/en/latest/> and which already has similar
>>>> functionality
>>>> <https://docs.wso2.com/display/EI660/Injecting+Parameters+as+Environment+Variables>,
>>>> and ported it to Synapse.
>>>> The first pull request including the relevant classes can be found
>>>> in linked PR to my local fork:
>>>> https://github.com/n-jay/synapse/pull/1.
>>>> Haven't merged it yet due to CD build failure because of a version
>>>> mismatch.
>>>>
>>>> I'm currently debugging this implementation.
>>>>
>>>> Would appreciate some feedback on this.
>>>> Kind regards
>>>>
>>>>
>>>>
>>>> [image: image.gif]
>>>>
>>>> On Fri, Aug 6, 2021 at 3:27 PM Nuwan Jayawardene <ns...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi All,
>>>>> I’m currently working on a project containerizing Synapse, taking it
>>>>> into a cloud-native form.
>>>>> The work done so far is as follows;
>>>>>
>>>>> - Dockerize Synapse using Dockerfile
>>>>> - Get Synapse to run within Kubernetes environment in a deployment
>>>>> - Setup a Kubernetes cluster in the Ambassador pattern with
>>>>> Synapse acting as a proxy between a dockerized client and server (for
>>>>> testing purposes).
>>>>>
>>>>> Moving forward I’m working on developing the following;
>>>>>
>>>>> 1. Inject configuration parameters to Synapse at runtime
>>>>> This would include parameters such as URI endpoints
>>>>> 2. Reduce Synapse image size by making runtime more lean.
>>>>> By measuring performance metrics and reducing
>>>>> legacy/redundant/superfluous components
>>>>>
>>>>> Would greatly appreciate your thoughts and feedback on this.
>>>>> Regards
>>>>>
>>>>> --
>>>>> *Nuwan Jayawardene*
>>>>> Assistant Lecturer, Informatics Institute of Technology (IIT).
>>>>> Research Assistant, University of Moratuwa
>>>>> Tel - +94 71 181 3299
>>>>> https://www.linkedin.com/m/in/nuwanjaya
>>>>>
>>>>>
>>>>> [image: image.gif]
>>>>> [image: image.gif]
>>>>>
>>>>
>>>>
>>>> --
>>>> *Nuwan Jayawardene*
>>>> Assistant Lecturer, Informatics Institute of Technology (IIT).
>>>> Research Assistant, University of Moratuwa
>>>> Tel - +94 71 181 3299
>>>> https://www.linkedin.com/m/in/nuwanjaya
>>>>
>>>>
>>>
>>> --
>>> *Nuwan Jayawardene*
>>> Assistant Lecturer, Informatics Institute of Technology (IIT).
>>> Research Assistant, University of Moratuwa
>>> Tel - +94 71 181 3299
>>> https://www.linkedin.com/m/in/nuwanjaya
>>>
>>>
>>
>> --
>> *Nuwan Jayawardene*
>> Assistant Lecturer, Informatics Institute of Technology (IIT).
>> Research Assistant, University of Moratuwa
>> Tel - +94 71 181 3299
>> https://www.linkedin.com/m/in/nuwanjaya
>>
>>
>
> --
> *Nuwan Jayawardene*
> Assistant Lecturer, Informatics Institute of Technology (IIT).
> Research Assistant, University of Moratuwa
>
> https://www.linkedin.com/m/in/nuwanjaya
>
>
--
*Nuwan Jayawardene*
Assistant Lecturer, Informatics Institute of Technology (IIT).
Research Assistant, University of Moratuwa
https://www.linkedin.com/m/in/nuwanjaya