You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by zh...@apache.org on 2022/12/19 09:13:45 UTC
[dolphinscheduler-website] branch master updated: Add blogs (#862)
This is an automated email from the ASF dual-hosted git repository.
zhongjiajie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler-website.git
The following commit(s) were added to refs/heads/master by this push:
new d7b445abb2 Add blogs (#862)
d7b445abb2 is described below
commit d7b445abb2ab896ca909f712c0b5a59fae454ea3
Author: lifeng <53...@users.noreply.github.com>
AuthorDate: Mon Dec 19 17:13:39 2022 +0800
Add blogs (#862)
* Update Best_Practice_DolphinScheduler_Python_API_CI-CD.md
Co-authored-by: Jay Chung <zh...@gmail.com>
---
blog/en-us/Apache_dolphinScheduler_3.0.3.md | 30 +++
...t_Practice_DolphinScheduler_Python_API_CI-CD.md | 253 +++++++++++++++++++++
...nches_on_the_AWS_AMI_Application_Marketplace.md | 141 ++++++++++++
blog/img/media/16714202226727/16714203387608.jpg | Bin 0 -> 84471 bytes
blog/img/media/16714202226727/16714204071793.jpg | Bin 0 -> 78473 bytes
blog/img/media/16714207955368/16714208087579.jpg | Bin 0 -> 87296 bytes
blog/img/media/16714207955368/16714208965363.jpg | Bin 0 -> 36188 bytes
blog/img/media/16714207955368/16714209112764.jpg | Bin 0 -> 33619 bytes
blog/img/media/16714207955368/16714209218058.jpg | Bin 0 -> 67620 bytes
blog/img/media/16714207955368/16714209322012.jpg | Bin 0 -> 40802 bytes
blog/img/media/16714207955368/16714209468073.jpg | Bin 0 -> 26522 bytes
blog/img/media/16714207955368/16714210083690.jpg | Bin 0 -> 126801 bytes
config/blog/en-us/release.json | 27 ++-
config/blog/en-us/tech.json | 18 +-
14 files changed, 456 insertions(+), 13 deletions(-)
diff --git a/blog/en-us/Apache_dolphinScheduler_3.0.3.md b/blog/en-us/Apache_dolphinScheduler_3.0.3.md
new file mode 100644
index 0000000000..19b258eda3
--- /dev/null
+++ b/blog/en-us/Apache_dolphinScheduler_3.0.3.md
@@ -0,0 +1,30 @@
+---
+title: DolphinScheduler released version 3.0.3, focusing on fixing 6 bugs
+keywords: Apache,DolphinScheduler,scheduler,big data,ETL,airflow,hadoop,orchestration,dataops,Kubernetes
+description: Recently, Apache DolphinScheduler released version 3.0.3.
+---
+# DolphinScheduler released version 3.0.3, focusing on fixing 6 bugs
+
+
+Recently, Apache DolphinScheduler released version 3.0.3. This version mainly fixes bugs based on version 3.0.2, with a total of 6 bugs fixed and 3 document modifications. Important bug fixes include: Fix timing scheduling trigger get command parameter null pointer exception #12419 Fix the problem that the scheduled task environment variable does not take effect #12955 Change default unix shell from sh to bash #12180 Malformed LDAP configuration file The full Changelog is as follows:
+
+# Bug Fixes
+* [Bug-12954] [Schedule] Fix the problem that workflow level configuration information does not take effect when timing trigger execution (#12955)
+* [Fix][UI] Download resource returns 401 (#12566)
+* Make sure all failed tasks will be saved in errorTaskMap (#12424)
+* Fix timing scheduling triggering main service report to get command parameter null pointer exception (#12419)
+* Changed default unix shell executor from sh to bash (#12180)
+* Fix LDAP error (#12929)
+
+# Document
+* [FEATURE-8030] [docs] Add Sqoop task documentation (#12855)
+* [Documentation fix] Add Slack Alert documentation (#12567)
+* [Optimization][Documentation] Update the structure and process startup process pictures (#12912)
+
+# Release Notes
+[https://github.com/apache/dolphinscheduler/releases/tag/3.0.3](https://github.com/apache/dolphinscheduler/releases/tag/3.0.3)
+
+# Acknowledgment
+Thanks to the following community contributors for their help in this version release👇👇💕💕
+
+zhuangchong, ntfs32, DarkAssassinator, simsicon, liqingwang, baihongbin, Tianqi-Dotes, tracehh Participate in contribution
\ No newline at end of file
diff --git a/blog/en-us/Best_Practice_DolphinScheduler_Python_API_CI-CD.md b/blog/en-us/Best_Practice_DolphinScheduler_Python_API_CI-CD.md
new file mode 100644
index 0000000000..fb2e579c3d
--- /dev/null
+++ b/blog/en-us/Best_Practice_DolphinScheduler_Python_API_CI-CD.md
@@ -0,0 +1,253 @@
+---
+title: Best Practice | DolphinScheduler Python API CI/CD
+keywords: Apache,DolphinScheduler,scheduler,big data,ETL,airflow,hadoop,orchestration,dataops,Kubernetes
+description: Apache DolphinScheduler is a distributed and extensible workflow scheduler platform with
+---
+# Best Practice | DolphinScheduler Python API CI/CD
+
+
+Written by Zhong Jiajie, Apache DolphinScheduler PMC
+
+# DolphinScheduler and Python API Introduction
+Apache DolphinScheduler is a distributed and extensible workflow scheduler platform with powerful DAG visual interfaces. It helps users easier to build and maintain workflow on any scale.
+
+In order to meet the needs of all users that have different preferences in the same team, DolphinScheduler provides a variety of ways to create workflows. The most popular way is through the web UI, which creates workflows with simple drag and drop and allows non-engineers to create. If you are an engineer and prefer to define workflows programmatically, you may consider using its Python API or YAML file definition to create it.
+
+PyDolphinScheduler is a Python API for Apache DolphinScheduler, which allows you to define your workflow by Python code, aka workflow-as-codes. You can write python code in any editor you like, just like using other python libraries, to create DolphinScheduler’s users, environment, project, and workflow. For more practice examples, you can refer to [[DolphinScheduler can schedule workflows with Python scripts!](https://medium.com/codex/dolphinscheduler-can-schedule-workflows-with-python- [...]
+
+# A Simple Python API Example
+As an out-of-box tool, Python API has an example named tutorial, which includes the basic concept and minimum codes to create and run our very first workflows, you can see more detailed code at https://github.com/apache/dolphinscheduler-sdk-python/blob/main/src/pydolphinscheduler/examples/tutorial.py. The core concept of DolphinScheduler and its Python API is DAG, also calls workflow in Python API. More figuratively, it is the whole picture shown in the following figure. Each DAG contain [...]
+
+# How to Trigger Python API Workflow
+## Single One
+So we already know the basic concept of Python API and we already have an example of it, then how can we trigger it and make it run and get our job done? To make it more pythonic, you can run it just like other Python scripts, via the terminal with a simple command.
+
+```
+python tutorial.py
+```
+The PyDolphinScheduler will help you set all the things and create a new workflow, after that you can see the new one in DolphinScheduler web UI.
+
+## Multiple Workflows
+What if I have multiple files with multiple workflows, how can I trigger them? Yeah, you may already think about it, you can trigger them one by one, just like we trigger the single one. we can be done with
+
+```
+python workflow1.py
+python workflow2.py
+python workflow3.py
+...
+python workflowN.py
+```
+We can add all the above commands into a single bash script, after that, all we need is to execute the bash script
+
+```
+bash <bash-script-contain-all>
+```
+It is useful, but when some workflows add or delete, we have to change the bash script too, which means we must consider changing our code synchronously. Otherwise, the bash script will fail, or some of our new workflows will not be triggered.
+
+The way to fix it is simple, we can dynamic detection the Python script in specific directories, and then pass the existing script to the Python interpreter, and we can change our script like
+
+```
+for file in $(find . -name "*.py"); do
+ python "$file"
+done
+```
+And that is, that is the final script we trigger all our DolphinScheduler Python API workflows on any scale. But it is a little different in reality, I mean that nearly no one deploys production workflows by manually triggering. So the next step we will talk about how to trigger our workflow in CI
+
+# Trigger in GitHub Action
+In this section, we will trigger our workflows via CI, we use GitHub Action as an example and believe other CI tool is almost the same.
+
+# What is GitHub Action
+GitHub Actions makes it easy to automate all your software workflows, now with world-class CI/CD. Build, test, and deploy your code right from GitHub. Make code reviews, branch management, and issue triaging work the way you want. With the popularity of GitHub and the open-source project movement, GitHub Action is very popular currently. You can see more detail in the GitHub Action document. And here is a hello world of GitHub Action:
+```
+name: GitHub Actions Demo
+on:
+ push:
+ branches:
+ - main
+jobs:
+ hello-world:
+ runs-on: ubuntu-latest
+ steps:
+ - name: Run my very first GitHub Actions
+ run: echo "🎉 Hello World."
+```
+You can save it as a YAML file and put it in your project with the path
+
+`.github/workflows/hello.yaml` to tell GitHub what you ask her to do. After that, each time you have a commit and push it to the branch named main, our config file named `hello.yaml` will be trigged, and it will do nothing but one, execute the bash command and echo “🎉 Hello World.” to the GitHub Actions console.
+
+# Combine GitHub Actions to Trigger Multiple Workflows
+In the above example, you may have already realized that GitHub Actions can run bash commands. And our workflows batch trigger script is also a bash script. To get triggered through GitHub Actions, we may change the command in our GitHub Actions example.
+```
+name: Execute Workflows
+on:
+ push:
+ branches:
+ - main
+jobs:
+ execute:
+ runs-on: ubuntu-latest
+ steps:
+ - name: Check out repository code
+ uses: actions/checkout@v3
+ - name: Execute
+ run: |
+ for file in $(find . -name "*.py"); do
+ python "$file"
+ done
+```
+
+It can only trigger and deploy workflows to GitHub Actions running hosts. Our DolphinScheduler cluster runs on the self-host server or could service like AWS instead of GitHub Actions, so we have to tell the bash script to submit our code to the DolphinScheduler cluster instead of the GitHub Actions servers. Fortunately, DolphinScheduler Python API provides a user-friendly configuration change in three ways. I recommend you change it via bash to change environment variables during the Gi [...]
+
+```
+# Modify Java Gateway Address
+export PYDS_JAVA_GATEWAY_ADDRESS="<YOUR-STATIC-IP-RUN-DOLPHINSCHEDULER-API-SERVER>"
+export PYDS_JAVA_GATEWAY_PORT="<PORT-RUN-DOLPHINSCHEDULER-API-SERVER>"
+```
+
+GitHub Actions support `env` syntax in the YAML file, which you can see more detail in github-actions: environment-variables, and we can now change our GitHub Actions config to
+```
+name: Execute Workflows
+on:
+ push:
+ branches:
+ - main
+jobs:
+ execute:
+ runs-on: ubuntu-latest
+ steps:
+ - name: Check out repository code
+ uses: actions/checkout@v3
+ - name: Execute
+ env:
+ PYDS_JAVA_GATEWAY_ADDRESS: <YOUR-STATIC-IP-RUN-DOLPHINSCHEDULER-API-SERVER>
+ PYDS_JAVA_GATEWAY_PORT: <PORT-RUN-DOLPHINSCHEDULER-API-SERVER>
+ run: |
+ for file in $(find . -name "*.py"); do
+ python "$file"
+ done
+```
+Therefore, each time our main branch has new commits, whether it is generated by merging PR or pushed from locally, it will trigger and deploy all of our workflows defined in DolphinScheduler Python API, to where your DolphinScheduler cluster deployed.
+
+In the not released version, we add a new mechanism token for authentication to DolphinScheduler Python API, which means in the next version we have to add a token when we try to connect from Python API to DolphinScheduler, see https://github.com/apache/dolphinscheduler-sdk-python/pull/13 for more detail. Also, we highly recommend our users turn on the token authentication to make our connection safe. Just like other configurations, the token can also change via environment variables by bash。
+
+But how to trigger from GitHub Actions when we enabled and turn on the token? In this case, we have to use GitHub Encrypted secrets for help. You can follow the step motion in the link to create your very first safety secret to your repository and remember the name of your secrets. And then use it in the GitHub Actions config
+```
+name: Execute Workflows
+on:
+ push:
+ branches:
+ - main
+jobs:
+ execute:
+ runs-on: ubuntu-latest
+ steps:
+ - name: Check out repository code
+ uses: actions/checkout@v3
+ - name: Execute
+ env:
+ PYDS_JAVA_GATEWAY_ADDRESS: <YOUR-STATIC-IP-RUN-DOLPHINSCHEDULER-API-SERVER>
+ PYDS_JAVA_GATEWAY_PORT: <PORT-RUN-DOLPHINSCHEDULER-API-SERVER>
+ PYDS_JAVA_GATEWAY_AUTH_TOKEN: ${{ secrets.YOUR-SECRET-NAME }}
+ run: |
+ for file in $(find . -name "*.py"); do
+ python "$file"
+ done
+```
+See？ That is not complex, it is like using ordinary environment variables in GitHub Actions. And that is all we need to do about deploying workflows from GitHub Actions.
+
+# About CI
+
+DolphinScheduler Python API script is a Python script, so it’s CI for Python language which may include black, Pylint, flake8, sort, autoflake and etc. If you choose to use Python API to create and maintain workflow instead of via web UI. I believe you already have personally preferred tools for code format and style check. I decide to talk about CI after then CD because it optional section. If you have your favorite, you can just use it and skip this section, but if you do not have one, [...]
+
+First of all, I prefer using pre-commit, it will run each time when Git is committed, it is useful due to I can detect some easy but often overlooked detail before I push the code to remote. pre-commit needs a config file and I would like to share what Python API uses for itself code-style and lint code, you can see more detail at https://github.com/apache/dolphinscheduler-sdk-python/blob/main/.pre-commit-config.yaml
+
+```default_stages: [commit, push]
+default_language_version:
+ # force all python hooks to run python3
+ python: python3
+repos:
+ # Python API Hooks
+ - repo: https://github.com/pycqa/isort
+ rev: 5.10.1
+ hooks:
+ - id: isort
+ name: isort (python)
+ - repo: https://github.com/psf/black
+ rev: 22.3.0
+ hooks:
+ - id: black
+ - repo: https://github.com/pycqa/flake8
+ rev: 4.0.1
+ hooks:
+ - id: flake8
+ additional_dependencies: [
+ 'flake8-docstrings>=1.6',
+ 'flake8-black>=0.2',
+ ]
+ # pre-commit run in the root, so we have to point out the full path of configuration
+ args: [
+ --config,
+ .flake8
+ ]
+ - repo: https://github.com/pycqa/autoflake
+ rev: v1.4
+ hooks:
+ - id: autoflake
+ args: [
+ --remove-all-unused-imports,
+ --ignore-init-module-imports,
+ --in-place
+ ]
+```
+It does not run complex checks, all of them are easy to know, and to keep pre-commit can be done as soon as possible. The detail is:
+
+* isort: Sort Python imports automatically
+* black: Formatter Python code automatically
+* autoflake: Removes unused imports and unused variables as reported by pyflakes automatically
+* flake8: Detect other code and documentation
+
+And pre-commit is for the local check, you can also run it in your GitHub Actions by adding a new job before the existing job named execute
+
+```name: Execute Workflows
+on:
+ push:
+ branches:
+ - main
+ pull_request:
+jobs:
+ lint:
+ runs-on: ubuntu-latest
+ steps:
+ - name: Check out repository code
+ uses: actions/checkout@v3
+ - name: Install Dependencies
+ run: |
+ python -m pip install --upgrade pre-commit
+ - name: lint
+ run: |
+ pre-commit install
+ pre-commit run --all-files
+ execute:
+ runs-on: ubuntu-latest
+ if: github.event_name == 'push'
+ needs: lint
+ steps:
+ - name: Check out repository code
+ uses: actions/checkout@v3
+ - name: Execute
+ env:
+ PYDS_JAVA_GATEWAY_ADDRESS: <YOUR-STATIC-IP-RUN-DOLPHINSCHEDULER-API-SERVER>
+ PYDS_JAVA_GATEWAY_PORT: <PORT-RUN-DOLPHINSCHEDULER-API-SERVER>
+ PYDS_JAVA_GATEWAY_AUTH_TOKEN: ${{ secrets.YOUR-SECRET-NAME }}
+ run: |
+ for file in $(find . -name "*.py"); do
+ python "$file"
+ done
+```
+
+Some of you may notice that besides adding a new job, we also add `pull_request` node under `on` and `if` node under `execute` job. Because code lint check should test both push and pull requests event, but we only want to execute workflow when there is a new commit to branch main. If we make execute workflow for the pull requests event, each commit to pull requests will be executed and deployed to our production environment, even though the pull requests are not accessed or not prepared [...]
+
+Recap
+* We show what DolphinScheduler and its Python API, and GitHub Actions are, How to create our very first workflow via DolphinScheduler Python API, and the first workflow in GitHub Actions.
+* Then we show how to create DolphinScheduler Python API’s CI/CD base on GitHub Actions step by step.
+* Finally, we create a GitHub Actions to detect code style, and auto lint our DolphinScheduler Python API’s workflow code.
\ No newline at end of file
diff --git a/blog/en-us/DolphinScheduler_launches_on_the_AWS_AMI_Application_Marketplace.md b/blog/en-us/DolphinScheduler_launches_on_the_AWS_AMI_Application_Marketplace.md
new file mode 100644
index 0000000000..a29a19d3d6
--- /dev/null
+++ b/blog/en-us/DolphinScheduler_launches_on_the_AWS_AMI_Application_Marketplace.md
@@ -0,0 +1,141 @@
+---
+title: DolphinScheduler launches on the AWS AMI Application Marketplace
+keywords: Apache,DolphinScheduler,scheduler,big data,ETL,airflow,hadoop,orchestration,dataops,Kubernetes
+description: Apache DolphinScheduler has officially launched on the AWS EC2 AMI application marketplace
+---
+# DolphinScheduler launches on the AWS AMI Application Marketplace
+
+
+# Introduction
+
+Apache DolphinScheduler has officially launched on the AWS EC2 AMI application marketplace, which means that if you want to use or experience DolphinScheduler in AWS, you can directly use this EC2 AMI to start the DolphinScheduler service. When started, your EC2 instance will start a DolphinScheduler standalone service, which is a complete DolphinScheduler component and can perform various tasks of DolphinScheduler.
+
+It should be noted that standalone is not used in the production environment, because all its services are in one process, the metadata is stored in memory by default, and the data will no longer exist when the service is terminated. But it is still very useful because we can quickly start a service for experience verification. If you want to start a DolphinScheduler cluster, you can refer to the Launch Cluster chapter of this article. For more AMI-related information, please check https [...]
+
+# Why launch in AWS?
+
+Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform, offering over 200 fully featured services from data centers globally. Millions of customers — including the fastest-growing startups, largest enterprises, and leading government agencies — are using AWS to lower costs, become more agile, and innovate faster.
+
+# What’s EC2?
+
+Amazon Elastic Compute Cloud (Amazon EC2) provides scalable computing capacity in the Amazon Web Services (AWS) Cloud. Using Amazon EC2 eliminates your need to invest in hardware up-front, so you can develop and deploy applications faster. You can use Amazon EC2 to launch as many or as few virtual servers as you need, configure security and networking, and manage storage. Amazon EC2 enables you to scale up or down to handle changes in requirements or spikes in popularity, reducing your n [...]
+
+# What’s EC2 AMI?
+As mentioned above, when starting an EC2 instance, you can specify a mirror, which is Amazon Machine Image (AMI). An Amazon Machine Image (AMI) is a supported and maintained image provided by AWS that provides the information required to launch an instance. You must specify an AMI when you launch an instance. You can launch multiple instances from a single AMI when you require multiple instances with the same configuration. You can use different AMIs to launch instances when you require [...]
+
+* One or more Amazon Elastic Block Store (Amazon EBS) snapshots, or, for instance-store-backed AMIs, a template for the root volume of the instance (for example, an operating system, an application server, and applications).
+* Launch permissions that control which AWS accounts can use the AMI to launch instances.
+* A block device mapping specifies the volumes to attach to the instance when it’s launched.
+
+The AMIs officially provided by AWS are all basic images, which generally only contain operating systems, such as Windows, Linux, and Mac. Another powerful function of AMI is that it allows users to customize AMI based on the base image, which means that users can install any software in the running EC2 instance, and finally take a snapshot of the instance content to define an AMI.
+
+# Usage
+## Launch Single Instance
+
+The way to launch EC2 instance through DolphinScheduler AMI is very simple, just click the launch instance on the EC2 page, then click browse more AMIs, and enter DolphinScheduler in the input box. Then select AMI and fill in the instance type, key pair, network, and click launch instance to start the instance.
+
+
+
+
+
+
+
+
+
+It takes some time to start the instance. We can check the network configuration at this time. We need to make sure that ports 12345 and 22 are enabled, which are the port of the DolphinScheduler service and the ssh port.
+
+
+
+3–5 minutes later, you can access the DolphinScheduler service through the public DNS or public IPv4 of the EC2 instance plus port 12345. The username and password to log in to the DolphinScheduler service are user and the current EC2 instance id respectively.
+
+Note: The password of DolphinScheduler is dynamic, and it will be automatic change after EC2 instance launch to keep your service safe. And you can find it in your EC2 console home. See the AMI container product for more detail about AWS requests for AMI providers.
+## Launch Cluster
+https://github.com/WhaleOps/packer_tmpl/blob/main/aws/ami/dolphinscheduler/README.md#cluster-server
+
+The above tutorial tells us how to start a single instance, so what should we do if we want to start a DolphinScheduler cluster?
+
+## Add New Key Pairs for Cluster
+
+In the next step we will use ssh connect to existing EC2 instances, and currently our cluster.sh script only supports one single key pair. So we need to create a new one and then use it when launching instances. Go to EC2 -> Network & Security -> Key Pairs -> Create Key Pair. Keep it carefully, otherwise, you can not log in to your instance.
+## Add a New Security Group for Cluster
+Go to EC2 -> Network & Security -> Security Groups -> Create Security Group to create a new security group, you should add the following points to Inbound rules for this security group:
+
+* 22: default ssh point
+* 2181: Zookeeper connect point
+* 5432: PostgreSQL connect point
+* 1234: Dolphin Scheduler’s MasterServer point
+* 5678: DolphinScheduler’s WorkerServer point
+* 12345: DolphinScheduler’s web UI point
+
+
+
+
+## Launch Multiple EC2 Instances
+
+Currently, you have to build this AMI by yourself and then launch a new EC2 instance from EC2 -> Images -> AMIs sidebar path, select the AMI you built, and then click Launch instance from AMI bottom, In EC2 -> Instances -> Launch an instance page, you should choose the existing key pair which you created in new key pair for cluster section, it can be found in Key pair -> Select key pair. Also, you should choose the existing security group which you created in the new security group for t [...]
+
+## Get cluster.sh and cluster_env.sh Script
+If you already clone this project, you should go to the directory packer_tmpl/aws/ami/dolphinscheduler/bin, and you can see two scripts named cluster.sh and cluster_env.sh. And if you do not clone this repository from GitHub, you can get those two scripts by the following command
+
+```
+wget https://raw.githubusercontent.com/WhaleOps/pa
+cker_tmpl/main/aws/ami/dolphinscheduler/bin/cluster.sh
+wget https://raw.githubusercontent.com/WhaleOps/packer_tmpl/main/aws/ami/dolphinscheduler/bin/cluster_env.sh
+```
+> NOTE: If your network can not connect to GitHub, the above command will fail with an error log like connecting to raw.githubusercontent.com (raw.githubusercontent.com)|0.0.0.0|:443… failed: Connection refused. You should find out a method to make your network can connect to host raw.githubusercontent.com or download these two scripts from the GitHub website.
+
+## Modify cluster_env.sh Script
+You have to modify your cluster_env.sh script, which includes your key pair and EC2 instance's IPv4 address or IPv4 DNS. For example, we launch 8 EC2 instances, and we want to deploy two master-server, 3 worker-server, an API -server, an alert server, one database, and a zookeeper server, and each instance IPv4 address as below:
+
+* 192.168.1.1: master-server
+* 192.168.1.2: master-server
+* 192.168.1.3: worker-server
+* 192.168.1.4: worker-server
+* 192.168.1.5: worker-server
+* 192.168.1.6: API-server
+* 192.168.1.7: alert-server
+* 192.168.1.8: metadata database (PostgreSQL), zookeeper
+
+We should tell our deploy plan to cluster_env.sh otherwise it will never know how to deploy it(here we only show some necessary changed content without comment)
+
+```
+export ips="192.168.1.1,192.168.1.2,192.168.1.3,192.168.1.4,192.168.1.5,192.168.1.6,192.168.1.7,192.168.1.8"
+
+```
+```
+export masters="192.168.1.1,192.168.1.2"
+export workers="192.168.1.3:default,192.168.1.4:default,192.168.1.5:default"
+export alertServer="192.168.1.6"
+export apiServers="192.168.1.7"
+export DATABASE_SERVER="192.168.1.8"
+export REGISTRY_SERVER="192.168.1.8"
+```
+Should also add your key pair location which you create in new key pair for the cluster, an absolute path is encouraged to use(here we only show some necessary changed content without comment)
+
+```
+# Do not change this if you use this AMI to launch your instance
+export INSTANCE_USER=${INSTANCE_USER:-"ubuntu"}
+# You have to change to your key pair path
+export INSTANCE_KEY_PAIR="/change/to/your/personal/to/key/pair"
+```
+
+## Execute cluster.sh Script
+After modifying cluster_env.sh you can execute the script by command
+
+`
+./cluster.sh start
+`
+The time it takes depends on your network speed, and after it finishes, your EC2 instances will be combined into a DolphinScheduler cluster.
+
+## What Should I do After Execute cluster.sh
+After that, you can log in DolphinScheduler service with user/EC2_DYNAMIC_INSTANCE_ID as the default username/password via instance’s [API-SERVER-Public-IPv4-address]:12345/dolphinscheduler/ui or [API-SERVER-Public-IPv4-DNS]:12345 /dolphinscheduler/ui. For about how to use DolphinScheduler you view the detail in the functions of DolphinScheduler.
+
+## Contributing
+We build AMI through the packer and make it completely open source. We welcome anyone interested in the project to view and contribute. DolphinScheduler AMI source code can be viewed at [ami-dolphinscheduler](https://github.com/WhaleOps/packer_tmpl/blob/main/aws/ami/dolphinscheduler/README.md). For how to contribute code, you can learn how to [contribute](https://github.com/WhaleOps/packer_tmpl/blob/main/aws/ami/dolphinscheduler/README.md#contributing).
+
+## Recap
+
+* Briefly introduce what AWS, EC2, and EC2 AMI are, and how to create an EC2 instance through AMI;
+* Introduces how to use DolphinScheduler AMI, how to start a single instance and form a cluster;
+* Emphasize again that standalone is only for testing and experience
+* How to contribute if you are interested in the project
\ No newline at end of file
diff --git a/blog/img/media/16714202226727/16714203387608.jpg b/blog/img/media/16714202226727/16714203387608.jpg
new file mode 100644
index 0000000000..54012fe90c
Binary files /dev/null and b/blog/img/media/16714202226727/16714203387608.jpg differ
diff --git a/blog/img/media/16714202226727/16714204071793.jpg b/blog/img/media/16714202226727/16714204071793.jpg
new file mode 100644
index 0000000000..88f2c0a0c6
Binary files /dev/null and b/blog/img/media/16714202226727/16714204071793.jpg differ
diff --git a/blog/img/media/16714207955368/16714208087579.jpg b/blog/img/media/16714207955368/16714208087579.jpg
new file mode 100644
index 0000000000..71509655c1
Binary files /dev/null and b/blog/img/media/16714207955368/16714208087579.jpg differ
diff --git a/blog/img/media/16714207955368/16714208965363.jpg b/blog/img/media/16714207955368/16714208965363.jpg
new file mode 100644
index 0000000000..c183c3d0fa
Binary files /dev/null and b/blog/img/media/16714207955368/16714208965363.jpg differ
diff --git a/blog/img/media/16714207955368/16714209112764.jpg b/blog/img/media/16714207955368/16714209112764.jpg
new file mode 100644
index 0000000000..bed13fa2f1
Binary files /dev/null and b/blog/img/media/16714207955368/16714209112764.jpg differ
diff --git a/blog/img/media/16714207955368/16714209218058.jpg b/blog/img/media/16714207955368/16714209218058.jpg
new file mode 100644
index 0000000000..3dec6c26e4
Binary files /dev/null and b/blog/img/media/16714207955368/16714209218058.jpg differ
diff --git a/blog/img/media/16714207955368/16714209322012.jpg b/blog/img/media/16714207955368/16714209322012.jpg
new file mode 100644
index 0000000000..6add809855
Binary files /dev/null and b/blog/img/media/16714207955368/16714209322012.jpg differ
diff --git a/blog/img/media/16714207955368/16714209468073.jpg b/blog/img/media/16714207955368/16714209468073.jpg
new file mode 100644
index 0000000000..e499baac46
Binary files /dev/null and b/blog/img/media/16714207955368/16714209468073.jpg differ
diff --git a/blog/img/media/16714207955368/16714210083690.jpg b/blog/img/media/16714207955368/16714210083690.jpg
new file mode 100644
index 0000000000..10913be897
Binary files /dev/null and b/blog/img/media/16714207955368/16714210083690.jpg differ
diff --git a/config/blog/en-us/release.json b/config/blog/en-us/release.json
index ed165a71a7..604a62bf70 100644
--- a/config/blog/en-us/release.json
+++ b/config/blog/en-us/release.json
@@ -1,61 +1,68 @@
+
{
+ "Apache_DolphinScheduler_3.0.3": {
+ "title": "DolphinScheduler released version 3.0.3, focusing on fixing 6 bugs",
+ "author": "Leonard Nie",
+ "dateStr": "2022-12-01",
+ "desc": "Recently, Apache DolphinScheduler released version 3.0.3....... "
+},
"Apache_DolphinScheduler_2.0.7": {
"title": "Apache DolphinScheduler 2.0.7 released, fixes complement and fault tolerance",
- "author": "Debra Chen",
+ "author": "Leonard Nie",
"dateStr": "2022-10-28",
"desc": "Recently, Apache DolphinScheduler version 2.0.7 was officially released...... "
},
"Apache_dolphinScheduler_3.0.1": {
"title": "Apache DolphinScheduler 3.0.1 released, with the scheduling kernel and UI optimized",
- "author": "Debra Chen",
+ "author": "Leonard Nie",
"dateStr": "2022-9-28",
"desc": "hanks to this Release Manager — @zhuangchong, who led the release of the 3.0.1...... "
},
"Apache_dolphinScheduler_3.0.0": {
"title": "Apache DolphinScheduler 3.0.0 Official Version Released!",
- "author": "Debra Chen",
+ "author": "Leonard Nie",
"dateStr": "2022-9-2",
"desc": "The official version 3.0.0 has undergone the most significant changes since its release... "
},
"Apache_dolphinScheduler_3.0.0_alpha": {
"title": "3.0.0 Alpha Release! Nine New Features and A Brand New UI Unlock New Capabilities For the Scheduling System",
- "author": "Debra Chen",
+ "author": "Leonard Nie",
"dateStr": "2022-4-16",
"desc": "n April 22, 2022, Apache DolphinScheduler officially announced the release of alpha version 3.0.0! ... "
},
"Apache_dolphinScheduler_2.0.5": {
"title": "Release News! Apache DolphinScheduler 2_0_5 optimizes The Fault Tolerance Process of Worker",
- "author": "Debra Chen",
+ "author": "Leonard Nie",
"dateStr": "2022-3-7",
"desc": "Today, Apache DolphinScheduler announced the official release of version 2.0.5. "
},
"Apache_dolphinScheduler_2.0.3": {
"title": "Apache DolphinScheduler 2.0.3 Release Announcement: DingTalk alert plugin adds signature verification, and supports data sources to obtain links from multiple sessions\n",
- "author": "Debra Chen",
+ "author": "Leonard Nie",
"dateStr": "2022-1-27",
"desc": "Apache DolphinScheduler Board Report"
},
"Apache_dolphinScheduler_2.0.2": {
"title": "Apache DolphinScheduler 2.0.2 Release Announcement:WorkflowAsCode is Launched!",
- "author": "Debra Chen",
+ "author": "Leonard Nie",
"dateStr": "2022-1-13",
"desc": " TWOS"
},
"Apache-DolphinScheduler-2.0.1": {
"title": "Apache DolphinScheduler 2.0.1 is here, and the highly anticipated one-click upgrade and plug-in finally come!",
- "author": "Debra Chen",
+ "author": "Leonard Nie",
"dateStr": "2021-12-20",
"desc": "Apache DolphinScheduler 2.0.1"
},
"DS-2.0-alpha-release": {
"title": "Refactoring, Plug-in, Performance Improves By 20 times, Apache DolphinScheduler 2.0 alpha Release Highlights Check!",
- "author": "Debra Chen",
+ "author": "Leonard Nie",
"dateStr": "2021-10-29",
"desc": "DolphinScheduler 2.0 alpha release"
},
"Introducing-Apache-DolphinScheduler-1.3.9": {
"title": "Introducing Apache DolphinScheduler 1.3.9, StandaloneServer is Available!",
- "author": "Debar Chen",
+ "author": "Leonard Nie",
"dateStr": "2021-10-22",
"desc": "Introducing Apache DolphinScheduler 1.3.9, StandaloneServer is Available!"
}
diff --git a/config/blog/en-us/tech.json b/config/blog/en-us/tech.json
index e93548c193..a56962d585 100644
--- a/config/blog/en-us/tech.json
+++ b/config/blog/en-us/tech.json
@@ -1,19 +1,31 @@
{
+ "DolphinScheduler Python API CI/CD": {
+ "title": "DolphinScheduler Python API CI/CD",
+ "author": "Leonard Nie",
+ "dateStr": "2022-12-16",
+ "desc": "DolphinScheduler Python API CI/CD... "
+ },
+ "DolphinScheduler launches on the AWS AMI Application Marketplace": {
+ "title": "DolphinScheduler launches on the AWS AMI Application Marketplace",
+ "author": "Leonard Nie",
+ "dateStr": "2022-12-10",
+ "desc": "Apache DolphinScheduler has officially launched on the AWS EC2 AMI application marketplace... "
+ },
"How_can_more_people_benefit_from_big_data": {
"title": "How can more people benefit from big data?",
- "author": "Debra Chen",
+ "author": "Leonard Nie",
"dateStr": "2022-10-29",
"desc": "During the ApacheCon Asia 2022, Chen Wei, who has more than 10 years... "
},
"Python_API_and_AWS_Support": {
"title": "apache DolphinScheduler Extends Capabilities Through Python API and AWS Support",
- "author": "Debra Chen",
+ "author": "Leonard Nie",
"dateStr": "2022-8-26",
"desc": "In the ever-changing world of technology, data is abundant. ... "
},
"DolphinScheduler_Kubernetes_Technology_in_action": {
"title": "Technical Practice of Apache DolphinScheduler in Kubernetes System",
- "author": "Debra Chen",
+ "author": "Leonard Nie",
"dateStr": "2022-2-24",
"desc": "Kubernetes is a cluster system based on container technology, "
},