You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by GitBox <gi...@apache.org> on 2021/04/30 03:53:30 UTC
[GitHub] [apisix-docker] izayoi-akira opened a new issue #167: How to migrate data？
izayoi-akira opened a new issue #167:
URL: https://github.com/apache/apisix-docker/issues/167
Some data on the local machine (/etcd_data dir)
Now I want to migrate the data to the test machine,I tried including directly copying /etcd_data directory, copying /member/snap/db file, Snapshot save and Snapshot restore, but none of them work
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-docker] izayoi-akira commented on issue #167: How to migrate data？
Posted by GitBox <gi...@apache.org>.
izayoi-akira commented on issue #167:
URL: https://github.com/apache/apisix-docker/issues/167#issuecomment-829786446
> @izayoi-akira please add some details about your snapshot operations. Indeed this is a right way
local server etcd_conf:
```
etcd:
volumes:
- ./etcd_data:/etcd_data
```
which be configured by /example/docker-compose.yml
then I run :
`docker exec -it docker-apisix_etcd_1 /bin/bash`
`etcdctl --endpoints localhost:2379 snapshot save /etcd_data/snapshot.db`
it works and I download the snapshot.db file from Host Machine.
later I use etcdctl snapshot restore like this:
`etcdctl snapshot restore snapshot.db --data-dir=/etcd_data`
it error with: Error: data-dir "/etcd_data" exists
then change the data-dir:
`etcdctl snapshot restore snapshot.db --data-dir=/etcd_data_new`
error with: Error: mkdir /etcd_data_new: permission denied
> > > > @izayoi-akira please add some details about your snapshot operations. Indeed this is a right way
> > >
> > >
> > > @tokers how about we introduce the `Terraform` which is for declarative configurations management. Thus we can export all the existing resources into several `.tf` files that represent the declarative resources. Also, we can import it via uploading files onto Dashboard or just run `terraform apply`. This will give us full-fledged functionality for user data import/export.
> > > Looking forward to your feedback.
> >
> >
> > Frankly, I'm not sure, as Terraform just likes a tool used on Cloud. Is it also can be used in bare-metal or containers deployed on-premise?
>
> @tokers
> yep, the most commonly used scenarios of Terraform are for cloud provisioning. While as a matter of fact, Terraform is a great tool that can handle all the cases as long as Declarative Configuration is needed. For example, we can also find the Github/Gitlab providers for the repository/branch/privilege/user management here [[1](https://registry.terraform.io/browse/providers)].
>
> With regards to APISIX, a minimum feasible Terraform `.tf` configurations may seems like this:
>
> ```terraform
> provider "apisix" {
> # Assume we use Admin API here
> url = "https://your-apisix-server-addr:9080/"
> access_user = "admin"
> access_key = "the-access-key-for-admin-api"
> }
>
> resource "apisix_route" "route_1" {
> name = "r1"
> desc = "for buzi1"
> uri = "/foo/bar"
> host = "foo.com"
> methods = ["GET", "HOST"]
> status = 1
> labels = {
> "k1": "v1",
> "k2": "v2"
> }
> upstream_id = ${apisix_upstream.upstream_1.id}
> }
>
> resource "apisix_upstream" "upstream_1" {
> name = "u1"
> type = "chash"
> key = "remote_addr"
> nodes = {
> "127.0.0.1:80": 1,
> "foo.com:80": 2
> }
> }
> ```
>
> This would bring us the reproducible APISIX resource definitions. For backup/restore, we need to support export these definitions as the format of Terraform templates stored in `.tf` files.
>
> Please let me know what do you think. Maybe we all need much more discussion. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-docker] github-actions[bot] commented on issue #167: How to migrate data？
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #167:
URL: https://github.com/apache/apisix-docker/issues/167#issuecomment-957294861
This issue has been marked as stale due to 30 days of inactivity. It will be closed in 2 weeks if no further activity occurs. If this issue is still relevant, please simply write any comment. Even if closed, you can still revive the issue at any time or discuss it on the dev@apisix.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-docker] github-actions[bot] commented on issue #167: How to migrate data？
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #167:
URL: https://github.com/apache/apisix-docker/issues/167#issuecomment-957294861
This issue has been marked as stale due to 30 days of inactivity. It will be closed in 2 weeks if no further activity occurs. If this issue is still relevant, please simply write any comment. Even if closed, you can still revive the issue at any time or discuss it on the dev@apisix.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-docker] tokers commented on issue #167: How to migrate data？
Posted by GitBox <gi...@apache.org>.
tokers commented on issue #167:
URL: https://github.com/apache/apisix-docker/issues/167#issuecomment-829952106
> > > > @izayoi-akira please add some details about your snapshot operations. Indeed this is a right way
> > >
> > >
> > > @tokers how about we introduce the `Terraform` which is for declarative configurations management. Thus we can export all the existing resources into several `.tf` files that represent the declarative resources. Also, we can import it via uploading files onto Dashboard or just run `terraform apply`. This will give us full-fledged functionality for user data import/export.
> > > Looking forward to your feedback.
> >
> >
> > Frankly, I'm not sure, as Terraform just likes a tool used on Cloud. Is it also can be used in bare-metal or containers deployed on-premise?
>
> @tokers
> yep, the most commonly used scenarios of Terraform are for cloud provisioning. While as a matter of fact, Terraform is a great tool that can handle all the cases as long as Declarative Configuration is needed. For example, we can also find the Github/Gitlab providers for the repository/branch/privilege/user management here [[1](https://registry.terraform.io/browse/providers)].
>
> With regards to APISIX, a minimum feasible Terraform `.tf` configurations may seems like this:
>
> ```terraform
> provider "apisix" {
> # Assume we use Admin API here
> url = "https://your-apisix-server-addr:9080/"
> access_user = "admin"
> access_key = "the-access-key-for-admin-api"
> }
>
> resource "apisix_route" "route_1" {
> name = "r1"
> desc = "for buzi1"
> uri = "/foo/bar"
> host = "foo.com"
> methods = ["GET", "HOST"]
> status = 1
> labels = {
> "k1": "v1",
> "k2": "v2"
> }
> upstream_id = ${apisix_upstream.upstream_1.id}
> }
>
> resource "apisix_upstream" "upstream_1" {
> name = "u1"
> type = "chash"
> key = "remote_addr"
> nodes = {
> "127.0.0.1:80": 1,
> "foo.com:80": 2
> }
> }
> ```
>
> This would bring us the reproducible APISIX resource definitions. For backup/restore, we need to support export these definitions as the format of Terraform templates stored in `.tf` files.
>
> Please let me know what do you think. Maybe we all need much more discussion. Thanks.
Sounds cool, would you like to send a proposal to the mailing list, and let's talk with more and more friends in community.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-docker] izayoi-akira closed issue #167: How to migrate data？
Posted by GitBox <gi...@apache.org>.
izayoi-akira closed issue #167:
URL: https://github.com/apache/apisix-docker/issues/167
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-docker] imjoey commented on issue #167: How to migrate data？
Posted by GitBox <gi...@apache.org>.
imjoey commented on issue #167:
URL: https://github.com/apache/apisix-docker/issues/167#issuecomment-829766880
> > > @izayoi-akira please add some details about your snapshot operations. Indeed this is a right way
> >
> >
> > @tokers how about we introduce the `Terraform` which is for declarative configurations management. Thus we can export all the existing resources into several `.tf` files that represent the declarative resources. Also, we can import it via uploading files onto Dashboard or just run `terraform apply`. This will give us full-fledged functionality for user data import/export.
> > Looking forward to your feedback.
>
> Frankly, I'm not sure, as Terraform just likes a tool used on Cloud. Is it also can be used in bare-metal or containers deployed on-premise?
@tokers
yep, the most commonly used scenarios of Terraform are for cloud provisioning. While as a matter of fact, Terraform is a great tool that can handle all the cases as long as Declarative Configuration is needed. For example, we can also find the Github/Gitlab providers for the repository/branch/privilege/user management here [[1]].
With regards to APISIX, a minimum feasible Terraform `.tf` configurations may seems like this:
```terraform
provider "apisix" {
# Assume we use Admin API here
url = "https://your-apisix-server-addr:9080/"
adminkey = "Password"
}
resource "apisix_route" "route_1" {
name = "r1"
desc = "for buzi1"
uri = "/foo/bar"
host = "foo.com"
methods = ["GET", "HOST"]
status = 1
labels = {
"k1": "v1",
"k2": "v2"
}
upstream_id = ${apisix_upstream.upstream_1.id}
}
resource "apisix_upstream" "upstream_1" {
name = "u1"
type = "chash"
key = "remote_addr"
nodes = {
"127.0.0.1:80": 1,
"foo.com:80": 2
}
}
```
This would bring us the reproducible APISIX resource definitions. For backup/restore, we need to support export these definitions as the format of Terraform templates stored in `.tf` files.
Please let me know what do you think. Maybe we all need much more discussion. Thanks.
[1]: https://registry.terraform.io/browse/providers
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-docker] tokers commented on issue #167: How to migrate data？
Posted by GitBox <gi...@apache.org>.
tokers commented on issue #167:
URL: https://github.com/apache/apisix-docker/issues/167#issuecomment-829733568
> > @izayoi-akira please add some details about your snapshot operations. Indeed this is a right way
>
> @tokers how about we introduce the `Terraform` which is for declarative configurations management. Thus we can export all the existing resources into several `.tf` files that represent the declarative resources. Also, we can import it via uploading files onto Dashboard or just run `terraform apply`. This will give us full-fledged functionality for user data import/export.
>
> Looking forward to your feedback.
Frankly, I'm not sure, as Terraform just likes a tool used on Cloud. Is it also can be used in bare-metal or containers deployed on-premise?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-docker] imjoey edited a comment on issue #167: How to migrate data？
Posted by GitBox <gi...@apache.org>.
imjoey edited a comment on issue #167:
URL: https://github.com/apache/apisix-docker/issues/167#issuecomment-829766880
> > > @izayoi-akira please add some details about your snapshot operations. Indeed this is a right way
> >
> >
> > @tokers how about we introduce the `Terraform` which is for declarative configurations management. Thus we can export all the existing resources into several `.tf` files that represent the declarative resources. Also, we can import it via uploading files onto Dashboard or just run `terraform apply`. This will give us full-fledged functionality for user data import/export.
> > Looking forward to your feedback.
>
> Frankly, I'm not sure, as Terraform just likes a tool used on Cloud. Is it also can be used in bare-metal or containers deployed on-premise?
@tokers
yep, the most commonly used scenarios of Terraform are for cloud provisioning. While as a matter of fact, Terraform is a great tool that can handle all the cases as long as Declarative Configuration is needed. For example, we can also find the Github/Gitlab providers for the repository/branch/privilege/user management here [[1]].
With regards to APISIX, a minimum feasible Terraform `.tf` configurations may seems like this:
```terraform
provider "apisix" {
# Assume we use Admin API here
url = "https://your-apisix-server-addr:9080/"
access_user = "admin"
access_key = "the-access-key-for-admin-api"
}
resource "apisix_route" "route_1" {
name = "r1"
desc = "for buzi1"
uri = "/foo/bar"
host = "foo.com"
methods = ["GET", "HOST"]
status = 1
labels = {
"k1": "v1",
"k2": "v2"
}
upstream_id = ${apisix_upstream.upstream_1.id}
}
resource "apisix_upstream" "upstream_1" {
name = "u1"
type = "chash"
key = "remote_addr"
nodes = {
"127.0.0.1:80": 1,
"foo.com:80": 2
}
}
```
This would bring us the reproducible APISIX resource definitions. For backup/restore, we need to support export these definitions as the format of Terraform templates stored in `.tf` files.
Please let me know what do you think. Maybe we all need much more discussion. Thanks.
[1]: https://registry.terraform.io/browse/providers
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-docker] github-actions[bot] commented on issue #167: How to migrate data？
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #167:
URL: https://github.com/apache/apisix-docker/issues/167#issuecomment-957294861
This issue has been marked as stale due to 30 days of inactivity. It will be closed in 2 weeks if no further activity occurs. If this issue is still relevant, please simply write any comment. Even if closed, you can still revive the issue at any time or discuss it on the dev@apisix.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-docker] github-actions[bot] commented on issue #167: How to migrate data？
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #167:
URL: https://github.com/apache/apisix-docker/issues/167#issuecomment-971426394
This issue has been closed due to lack of activity. If you think that is incorrect, or the issue requires additional review, you can revive the issue at any time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-docker] tokers commented on issue #167: How to migrate data？
Posted by GitBox <gi...@apache.org>.
tokers commented on issue #167:
URL: https://github.com/apache/apisix-docker/issues/167#issuecomment-829043739
@izayoi-akira please add some details about your snapshot operations. Indeed this is a right way
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-docker] izayoi-akira commented on issue #167: How to migrate data？
Posted by GitBox <gi...@apache.org>.
izayoi-akira commented on issue #167:
URL: https://github.com/apache/apisix-docker/issues/167#issuecomment-829785799
> @izayoi-akira please add some details about your snapshot operations. Indeed this is a right way
local server etcd_conf:
```
etcd:
volumes:
- ./etcd_data:/etcd_data
```
which be configured by /example/docker-compose.yml
then I run :
`docker exec -it docker-apisix_etcd_1 /bin/bash`
`etcdctl --endpoints localhost:2379 snapshot save /etcd_data/snapshot.db`
it works and I download the snapshot.db file from Host Machine.
later I use etcdctl snapshot restore like this:
`etcdctl snapshot restore snapshot.db --data-dir=/etcd_data`
it error with: Error: data-dir "/etcd_data" exists
then change the data-dir:
`etcdctl snapshot restore snapshot.db --data-dir=/etcd_data_new`
error with: Error: mkdir /etcd_data_new: permission denied
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-docker] imjoey commented on issue #167: How to migrate data？
Posted by GitBox <gi...@apache.org>.
imjoey commented on issue #167:
URL: https://github.com/apache/apisix-docker/issues/167#issuecomment-833210209
> > > > > @izayoi-akira please add some details about your snapshot operations. Indeed this is a right way
> > > >
> > > >
> > > > @tokers how about we introduce the `Terraform` which is for declarative configurations management. Thus we can export all the existing resources into several `.tf` files that represent the declarative resources. Also, we can import it via uploading files onto Dashboard or just run `terraform apply`. This will give us full-fledged functionality for user data import/export.
> > > > Looking forward to your feedback.
> > >
> > >
> > > Frankly, I'm not sure, as Terraform just likes a tool used on Cloud. Is it also can be used in bare-metal or containers deployed on-premise?
> >
> >
> > @tokers
> > yep, the most commonly used scenarios of Terraform are for cloud provisioning. While as a matter of fact, Terraform is a great tool that can handle all the cases as long as Declarative Configuration is needed. For example, we can also find the Github/Gitlab providers for the repository/branch/privilege/user management here [[1](https://registry.terraform.io/browse/providers)].
> > With regards to APISIX, a minimum feasible Terraform `.tf` configurations may seems like this:
> > ```terraform
> > provider "apisix" {
> > # Assume we use Admin API here
> > url = "https://your-apisix-server-addr:9080/"
> > access_user = "admin"
> > access_key = "the-access-key-for-admin-api"
> > }
> >
> > resource "apisix_route" "route_1" {
> > name = "r1"
> > desc = "for buzi1"
> > uri = "/foo/bar"
> > host = "foo.com"
> > methods = ["GET", "HOST"]
> > status = 1
> > labels = {
> > "k1": "v1",
> > "k2": "v2"
> > }
> > upstream_id = ${apisix_upstream.upstream_1.id}
> > }
> >
> > resource "apisix_upstream" "upstream_1" {
> > name = "u1"
> > type = "chash"
> > key = "remote_addr"
> > nodes = {
> > "127.0.0.1:80": 1,
> > "foo.com:80": 2
> > }
> > }
> > ```
> >
> >
> > This would bring us the reproducible APISIX resource definitions. For backup/restore, we need to support export these definitions as the format of Terraform templates stored in `.tf` files.
> > Please let me know what do you think. Maybe we all need much more discussion. Thanks.
>
> Sounds cool, would you like to send a proposal to the mailing list, and let's talk with more and more friends in community.
@tokers Really sorry for my late reply. I will send the proposal to mailing list for further disuccsions. Thanks for the guidance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-docker] imjoey commented on issue #167: How to migrate data？
Posted by GitBox <gi...@apache.org>.
imjoey commented on issue #167:
URL: https://github.com/apache/apisix-docker/issues/167#issuecomment-829308222
> @izayoi-akira please add some details about your snapshot operations. Indeed this is a right way
@tokers how about we introduce the `Terraform` which is for declarative configurations management. Thus we can export all the existing resources into several `.tf` files that represent the declarative resources. Also, we can import it via uploading files onto Dashboard or just run `terraform apply`. This will give us full-fledged functionality for user data import/export.
Looking forward to your feedback.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-docker] izayoi-akira removed a comment on issue #167: How to migrate data？
Posted by GitBox <gi...@apache.org>.
izayoi-akira removed a comment on issue #167:
URL: https://github.com/apache/apisix-docker/issues/167#issuecomment-829786446
> @izayoi-akira please add some details about your snapshot operations. Indeed this is a right way
local server etcd_conf:
```
etcd:
volumes:
- ./etcd_data:/etcd_data
```
which be configured by /example/docker-compose.yml
then I run :
`docker exec -it docker-apisix_etcd_1 /bin/bash`
`etcdctl --endpoints localhost:2379 snapshot save /etcd_data/snapshot.db`
it works and I download the snapshot.db file from Host Machine.
later I use etcdctl snapshot restore like this:
`etcdctl snapshot restore snapshot.db --data-dir=/etcd_data`
it error with: Error: data-dir "/etcd_data" exists
then change the data-dir:
`etcdctl snapshot restore snapshot.db --data-dir=/etcd_data_new`
error with: Error: mkdir /etcd_data_new: permission denied
> > > > @izayoi-akira please add some details about your snapshot operations. Indeed this is a right way
> > >
> > >
> > > @tokers how about we introduce the `Terraform` which is for declarative configurations management. Thus we can export all the existing resources into several `.tf` files that represent the declarative resources. Also, we can import it via uploading files onto Dashboard or just run `terraform apply`. This will give us full-fledged functionality for user data import/export.
> > > Looking forward to your feedback.
> >
> >
> > Frankly, I'm not sure, as Terraform just likes a tool used on Cloud. Is it also can be used in bare-metal or containers deployed on-premise?
>
> @tokers
> yep, the most commonly used scenarios of Terraform are for cloud provisioning. While as a matter of fact, Terraform is a great tool that can handle all the cases as long as Declarative Configuration is needed. For example, we can also find the Github/Gitlab providers for the repository/branch/privilege/user management here [[1](https://registry.terraform.io/browse/providers)].
>
> With regards to APISIX, a minimum feasible Terraform `.tf` configurations may seems like this:
>
> ```terraform
> provider "apisix" {
> # Assume we use Admin API here
> url = "https://your-apisix-server-addr:9080/"
> access_user = "admin"
> access_key = "the-access-key-for-admin-api"
> }
>
> resource "apisix_route" "route_1" {
> name = "r1"
> desc = "for buzi1"
> uri = "/foo/bar"
> host = "foo.com"
> methods = ["GET", "HOST"]
> status = 1
> labels = {
> "k1": "v1",
> "k2": "v2"
> }
> upstream_id = ${apisix_upstream.upstream_1.id}
> }
>
> resource "apisix_upstream" "upstream_1" {
> name = "u1"
> type = "chash"
> key = "remote_addr"
> nodes = {
> "127.0.0.1:80": 1,
> "foo.com:80": 2
> }
> }
> ```
>
> This would bring us the reproducible APISIX resource definitions. For backup/restore, we need to support export these definitions as the format of Terraform templates stored in `.tf` files.
>
> Please let me know what do you think. Maybe we all need much more discussion. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix-docker] github-actions[bot] closed issue #167: How to migrate data？
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed issue #167:
URL: https://github.com/apache/apisix-docker/issues/167
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org