You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@devlake.apache.org by "warren830 (via GitHub)" <gi...@apache.org> on 2023/03/16 06:12:00 UTC
[GitHub] [incubator-devlake-website] warren830 opened a new pull request, #469: add mysql troubleshooting
warren830 opened a new pull request, #469:
URL: https://github.com/apache/incubator-devlake-website/pull/469
# Summary
This pr show how to delete old binlogs for mysql in order to free size of mysql-storage
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake-website] hezyin commented on a diff in pull request #469: add mysql troubleshooting
Posted by "hezyin (via GitHub)" <gi...@apache.org>.
hezyin commented on code in PR #469:
URL: https://github.com/apache/incubator-devlake-website/pull/469#discussion_r1155144241
##########
docs/Troubleshooting/MySqlSizeIssue.md:
##########
@@ -0,0 +1,84 @@
+---
+title: "Maintenance Troubleshooting"
+sidebar_position: 4
+description: >
+ Maintenance Troubleshooting
+---
+
+## How to manage the quickly increasing MySQL disk consumption?
+
+DevLake is designed to collect data by first deleting the existing data and
Review Comment:
Can we add a sentence explaining why DevLake choose "delete + insert" instead of "update"?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake-website] klesh commented on a diff in pull request #469: add mysql troubleshooting
Posted by "klesh (via GitHub)" <gi...@apache.org>.
klesh commented on code in PR #469:
URL: https://github.com/apache/incubator-devlake-website/pull/469#discussion_r1138179991
##########
docs/Troubleshooting/MySqlSizeIssue.md:
##########
@@ -0,0 +1,36 @@
+---
+title: "MySQL Troubleshooting"
Review Comment:
I think "Maintenance Troubleshooting" would be more appropriate
##########
docs/Troubleshooting/MySqlSizeIssue.md:
##########
@@ -0,0 +1,36 @@
+---
+title: "MySQL Troubleshooting"
+sidebar_position: 4
+description: >
+ MySQL Troubleshooting
+---
+
+### How to delete old MySQL binlog files?
Review Comment:
It would be better to describe the symptom before we address the cause:
"Why the size of the database grows drastically?" sth like that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake-website] klesh commented on a diff in pull request #469: add mysql troubleshooting
Posted by "klesh (via GitHub)" <gi...@apache.org>.
klesh commented on code in PR #469:
URL: https://github.com/apache/incubator-devlake-website/pull/469#discussion_r1155681642
##########
docs/Troubleshooting/MySqlSizeIssue.md:
##########
@@ -0,0 +1,91 @@
+---
+title: "Maintenance Troubleshooting"
+sidebar_position: 4
+description: >
+ Maintenance Troubleshooting
+---
+
+## How to manage the quickly increasing MySQL disk consumption?
+
+DevLake is designed to collect data by first deleting the existing data and
+then inserting new data. While this approach ensures that the latest data is
+always available, it leads to a rapid increase in MySQL disk consumption.
+This growth is primarily caused by the large size of the binary logs generated
+after each data collection cycle.
+
+### Why dose DevLake choose to delete the existing data and then insert new data?
+
+Because we want to ensure that the latest data is
+always available. If we don't delete the existing data, some old data which has been deleted
Review Comment:
should "available" be reworded with "accurate"
##########
docs/Troubleshooting/MySqlSizeIssue.md:
##########
@@ -0,0 +1,91 @@
+---
+title: "Maintenance Troubleshooting"
+sidebar_position: 4
+description: >
+ Maintenance Troubleshooting
+---
+
+## How to manage the quickly increasing MySQL disk consumption?
+
+DevLake is designed to collect data by first deleting the existing data and
+then inserting new data. While this approach ensures that the latest data is
+always available, it leads to a rapid increase in MySQL disk consumption.
+This growth is primarily caused by the large size of the binary logs generated
+after each data collection cycle.
+
+### Why dose DevLake choose to delete the existing data and then insert new data?
+
+Because we want to ensure that the latest data is
+always available. If we don't delete the existing data, some old data which has been deleted
+from the source system will still be available in the DevLake database.
Review Comment:
I think "source system" should be replaced with "previous step/subtask" sth like that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake-website] hezyin commented on a diff in pull request #469: add mysql troubleshooting
Posted by "hezyin (via GitHub)" <gi...@apache.org>.
hezyin commented on code in PR #469:
URL: https://github.com/apache/incubator-devlake-website/pull/469#discussion_r1145579408
##########
docs/Troubleshooting/MySqlSizeIssue.md:
##########
@@ -0,0 +1,36 @@
+---
+title: "Maintenance Troubleshooting"
+sidebar_position: 4
+description: >
+ Maintenance Troubleshooting
+---
+
+### How to delete old MySQL binlog files?
+
+1. Connect to your MySQL server using the MySQL client or any other database management tool such as PhpMyAdmin, MySQL Workbench, etc.
+
+2. Run the following command to check the current status of your binary log files:
+
+```sql
+SHOW BINARY LOGS;
+```
+This will display a list of all the binary log files that are currently available on your MySQL server.
+
+3. Determine the last binary log file that you want to keep. This is the file that you want to retain for any future point-in-time recovery or replication purposes.
+
+4. Run the following command to purge all binary logs that are older than the binary log file that you want to retain:
+
+```sql
+PURGE BINARY LOGS TO 'binary_log_file_name';
+```
+
+Replace binary_log_file_name with the name of the last binary log file that you want to keep.
+
+5. After running the command, MySQL will delete all binary log files that are older than the specified file. You can verify that the purge was successful by running the SHOW BINARY LOGS; command again.
+
+Note: Keep in mind that deleting old binary log files can affect point-in-time recovery and replication capabilities, so it's important to only delete files that are no longer needed.
Review Comment:
Shall we also mention that they can "automate" this process away by setting the `expire_logs_days` in the MySQL configuration file?
##########
docs/Troubleshooting/MySqlSizeIssue.md:
##########
@@ -0,0 +1,36 @@
+---
+title: "Maintenance Troubleshooting"
+sidebar_position: 4
+description: >
+ Maintenance Troubleshooting
+---
+
+### How to delete old MySQL binlog files?
+
+1. Connect to your MySQL server using the MySQL client or any other database management tool such as PhpMyAdmin, MySQL Workbench, etc.
+
+2. Run the following command to check the current status of your binary log files:
+
+```sql
+SHOW BINARY LOGS;
+```
+This will display a list of all the binary log files that are currently available on your MySQL server.
+
+3. Determine the last binary log file that you want to keep. This is the file that you want to retain for any future point-in-time recovery or replication purposes.
+
+4. Run the following command to purge all binary logs that are older than the binary log file that you want to retain:
+
+```sql
+PURGE BINARY LOGS TO 'binary_log_file_name';
Review Comment:
Maybe it's simpler for users to specify a date instead of an actual binlog file?
`PURGE BINARY LOGS BEFORE '<date>' `
##########
docs/Troubleshooting/MySqlSizeIssue.md:
##########
@@ -0,0 +1,36 @@
+---
+title: "MySQL Troubleshooting"
+sidebar_position: 4
+description: >
+ MySQL Troubleshooting
+---
+
+### How to delete old MySQL binlog files?
Review Comment:
@warren830 I second @klesh's suggestion on describing the symptom briefly first. Otherwise, it's not clear to users why they'd like to know how to delete old MySQL binlog files.
My recommended title for this section is "How to manage the quickly increasing MySQL disk consumption?"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake-website] warren830 commented on a diff in pull request #469: add mysql troubleshooting
Posted by "warren830 (via GitHub)" <gi...@apache.org>.
warren830 commented on code in PR #469:
URL: https://github.com/apache/incubator-devlake-website/pull/469#discussion_r1144510508
##########
docs/Troubleshooting/MySqlSizeIssue.md:
##########
@@ -0,0 +1,36 @@
+---
+title: "MySQL Troubleshooting"
+sidebar_position: 4
+description: >
+ MySQL Troubleshooting
+---
+
+### How to delete old MySQL binlog files?
Review Comment:
But actually here is talking about how to resolve
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake-website] warren830 commented on a diff in pull request #469: add mysql troubleshooting
Posted by "warren830 (via GitHub)" <gi...@apache.org>.
warren830 commented on code in PR #469:
URL: https://github.com/apache/incubator-devlake-website/pull/469#discussion_r1145605199
##########
docs/Troubleshooting/MySqlSizeIssue.md:
##########
@@ -0,0 +1,36 @@
+---
+title: "Maintenance Troubleshooting"
+sidebar_position: 4
+description: >
+ Maintenance Troubleshooting
+---
+
+### How to delete old MySQL binlog files?
+
+1. Connect to your MySQL server using the MySQL client or any other database management tool such as PhpMyAdmin, MySQL Workbench, etc.
+
+2. Run the following command to check the current status of your binary log files:
+
+```sql
+SHOW BINARY LOGS;
+```
+This will display a list of all the binary log files that are currently available on your MySQL server.
+
+3. Determine the last binary log file that you want to keep. This is the file that you want to retain for any future point-in-time recovery or replication purposes.
+
+4. Run the following command to purge all binary logs that are older than the binary log file that you want to retain:
+
+```sql
+PURGE BINARY LOGS TO 'binary_log_file_name';
+```
+
+Replace binary_log_file_name with the name of the last binary log file that you want to keep.
+
+5. After running the command, MySQL will delete all binary log files that are older than the specified file. You can verify that the purge was successful by running the SHOW BINARY LOGS; command again.
+
+Note: Keep in mind that deleting old binary log files can affect point-in-time recovery and replication capabilities, so it's important to only delete files that are no longer needed.
Review Comment:
Ok, and we will also add it to docker-compose.yaml
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake-website] hezyin merged pull request #469: add mysql troubleshooting
Posted by "hezyin (via GitHub)" <gi...@apache.org>.
hezyin merged PR #469:
URL: https://github.com/apache/incubator-devlake-website/pull/469
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org