You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2022/04/07 06:17:35 UTC
[GitHub] [shardingsphere] newchendb opened a new issue, #16631: the data is not written to the database in time
newchendb opened a new issue, #16631:
URL: https://github.com/apache/shardingsphere/issues/16631
shardingsphere-proxy version is 5.1.0。
Under what circumstances will the program submit data and return success, but the data is not written to the database in time, sometimes delaying for several seconds.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] jingshanglu commented on issue #16631: the data is not written to the database in time
Posted by GitBox <gi...@apache.org>.
jingshanglu commented on issue #16631:
URL: https://github.com/apache/shardingsphere/issues/16631#issuecomment-1094527004
Can you try this on mysql client?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] jingshanglu commented on issue #16631: the data is not written to the database in time
Posted by GitBox <gi...@apache.org>.
jingshanglu commented on issue #16631:
URL: https://github.com/apache/shardingsphere/issues/16631#issuecomment-1092410858
I see your just insert to one table and select from two table, does the other table have master-slave synchronization delay?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] newchendb commented on issue #16631: the data is not written to the database in time
Posted by GitBox <gi...@apache.org>.
newchendb commented on issue #16631:
URL: https://github.com/apache/shardingsphere/issues/16631#issuecomment-1092403014
@jingshanglu Yes, before that, I thought it was caused by MySQL master-slave synchronization delay, so I changed it to this, but the problem still exists.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] newchendb closed issue #16631: the data is not written to the database in time
Posted by GitBox <gi...@apache.org>.
newchendb closed issue #16631: the data is not written to the database in time
URL: https://github.com/apache/shardingsphere/issues/16631
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] newchendb commented on issue #16631: the data is not written to the database in time
Posted by GitBox <gi...@apache.org>.
newchendb commented on issue #16631:
URL: https://github.com/apache/shardingsphere/issues/16631#issuecomment-1091337565
server.yaml
`mode:
type: Standalone
repository:
type: File
overwrite: true
rules:
- !AUTHORITY
users:
- root@%:123456
provider:
type: ALL_PRIVILEGES_PERMITTED
- !TRANSACTION
defaultType: XA
providerType: Atomikos
props:
proxy-opentracing-enabled: false
proxy-hint-enabled: false
sql-show: false
check-table-metadata-enabled: false
show-process-list-enabled: false
proxy-backend-query-fetch-size: -1
check-duplicate-table-enabled: false
proxy-frontend-executor-size: 0
proxy-backend-executor-suitable: OLAP
proxy-frontend-max-connections: 0
sql-federation-enabled: false
proxy-backend-driver-type: JDBC
allow-range-query-with-inline-sharding: true
sql-simple: false
`
config-demo.yaml
`schemaName: winsky_saas
dataSources:
write_ds_0:
url: jdbc:mysql://10.0.0.15:3306/winsky_saas?useUnicode=true&serverTimezone=Asia/Shanghai&characterEncoding=UTF8&useSSL=false
username: ydzsaasdb
password: 123456
connectionTimeoutMilliseconds: 10000
idleTimeoutMilliseconds: 60000
maxLifetimeMilliseconds: 6000000
maxPoolSize: 20
minPoolSize: 2
read_ds_0:
url: jdbc:mysql://10.0.0.8:3306/winsky_saas?useUnicode=true&serverTimezone=Asia/Shanghai&characterEncoding=UTF8&useSSL=false
username: ydzsaasdb
password: 123456
connectionTimeoutMilliseconds: 10000
idleTimeoutMilliseconds: 60000
maxLifetimeMilliseconds: 6000000
maxPoolSize: 20
minPoolSize: 2
write_ds_1:
url: jdbc:mysql://10.0.0.15:3307/winsky_saas?useUnicode=true&serverTimezone=Asia/Shanghai&characterEncoding=UTF8&useSSL=false
username: ydzsaasdb
password: 123456
connectionTimeoutMilliseconds: 10000
idleTimeoutMilliseconds: 60000
maxLifetimeMilliseconds: 6000000
maxPoolSize: 20
minPoolSize: 2
read_ds_1:
url: jdbc:mysql://10.0.0.8:3307/winsky_saas?useUnicode=true&serverTimezone=Asia/Shanghai&characterEncoding=UTF8&useSSL=false
username: ydzsaasdb
password: 123456
connectionTimeoutMilliseconds: 10000
idleTimeoutMilliseconds: 60000
maxLifetimeMilliseconds: 6000000
maxPoolSize: 20
minPoolSize: 2
rules:
- !SHARDING
tables:
t_charge:
actualDataNodes: pr_ds_${0..1}.t_charge_${2022..2025}0${1..9},pr_ds_${0..1}.t_charge_${2022..2025}1${0..2}
tableStrategy:
standard:
shardingColumn: upload_time
shardingAlgorithmName: s_month_inline
t_charge_info:
actualDataNodes: pr_ds_${0..1}.t_charge_info_${2022..2025}0${1..9},pr_ds_${0..1}.t_charge_info_${2022..2025}1${0..2}
tableStrategy:
standard:
shardingColumn: upload_time
shardingAlgorithmName: s_month_inline
t_charge_unpaid:
actualDataNodes: pr_ds_${0..1}.t_charge_unpaid
t_charge_info_unpaid:
actualDataNodes: pr_ds_${0..1}.t_charge_info_unpaid
t_stats_skgroup_daily:
actualDataNodes: pr_ds_${0..1}.t_stats_skgroup_daily_${2022..2025}0${1..9},pr_ds_${0..1}.t_stats_skgroup_daily_${2022..2025}1${0..2}
tableStrategy:
standard:
shardingColumn: stats_date
shardingAlgorithmName: s_month_inline
t_charge_process:
actualDataNodes: pr_ds_${0..1}.t_charge_process_${2022..2025}0${1..9},pr_ds_${0..1}.t_charge_process_${2022..2025}1${0..2}
tableStrategy:
standard:
shardingColumn: upload_time
shardingAlgorithmName: s_month_inline
t_evcipa_notify_order:
actualDataNodes: pr_ds_${0..1}.t_evcipa_notify_order_${2022..2025}0${1..9},pr_ds_${0..1}.t_evcipa_notify_order_${2022..2025}1${0..2}
tableStrategy:
standard:
shardingColumn: order_upload_time
shardingAlgorithmName: s_month_inline
t_evcipa_notify_order_detail:
actualDataNodes: pr_ds_${0..1}.t_evcipa_notify_order_detail_${2022..2025}0${1..9},pr_ds_${0..1}.t_evcipa_notify_order_detail_${2022..2025}1${0..2}
tableStrategy:
standard:
shardingColumn: occur_time
shardingAlgorithmName: s_month_inline
t_evcipa_notify_checkorder:
actualDataNodes: pr_ds_${0..1}.t_evcipa_notify_checkorder_${2022..2025}
tableStrategy:
standard:
shardingColumn: start_time
shardingAlgorithmName: s_year_inline
undo_log:
actualDataNodes: pr_ds_${0..1}.undo_log
bindingTables:
- t_charge,t_charge_info,t_evcipa_notify_order,t_evcipa_notify_order_detail
- t_charge_unpaid,t_charge_info_unpaid
- t_charge_process
- t_evcipa_notify_checkorder
broadcastTables:
- undo_log
defaultDatabaseStrategy:
standard:
shardingColumn: saas_id
shardingAlgorithmName: database_inline
defaultKeyGenerateStrategy:
keyGeneratorName: snowflake
column: id
shardingAlgorithms:
database_inline:
type: INLINE
props:
algorithm-expression: pr_ds_${saas_id % 2}
s_month_inline:
type: INTERVAL
props:
datetime-interval-unit: MONTHS
datetime-interval-amount: 1
sharding-suffix-pattern: yyyyMM
datetime-pattern: yyyy-MM-dd HH:mm:ss
datetime-lower: "2022-01-01 00:00:00"
datetime-upper: "2025-12-31 23:59:59"
s_year_inline:
type: INTERVAL
props:
datetime-interval-unit: YEARS
datetime-interval-amount: 1
sharding-suffix-pattern: yyyy
datetime-pattern: yyyy-MM-dd HH:mm:ss
datetime-lower: "2022-01-01 00:00:00"
datetime-upper: "2025-12-31 23:59:59"
keyGenerators:
snowflake:
type: SNOWFLAKE
props:
worker-id: 123
- !READWRITE_SPLITTING
dataSources:
pr_ds_0:
type: Static
props:
write-data-source-name: write_ds_0
read-data-source-names: read_ds_0
pr_ds_1:
type: Static
props:
write-data-source-name: write_ds_1
read-data-source-names: read_ds_1
`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] newchendb commented on issue #16631: the data is not written to the database in time
Posted by GitBox <gi...@apache.org>.
newchendb commented on issue #16631:
URL: https://github.com/apache/shardingsphere/issues/16631#issuecomment-1092416662
@jingshanglu No, the associated table is a sub table. This operation does not involve sub tables.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] jingshanglu commented on issue #16631: the data is not written to the database in time
Posted by GitBox <gi...@apache.org>.
jingshanglu commented on issue #16631:
URL: https://github.com/apache/shardingsphere/issues/16631#issuecomment-1092399207
> 
>
> @jingshanglu Very simple SQL. The data is inserted in the above SQL, but sometimes the corresponding data is not found in the following query
This scenario should not lead to write delay. The possible scenario is to write to the master DB and read from the slave DB.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] jingshanglu commented on issue #16631: the data is not written to the database in time
Posted by GitBox <gi...@apache.org>.
jingshanglu commented on issue #16631:
URL: https://github.com/apache/shardingsphere/issues/16631#issuecomment-1091302999
Can you show proxy config? can you show a demo?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] jingshanglu commented on issue #16631: the data is not written to the database in time
Posted by GitBox <gi...@apache.org>.
jingshanglu commented on issue #16631:
URL: https://github.com/apache/shardingsphere/issues/16631#issuecomment-1091613797
Can you show the SQL demo?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] jingshanglu commented on issue #16631: the data is not written to the database in time
Posted by GitBox <gi...@apache.org>.
jingshanglu commented on issue #16631:
URL: https://github.com/apache/shardingsphere/issues/16631#issuecomment-1092468987
OK, If it appears again, you can simplify the demo and post it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] jingshanglu commented on issue #16631: the data is not written to the database in time
Posted by GitBox <gi...@apache.org>.
jingshanglu commented on issue #16631:
URL: https://github.com/apache/shardingsphere/issues/16631#issuecomment-1092399835
The scene in the screenshot is read and written on the same DB.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] newchendb commented on issue #16631: the data is not written to the database in time
Posted by GitBox <gi...@apache.org>.
newchendb commented on issue #16631:
URL: https://github.com/apache/shardingsphere/issues/16631#issuecomment-1092416870
This is accidental
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] newchendb commented on issue #16631: the data is not written to the database in time
Posted by GitBox <gi...@apache.org>.
newchendb commented on issue #16631:
URL: https://github.com/apache/shardingsphere/issues/16631#issuecomment-1092495144
The above is my test code. Query the list after adding
The above is the log of my code. There is no newly added data in the list query
The above is the SQL log。
but it is not always the case.
@jingshanglu
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] newchendb commented on issue #16631: the data is not written to the database in time
Posted by GitBox <gi...@apache.org>.
newchendb commented on issue #16631:
URL: https://github.com/apache/shardingsphere/issues/16631#issuecomment-1091123788
There is no such problem with direct database connection. There will be such a problem with shardingsphere-proxy
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] newchendb commented on issue #16631: the data is not written to the database in time
Posted by GitBox <gi...@apache.org>.
newchendb commented on issue #16631:
URL: https://github.com/apache/shardingsphere/issues/16631#issuecomment-1092373738
@jingshanglu Very simple SQL. The data is inserted in the above SQL, but sometimes the corresponding data is not found in the following query
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] newchendb commented on issue #16631: the data is not written to the database in time
Posted by GitBox <gi...@apache.org>.
newchendb commented on issue #16631:
URL: https://github.com/apache/shardingsphere/issues/16631#issuecomment-1095752983
I have abandoned sharding proxy and used shardingjdbc instead
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org