You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kvrocks.apache.org by GitBox <gi...@apache.org> on 2022/12/14 08:55:13 UTC
[GitHub] [incubator-kvrocks] vhrtanek opened a new issue, #1180: Kvrocks sets record in all open DB instances instead of only in one
vhrtanek opened a new issue, #1180:
URL: https://github.com/apache/incubator-kvrocks/issues/1180
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-kvrocks/issues) and found no similar issues.
### Version
kvrocks:
Kvrocks 2.2.0
OS:
macOS Monterey 12.5.1
Python:
Python 3.7
Python modules:
apache-beam==2.42.0
confluent-kafka==1.9.2
redis==4.3.4
Flink cluster:
Flink 1.14.0
### Minimal reproduce step
I am utilizing apache beam pipeline, which reads data from kafka, does simple transformations, and writes data into kvrocks via redis module. The problem is that the same record is written to all defined connections regardless of different message formats or that some kvrocks connection is commented out e.g. `r_db_hub` like in the example below
Here is simplified py script, reproducibility won't be really possible, but i hope you will get an idea
```
import apache_beam as beam
import redis
import confluent_kafka
#r_db_hub = redis.Redis(host='localhost', port=6666, db=0)
r_db_sat = redis.Redis(host='localhost', port=6666, db=1)
r_db_lnk = redis.Redis(host='localhost', port=6666, db=2)
link_topic = p | 'Read link topic' >> ReadFromKafka(
consumer_config=config_read,
max_num_records=2,
topics=['TEST_TOPIC'])
des_msg_link = link_topic | 'Deserialize link' >> beam.ParDo(
BeamDeSerializeAvro('TEST_TOPIC-value', schema_registry_read_client)) | 'Add key to link' >> beam.ParDo(
CreateLinkKey())
des_msg_link | 'Transform and save link satellite' >> beam.ParDo(
WriteLinkSatToRedis())
des_msg_link | 'Transform and save link' >> beam.ParDo(
WriteLinkToRedis())
```
### What did you expect to see?
I expect to see two different records saved in two different DB based on these connection settings:
```
r_db_sat = redis.Redis(host='localhost', port=6666, db=1)
r_db_lnk = redis.Redis(host='localhost', port=6666, db=2)
```
also, no record save in `r_db_hub` since it's commented out.
### What did you see instead?
I see the same record in all the databases. Like this
```
------------------------------------------------------------
LINKS
------------------------------------------------------------
b'{"HKEY": "6512bd43d9caa6e02c990b0a82652dca", "LAST_MODIFIED_TIMESTAMP": "2022-09-13 08:15:25.410000+00:00", "SAT_COLUMN_7": "1", "SAT_COLUMN_6": "1", "SAT_COLUMN_5": "1", "SAT_COLUMN_4": "1", "SAT_COLUMN_3": "1", "SAT_COLUMN_2": "1", "SAT_COLUMN_1": "1", "SOURCE": "SYN_S2V_SOURCE_LINK_TEST", "HASHDIFF": "7fa8282ad93047a4d6fe6111c93b308a"}'
b'{"HKEY": "b6d767d2f8ed5d21a44b0e5886680cb9", "LAST_MODIFIED_TIMESTAMP": "2022-09-14 13:27:49.518000+00:00", "SAT_COLUMN_7": "2", "SAT_COLUMN_6": "2", "SAT_COLUMN_5": "2", "SAT_COLUMN_4": "2", "SAT_COLUMN_3": "2", "SAT_COLUMN_2": "2", "SAT_COLUMN_1": "2", "SOURCE": "SYN_S2V_SOURCE_LINK_TEST", "HASHDIFF": "79d886010186eb60e3611cd4a5d0bcae"}'
------------------------------------------------------------
HUBS
------------------------------------------------------------
b'{"HKEY": "6512bd43d9caa6e02c990b0a82652dca", "LAST_MODIFIED_TIMESTAMP": "2022-09-13 08:15:25.410000+00:00", "SAT_COLUMN_7": "1", "SAT_COLUMN_6": "1", "SAT_COLUMN_5": "1", "SAT_COLUMN_4": "1", "SAT_COLUMN_3": "1", "SAT_COLUMN_2": "1", "SAT_COLUMN_1": "1", "SOURCE": "SYN_S2V_SOURCE_LINK_TEST", "HASHDIFF": "7fa8282ad93047a4d6fe6111c93b308a"}'
b'{"HKEY": "b6d767d2f8ed5d21a44b0e5886680cb9", "LAST_MODIFIED_TIMESTAMP": "2022-09-14 13:27:49.518000+00:00", "SAT_COLUMN_7": "2", "SAT_COLUMN_6": "2", "SAT_COLUMN_5": "2", "SAT_COLUMN_4": "2", "SAT_COLUMN_3": "2", "SAT_COLUMN_2": "2", "SAT_COLUMN_1": "2", "SOURCE": "SYN_S2V_SOURCE_LINK_TEST", "HASHDIFF": "79d886010186eb60e3611cd4a5d0bcae"}'
------------------------------------------------------------
SAT
------------------------------------------------------------
b'{"HKEY": "6512bd43d9caa6e02c990b0a82652dca", "LAST_MODIFIED_TIMESTAMP": "2022-09-13 08:15:25.410000+00:00", "SAT_COLUMN_7": "1", "SAT_COLUMN_6": "1", "SAT_COLUMN_5": "1", "SAT_COLUMN_4": "1", "SAT_COLUMN_3": "1", "SAT_COLUMN_2": "1", "SAT_COLUMN_1": "1", "SOURCE": "SYN_S2V_SOURCE_LINK_TEST", "HASHDIFF": "7fa8282ad93047a4d6fe6111c93b308a"}'
b'{"HKEY": "b6d767d2f8ed5d21a44b0e5886680cb9", "LAST_MODIFIED_TIMESTAMP": "2022-09-14 13:27:49.518000+00:00", "SAT_COLUMN_7": "2", "SAT_COLUMN_6": "2", "SAT_COLUMN_5": "2", "SAT_COLUMN_4": "2", "SAT_COLUMN_3": "2", "SAT_COLUMN_2": "2", "SAT_COLUMN_1": "2", "SOURCE": "SYN_S2V_SOURCE_LINK_TEST", "HASHDIFF": "79d886010186eb60e3611cd4a5d0bcae"}'
------------------------------------------------------------
```
### Anything Else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kvrocks] PragmaTwice commented on issue #1180: Kvrocks sets record in all open DB instances instead of only in one
Posted by GitBox <gi...@apache.org>.
PragmaTwice commented on issue #1180:
URL: https://github.com/apache/incubator-kvrocks/issues/1180#issuecomment-1351113030
> Thanks again for navigating me in the right direction. I need to interact with kvrocks via python and `redis` module, therefor I don't know if cli implementation would be sufficient for me
I think you can use some general command executing API in the client library if it has no direct API to do so, e.g. `redis.execute_command("namespace", ...)`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kvrocks] PragmaTwice closed issue #1180: Kvrocks sets record in all open DB instances instead of only in one
Posted by GitBox <gi...@apache.org>.
PragmaTwice closed issue #1180: Kvrocks sets record in all open DB instances instead of only in one
URL: https://github.com/apache/incubator-kvrocks/issues/1180
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kvrocks] PragmaTwice commented on issue #1180: Kvrocks sets record in all open DB instances instead of only in one
Posted by GitBox <gi...@apache.org>.
PragmaTwice commented on issue #1180:
URL: https://github.com/apache/incubator-kvrocks/issues/1180#issuecomment-1350688763
Same as #492.
Feel free to ask if you have more questions, otherwise I will close this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kvrocks] vhrtanek commented on issue #1180: Kvrocks sets record in all open DB instances instead of only in one
Posted by GitBox <gi...@apache.org>.
vhrtanek commented on issue #1180:
URL: https://github.com/apache/incubator-kvrocks/issues/1180#issuecomment-1350876899
Thanks again for navigating me in the right direction. I need to interact with kvrocks via python and `redis` module, therefor I don't know if kvrocks implementation would be sufficient for me, but if I understood correctly `redis-namespace` module does exactly the same as what kvrocks namespaces.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kvrocks] git-hulk commented on issue #1180: Kvrocks sets record in all open DB instances instead of only in one
Posted by GitBox <gi...@apache.org>.
git-hulk commented on issue #1180:
URL: https://github.com/apache/incubator-kvrocks/issues/1180#issuecomment-1350773780
@vhrtanek As twice mentioned, the select DB command has no effect in Kvrocks, just for compatibility with Redis clients. If you want to use multi databases, can use the namespace instead, it's totally the same as the Redis DB except for each namespace has its password.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kvrocks] vhrtanek commented on issue #1180: Kvrocks sets record in all open DB instances instead of only in one
Posted by GitBox <gi...@apache.org>.
vhrtanek commented on issue #1180:
URL: https://github.com/apache/incubator-kvrocks/issues/1180#issuecomment-1350713991
Thanks for a quick reply, I'll check it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kvrocks] git-hulk commented on issue #1180: Kvrocks sets record in all open DB instances instead of only in one
Posted by GitBox <gi...@apache.org>.
git-hulk commented on issue #1180:
URL: https://github.com/apache/incubator-kvrocks/issues/1180#issuecomment-1350937065
> I need to interact with kvrocks via python and redis module, therefor I don't know if cli implementation would be sufficient for me
Yes, using any python client is ok
> if I understood correctly redis-namespace module does exactly the same as what kvrocks namespaces
Yes, Kvrocks also uses the namespace prefix to isolate data between namespaces, some difference is Kvrocks can have its password/stats for each namespace.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org