You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kvrocks.apache.org by "iushas (via GitHub)" <gi...@apache.org> on 2024/01/05 07:43:22 UTC

[I] kvrocks2redis run failed [kvrocks]

iushas opened a new issue, #1988:
URL: https://github.com/apache/kvrocks/issues/1988

   ### Search before asking
   
   - [X] I had searched in the [issues](https://github.com/apache/kvrocks/issues) and found no similar issues.
   
   
   ### Version
   
   unstable
   
   ### Minimal reproduce step
   
   run kvrocks2redis -c kvrocks2redis.conf
   
   ### What did you expect to see?
   
   success
   
   ### What did you see instead?
   
   coredump
   
   - gdb kvrocks2redis ./core
   ```
   Core was generated by `kvrocks2redis -c kvrocks2redis.conf'.
   Program terminated with signal SIGSEGV, Segmentation fault.
   #0  0x0000557a43551a96 in engine::Storage::LatestSeqNumber (this=<optimized out>) at incubator-kvrocks/src/storage/storage.cc:529
   529 incubator-kvrocks/src/storage/storage.cc: No such file or directory.
   ```
   
   - there is no error in logfile
   
   ```
   Log file created at: 2024/01/05 15:03:37
   Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg
   I20240105 15:03:37.463881 3476657 main.cc:109] kvrocks2redis unstable (commit 230d9726)
   I20240105 15:03:37.832939 3476657 storage.cc:343] [storage] Success to load the data from disk: 366 ms
   I20240105 15:03:37.833065 3476657 sync.cc:61] [kvrocks2redis] Start sync the data from kvrocks to redis
   ~
   ```
   
   
   
   ### Anything Else?
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [ ] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] kvrocks2redis run failed [kvrocks]

Posted by "maochongxin (via GitHub)" <gi...@apache.org>.
maochongxin commented on issue #1988:
URL: https://github.com/apache/kvrocks/issues/1988#issuecomment-1920544563

   > > ![](https://private-user-images.githubusercontent.com/24621345/301079323-e735f512-ca39-44ea-89c9-765f1f0915c0.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MDY3NTU0NzQsIm5iZiI6MTcwNjc1NTE3NCwicGF0aCI6Ii8yNDYyMTM0NS8zMDEwNzkzMjMtZTczNWY1MTItY2EzOS00NGVhLTg5YzktNzY1ZjFmMDkxNWMwLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDAyMDElMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwMjAxVDAyMzkzNFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTU3MTg0MmQ2OGM0OGY2ODc2MGIwNjlhNGJmZWU2ZjFhODMzYTZhNTYzOWQyN2EwZGRhMWU5MDVhNDFlOTJkNTImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.W6EO5_u-nga5j8iiG5XuixRPQEjUq__PvUCSqOBNcAA)
   > 
   > > 
   > 
   > > Hi @iushas , I have tested and it runs smoothly without encountering your issue. Here are the steps I followed for my testing:
   > 
   > > 
   > 
   > > commit: [e95e6f7](https://github.com/apache/kvrocks/commit/e95e6f75b6f2eaa6568ad52bf0c12ee1329bc90e)
   > 
   > > 
   > 
   > > ```
   > 
   > > # build
   > 
   > > ./x.py build
   > 
   > > 
   > 
   > > # start kvrocks
   > 
   > > ./kvrocks -c kvrocks.conf
   > 
   > > 
   > 
   > > # start redis
   > 
   > > redis-server
   > 
   > > 
   > 
   > > # start kvrocks2redis
   > 
   > > ./kvrocks2redis -c kvrocks2redis.conf
   > 
   > > 
   > 
   > > # test
   > 
   > > python3 populate-kvrocks.py --flushdb=true --password=''
   > 
   > > python3 check_consistency.py --src_password=''
   > 
   > > ```
   > 
   > > 
   > 
   > > kvrocks2redis.conf , kvrocks working directory /tmp/kvrocks
   > 
   > > 
   > 
   > > ```
   > 
   > > ################################ GENERAL #####################################
   > 
   > > 
   > 
   > > # The value should be INFO, WARNING, ERROR, FATAL.
   > 
   > > #
   > 
   > > # Default: INFO
   > 
   > > log-level INFO
   > 
   > > 
   > 
   > > # By default kvrocks2redis does not run as a daemon. Use 'yes' if you need it.
   > 
   > > # Note that kvrocks2redis will write a pid file in /var/run/kvrocks2redis.pid when daemonized.
   > 
   > > #
   > 
   > > # Default: no
   > 
   > > daemonize no
   > 
   > > 
   > 
   > > # The kvrocks working directory.
   > 
   > > # Note that you must specify a directory here, not a file name.
   > 
   > > #
   > 
   > > # Default: ./data
   > 
   > > data-dir /tmp/kvrocks
   > 
   > > 
   > 
   > > # Intermediate files are output to this directory when the kvrocks2redis program runs.
   > 
   > > #
   > 
   > > # Default: ./
   > 
   > > output-dir ./
   > 
   > > 
   > 
   > > 
   > 
   > > # Enable cluster mode.
   > 
   > > #
   > 
   > > # Default: no
   > 
   > > cluster-enabled no
   > 
   > > 
   > 
   > > ################################ NAMESPACE AND Sync Target Redis #####################################
   > 
   > > # Synchronize the specified namespace data to the specified Redis DB.
   > 
   > > # Warning: It will flush the target redis DB data.
   > 
   > > #
   > 
   > > # namespace.{namespace} <redis_ip> <redis_port> [<redis_auth> <redis_db_number>]
   > 
   > > # Default redis_db_number is 0
   > 
   > > namespace.__namespace 127.0.0.1 6379
   > 
   > > ```
   > 
   > 
   > 
   > ![image](https://github.com/apache/kvrocks/assets/7236171/e2558e5e-0bec-4e0f-ad59-b4ff1c99f0c1)
   > 
   > 
   > 
   > 
   > 
   > Still having problems, can you send me your binary file to try?
   
   let's discuss this on WeChat 18128804823


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] kvrocks2redis run failed [kvrocks]

Posted by "iushas (via GitHub)" <gi...@apache.org>.
iushas closed issue #1988: kvrocks2redis run failed
URL: https://github.com/apache/kvrocks/issues/1988


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] kvrocks2redis run failed [kvrocks]

Posted by "maochongxin (via GitHub)" <gi...@apache.org>.
maochongxin commented on issue #1988:
URL: https://github.com/apache/kvrocks/issues/1988#issuecomment-1878310229

   > In addition, the old version file contains the configuration information of the source instance. Is there no need to configure it now?
   
   Yes, in the new version, kvrocks2redis uses the SecondaryMode to open the RocksDB instance for synchronization instead of using network synchronization for incremental data. Therefore, in the configuration, the data-dir is set to the working directory of kvrocks


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] kvrocks2redis run failed [kvrocks]

Posted by "PragmaTwice (via GitHub)" <gi...@apache.org>.
PragmaTwice commented on issue #1988:
URL: https://github.com/apache/kvrocks/issues/1988#issuecomment-1921188447

   @iushas  Could you attach your coredump file and kvrocks2redis binary?
   
   And it would be very helpful if you put the crash stacktrace here. (type `bt` in gdb)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] kvrocks2redis run failed [kvrocks]

Posted by "iushas (via GitHub)" <gi...@apache.org>.
iushas commented on issue #1988:
URL: https://github.com/apache/kvrocks/issues/1988#issuecomment-1878285147

   - 1、compile kvrocks2redis only 
   ![image](https://github.com/apache/kvrocks/assets/7236171/4bf5016d-18c4-438f-bd61-99823a4a4a17)
   
   maybe this step has something wrong?  some codes do not been compiled in program?
   but version v2.7.0 is normal.
   
   - 2、run kvrocks2redis -c kvrocks2redis.conf
   ```
   log-level INFO
   daemonize no
   data-dir /home/redis/data
   output-dir /home/redis
   
   cluster-enabled no
   
   namespace.__namespace ip 6379 1234567
   
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] kvrocks2redis run failed [kvrocks]

Posted by "maochongxin (via GitHub)" <gi...@apache.org>.
maochongxin commented on issue #1988:
URL: https://github.com/apache/kvrocks/issues/1988#issuecomment-1878271735

   Hi @iushas, can you provide the reproduction steps or more stack trace information?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] kvrocks2redis run failed [kvrocks]

Posted by "iushas (via GitHub)" <gi...@apache.org>.
iushas commented on issue #1988:
URL: https://github.com/apache/kvrocks/issues/1988#issuecomment-1878312816

   > > In addition, the old version file contains the configuration information of the source instance. Is there no need to configure it now?
   > 
   > Yes, in the new version, kvrocks2redis uses the SecondaryMode to open the RocksDB instance for synchronization instead of using network synchronization for incremental data. Therefore, in the configuration, the data-dir is set to the working directory of kvrocks
   
   OK, I've made sure the working directory of kvrocks is configured correctly


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] kvrocks2redis run failed [kvrocks]

Posted by "iushas (via GitHub)" <gi...@apache.org>.
iushas commented on issue #1988:
URL: https://github.com/apache/kvrocks/issues/1988#issuecomment-1920466060

   > ![](https://private-user-images.githubusercontent.com/24621345/301079323-e735f512-ca39-44ea-89c9-765f1f0915c0.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MDY3NTU0NzQsIm5iZiI6MTcwNjc1NTE3NCwicGF0aCI6Ii8yNDYyMTM0NS8zMDEwNzkzMjMtZTczNWY1MTItY2EzOS00NGVhLTg5YzktNzY1ZjFmMDkxNWMwLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDAyMDElMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwMjAxVDAyMzkzNFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTU3MTg0MmQ2OGM0OGY2ODc2MGIwNjlhNGJmZWU2ZjFhODMzYTZhNTYzOWQyN2EwZGRhMWU5MDVhNDFlOTJkNTImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.W6EO5_u-nga5j8iiG5XuixRPQEjUq__PvUCSqOBNcAA)
   > 
   > Hi @iushas , I have tested and it runs smoothly without encountering your issue. Here are the steps I followed for my testing:
   > 
   > commit: [e95e6f7](https://github.com/apache/kvrocks/commit/e95e6f75b6f2eaa6568ad52bf0c12ee1329bc90e)
   > 
   > ```
   > # build
   > ./x.py build
   > 
   > # start kvrocks
   > ./kvrocks -c kvrocks.conf
   > 
   > # start redis
   > redis-server
   > 
   > # start kvrocks2redis
   > ./kvrocks2redis -c kvrocks2redis.conf
   > 
   > # test
   > python3 populate-kvrocks.py --flushdb=true --password=''
   > python3 check_consistency.py --src_password=''
   > ```
   > 
   > kvrocks2redis.conf , kvrocks working directory /tmp/kvrocks
   > 
   > ```
   > ################################ GENERAL #####################################
   > 
   > # The value should be INFO, WARNING, ERROR, FATAL.
   > #
   > # Default: INFO
   > log-level INFO
   > 
   > # By default kvrocks2redis does not run as a daemon. Use 'yes' if you need it.
   > # Note that kvrocks2redis will write a pid file in /var/run/kvrocks2redis.pid when daemonized.
   > #
   > # Default: no
   > daemonize no
   > 
   > # The kvrocks working directory.
   > # Note that you must specify a directory here, not a file name.
   > #
   > # Default: ./data
   > data-dir /tmp/kvrocks
   > 
   > # Intermediate files are output to this directory when the kvrocks2redis program runs.
   > #
   > # Default: ./
   > output-dir ./
   > 
   > 
   > # Enable cluster mode.
   > #
   > # Default: no
   > cluster-enabled no
   > 
   > ################################ NAMESPACE AND Sync Target Redis #####################################
   > # Synchronize the specified namespace data to the specified Redis DB.
   > # Warning: It will flush the target redis DB data.
   > #
   > # namespace.{namespace} <redis_ip> <redis_port> [<redis_auth> <redis_db_number>]
   > # Default redis_db_number is 0
   > namespace.__namespace 127.0.0.1 6379
   > ```
   
   ![image](https://github.com/apache/kvrocks/assets/7236171/e2558e5e-0bec-4e0f-ad59-b4ff1c99f0c1)
   
   
   Still having problems, can you send me your binary file to try?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] kvrocks2redis run failed [kvrocks]

Posted by "iushas (via GitHub)" <gi...@apache.org>.
iushas commented on issue #1988:
URL: https://github.com/apache/kvrocks/issues/1988#issuecomment-1878286712

   
   In addition, the old version file contains the configuration information of the source instance. Is there no need to configure it now?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org