redis-shake

https://github.com/alibaba/RedisShake/wiki/%E8%BF%90%E8%A1%8C%E7%9B%91%E6%8E%A7

redis-shake is a tool for Redis data migration and data filtering.

Feature

• 🚄 High performance
• ✅ Tested on Redis 5.0, Redis 6.0 and Redis 7.0
• 🤗 Support custom filtering rules
• 💪 Support large instance migration
• 💖 Support​​restore​​​ mode and​​sync​​ mode
• ☁️ Support Aliyun Redis and ElastiCache

For older versions of redis-shake (support codis, twemproxy) please visit ​​here​​.

​​

​​

Document

Install

Binary package

Download from Release: ​​https://github.com/alibaba/RedisShake/releases​​

Compile from source

After downloading the source code, run the ​​sh build.sh​​ command to compile.

git clone https://github.com/alibaba/RedisShake
cd RedisShake
sh build.sh

Usage

1. Edit​​sync.toml​​​ or​​restore.toml​​.
2. Start redis-shake.

./bin/redis-shake redis-shake.toml
# or
./bin/redis-shake restore.toml

3. Check data synchronization status.

Configure

The redis-shake configuration file refers to ​​sync.toml​​​ or ​​restore.toml​​.

Data filtering

redis-shake supports custom filtering rules using lua scripts. redis-shake can be started with the following command:

./bin/redis-shake sync.toml filter/xxx.lua

Some following filter templates are provided in ​​filter​​ directory:

1. ​​filter/print.lua​​：print all commands
2. ​​filter/swap_db.lua​​：swap the data of db0 and db1

Custom filter rules

Refer to ​​filter/print.lua​​ to create a new lua script, and implement the filter function in the lua script. The arguments of the function are:

• id: command id
• is_base: is the command read from the dump.rdb file
• group: command group, see the description file under​​redis/src/commands​​
• cmd_name: command name
• keys: keys in command
• slots: slots in command
• db_id: database id
• timestamp_ms: timestamp of the command in milliseconds. The current version does not support it.

The return value is:

• code

• 0: allow this command to pass
• 1: this command is not allowed to pass
• 2: this command should not appear, and let redis-shake exit with an error

• db_id: redirected db_id

Contribution

Lua script

Welcome to share more creative lua scripts.

1. Add lua scripts under​​filters/​​.
2. Add description to​​README.md​​.
3. Submit a pull request.

Redis Module support

1. Add code under​​internal/rdb/types​​.
2. Add a command file under​​scripts/commands​​​, and use the script to generate a​​table.go​​​ file and move it to the​​internal/commands​​ directory.
3. Add test cases under​​test/cases​​.
4. Submit a pull request.

感谢

redis-shake 旧版是阿里云基于豌豆荚开源的 redis-port 进行二次开发的一个支持 Redis 异构集群实时同步的工具。 redis-shake v3 在 redis-shake 旧版的基础上重新组织代码结构，使其更具可维护性的版本。

redis-shake v3 参考借鉴了以下项目：

• ​​https://github.com/HDT3213/rdb​​
• ​​https://github.com/sripathikrishnan/redis-rdb-tools​​