diff --git a/.github/workflows/nrg.yml b/.github/workflows/nrg.yml new file mode 100644 index 00000000..5277e8f9 --- /dev/null +++ b/.github/workflows/nrg.yml @@ -0,0 +1,53 @@ +name: Regenerate READMEs +on: + push: + branches: [master] + paths: + - 'README.src.md' + - '.github/workflows/nrg.yml' + pull_request: + paths: + - 'README.src.md' + - 'README.md' + - 'README.zh_CN.md' + - '.github/workflows/nrg.yml' +permissions: + contents: read +jobs: + regenerate: + if: github.event_name == 'push' + runs-on: ubuntu-latest + permissions: + contents: write + steps: + - uses: actions/checkout@v4 + with: + token: ${{ secrets.GITHUB_TOKEN }} + - uses: nanolaba/nrg-action@9fae0216ed0c828f520e555912776ca624d84fb5 # v1 + with: + file: README.src.md + nrg-version: '1.2' + - name: Commit regenerated READMEs + run: | + paths=("README.md" "README.zh_CN.md") + status="$(git status --porcelain -- "${paths[@]}")" + if [ -n "$status" ]; then + changed=$(printf '%s\n' "$status" | sed 's/^...//' | sed 's/^/ - /') + git config user.name "github-actions[bot]" + git config user.email "41898282+github-actions[bot]@users.noreply.github.com" + git add -A -- "${paths[@]}" + git commit -m "docs: regenerate READMEs from README.src.md" \ + -m $'Files updated:\n'"$changed" + git push + fi + + drift-check: + if: github.event_name == 'pull_request' + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: nanolaba/nrg-action@9fae0216ed0c828f520e555912776ca624d84fb5 # v1 + with: + file: README.src.md + mode: check + nrg-version: '1.2' diff --git a/README.md b/README.md index 9797f374..0d82cb99 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,6 @@ + + + Table of Contents ([中文说明](./README.zh_CN.md)) ================= diff --git a/README.src.md b/README.src.md new file mode 100644 index 00000000..58fef2b9 --- /dev/null +++ b/README.src.md @@ -0,0 +1,1464 @@ + + + + +Table of Contents ([中文说明](./README.zh_CN.md)) +内容索引([Table of Contents](./README.md)) +================= + +* [1. Redis-replicator](#1-redis-replicator) + * [1. Redis-replicator](#1-redis-replicator) + * [1.1. Brief Introduction](#11-brief-introduction) + * [1.1. 简介](#11-简介) + * [1.2. Chat with Author](#12-chat-with-author) + * [1.2. QQ讨论组](#12-qq讨论组) + * [1.3. Contact the Author](#13-contact-the-author) + * [1.3. 联系作者](#13-联系作者) +* [2. Installation](#2-installation) + * [2. 安装](#2-安装) + * [2.1. Requirements](#21-requirements) + * [2.1. 安装前置条件](#21-安装前置条件) + * [2.2. Maven Dependency](#22-maven-dependency) + * [2.2. Maven依赖](#22-maven依赖) + * [2.3. Install from Source Code](#23-install-from-source-code) + * [2.3. 安装源码到本地maven仓库](#23-安装源码到本地maven仓库) + * [2.4. Select a Version](#24-select-a-version) + * [2.4. 选择一个版本](#24-选择一个版本) +* [3. Simple Usage](#3-simple-usage) + * [3. 简要用法](#3-简要用法) + * [3.1. Basic Usage](#31-basic-usage) + * [3.1. 用法](#31-用法) + * [3.2. Backup Remote RDB Snapshot](#32-backup-remote-rdb-snapshot) + * [3.2. 备份远程redis的rdb文件](#32-备份远程redis的rdb文件) + * [3.3. Backup Remote Commands](#33-backup-remote-commands) + * [3.3. 备份远程redis的实时命令](#33-备份远程redis的实时命令) + * [3.4. Convert RDB to Dump Format](#34-convert-rdb-to-dump-format) + * [3.4. 将rdb转换成dump格式](#34-将rdb转换成dump格式) + * [3.5. RDB Check](#35-rdb-check) + * [3.5. 检查Rdb的正确性](#35-检查rdb的正确性) + * [3.6. Scan and PSYNC](#36-scan-and-psync) + * [3.6. Scan与PSYNC](#36-scan与psync) + * [3.7. Other Examples](#37-other-examples) + * [3.7. 其他示例](#37-其他示例) +* [4. Advanced Topics](#4-advanced-topics) + * [4. 高级主题](#4-高级主题) + * [4.1. Command Extension](#41-command-extension) + * [4.1. 命令扩展](#41-命令扩展) + * [4.1.1. Write a Command](#411-write-a-command) + * [4.1.1. 首先写一个command类](#411-首先写一个command类) + * [4.1.2. Write a Command Parser](#412-write-a-command-parser) + * [4.1.2. 然后写一个command parser](#412-然后写一个command-parser) + * [4.1.3. Register the Parser](#413-register-the-parser) + * [4.1.3. 注册这个command parser到replicator](#413-注册这个command-parser到replicator) + * [4.1.4. Handle Command Event](#414-handle-command-event) + * [4.1.4. 处理这个注册的command事件](#414-处理这个注册的command事件) + * [4.1.5. Putting It All Together](#415-putting-it-all-together) + * [4.1.5. 结合到一起](#415-结合到一起) + * [4.2. Module Extension](#42-module-extension) + * [4.2. Module扩展(redis-4.0及以上)](#42-module扩展redis-40及以上) + * [4.2.1. Compile Redis Test Modules](#421-compile-redis-test-modules) + * [4.2.1. 编译redis源码中的测试modules](#421-编译redis源码中的测试modules) + * [4.2.2. Uncomment in redis.conf](#422-uncomment-in-redisconf) + * [4.2.2. 打开redis配置文件redis.conf中相关注释](#422-打开redis配置文件redisconf中相关注释) + * [4.2.3. Write a Module Parser](#423-write-a-module-parser) + * [4.2.3. 写一个module parser](#423-写一个module-parser) + * [4.2.4. Write a Command Parser](#424-write-a-command-parser) + * [4.2.4. 再写一个command parser](#424-再写一个command-parser) + * [4.2.5. Register Parsers and Handle Events](#425-register-parsers-and-handle-events) + * [4.2.5. 注册module parser和command parser并处理相关事件](#425-注册module-parser和command-parser并处理相关事件) + * [4.2.6. Putting It All Together](#426-putting-it-all-together) + * [4.2.6. 结合到一起](#426-结合到一起) + * [4.3. Stream](#43-stream) + * [4.3. Stream](#43-stream) + * [4.4. Write Your Own RDB Parser](#44-write-your-own-rdb-parser) + * [4.4. 编写你自己的rdb解析器](#44-编写你自己的rdb解析器) + * [4.5. Redis URI](#45-redis-uri) + * [4.5. Redis URI](#45-redis-uri) +* [5. Other Topics](#5-other-topics) + * [5. 其他主题](#5-其他主题) + * [5.1. Built-in Command Parsers](#51-built-in-command-parsers) + * [5.1. 内置的Command Parser](#51-内置的command-parser) + * [5.2. EOFException](#52-eofexception) + * [5.2. 当出现EOFException](#52-当出现eofexception) + * [5.3. Trace Event Log](#53-trace-event-log) + * [5.3. 跟踪事件日志log](#53-跟踪事件日志log) + * [5.4. SSL Connection](#54-ssl-connection) + * [5.4. SSL安全链接](#54-ssl安全链接) + * [5.5. Authentication](#55-authentication) + * [5.5. redis认证](#55-redis认证) + * [5.6. Avoid Full Sync](#56-avoid-full-sync) + * [5.6. 避免全量同步](#56-避免全量同步) + * [5.7. Lifecycle Events](#57-lifecycle-events) + * [5.7. 生命周期事件](#57-生命周期事件) + * [5.8. Handle Huge Key-Value Pairs](#58-handle-huge-key-value-pairs) + * [5.8. 处理巨大的KV](#58-处理巨大的kv) + * [5.9. Redis 6 Support](#59-redis-6-support) + * [5.9. Redis6支持](#59-redis6支持) + * [5.9.1. SSL Support](#591-ssl-support) + * [5.9.1. SSL支持](#591-ssl支持) + * [5.9.2. ACL Support](#592-acl-support) + * [5.9.2. ACL支持](#592-acl支持) + * [5.10. Redis 7 Support](#510-redis-7-support) + * [5.10. Redis7支持](#510-redis7支持) + * [5.10.1. Function](#5101-function) + * [5.11. Redis 7.4 Support](#511-redis-74-support) + * [5.11. Redis7.4支持](#511-redis74支持) + * [5.11.1. TTL Hash](#5111-ttl-hash) +* [6. Contributors](#6-contributors) + * [6. 贡献者](#6-贡献者) +* [7. Consulting](#7-consulting) + * [7. 商业咨询](#7-商业咨询) +* [8. References](#8-references) + * [8. 相关引用](#8-相关引用) +* [9. Supported By](#9-supported-by) + * [9. 致谢](#9-致谢) + * [9.1. 宁文君](#91-宁文君) + * [9.1. 宁文君](#91-宁文君) + * [9.2. YourKit](#92-yourkit) + * [9.2. YourKit](#92-yourkit) + * [9.3. IntelliJ IDEA](#93-intellij-idea) + * [9.3. IntelliJ IDEA](#93-intellij-idea) + * [9.4. Redisson](#94-redisson) + * [9.4. Redisson](#94-redisson) + + +# 1. Redis-replicator +# 1. Redis-replicator + +Buy Me A Coffee +Buy Me A Coffee + +## 1.1. Brief Introduction +## 1.1. 简介 +[![Java CI with Maven](https://github.com/leonchen83/redis-replicator/actions/workflows/maven.yml/badge.svg?branch=master)](https://github.com/leonchen83/redis-replicator/actions/workflows/maven.yml) +[![Coverage Status](https://coveralls.io/repos/github/leonchen83/redis-replicator/badge.svg?branch=master)](https://coveralls.io/github/leonchen83/redis-replicator?branch=master) +[![Maven Central](https://maven-badges.herokuapp.com/maven-central/com.moilioncircle/redis-replicator/badge.svg)](https://maven-badges.herokuapp.com/maven-central/com.moilioncircle/redis-replicator) +[![Javadocs](http://www.javadoc.io/badge/com.moilioncircle/redis-replicator.svg)](http://www.javadoc.io/doc/com.moilioncircle/redis-replicator) +[![Hex.pm](https://img.shields.io/hexpm/l/plug.svg?maxAge=2592000)](https://github.com/leonchen83/redis-replicator/blob/master/LICENSE) +[![LICENSE](https://img.shields.io/badge/license-Anti%20996-blue.svg?style=flat-square)](./ANTI-996-LICENSE) +[![LICENSE](https://img.shields.io/badge/license-Anti%20996-blue.svg?style=flat-square)](./ANTI-996-LICENSE_CN) + + +Redis Replicator is an implementation of the Redis Replication protocol written in Java. It can parse, filter, and broadcast RDB and AOF events in real-time. It can also synchronize Redis data to a local cache or a database. In this document, `Command` refers to writable commands (e.g., `set`, `hmset`) and excludes readable commands (e.g., `get`, `hmget`). Supports Redis 8.4.x and older versions. +Redis Replicator是一款RDB解析以及AOF解析的工具. 此工具完整实现了Redis Replication协议. 支持SYNC, PSYNC, PSYNC2等三种同步命令. 还支持远程RDB文件备份以及数据同步等功能. 此文中提到的 `命令` 特指Redis中的写(比如 `set`,`hmset`)命令，不包括读命令(比如 `get`,`hmget`), 支持的redis版本范围从2.6到8.4.x + +## 1.2. Chat with Author +## 1.2. QQ讨论组 + + +[![Join the chat at https://gitter.im/leonchen83/redis-replicator](https://badges.gitter.im/leonchen83/redis-replicator.svg)](https://gitter.im/leonchen83/redis-replicator?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) +**479688557** + +## 1.3. Contact the Author +## 1.3. 联系作者 + +**chen.bao.yi@gmail.com** +**chen.bao.yi@qq.com** + +# 2. Installation +# 2. 安装 +## 2.1. Requirements +## 2.1. 安装前置条件 +- **Compile:** JDK 9+ +编译最低jdk版本 jdk9+ +- **Runtime:** JDK 8+ +运行最低jdk版本 jdk8+ +- **Maven:** 3.3.1+ +maven-3.3.1+ +- **Redis:** 2.6 - 8.4 +redis 2.6 - 8.4 + +## 2.2. Maven Dependency +## 2.2. Maven依赖 +```xml +```xml + + + com.moilioncircle + com.moilioncircle + redis-replicator + redis-replicator + 3.11.0 + 3.11.0 + + +``` + +## 2.3. Install from Source Code +## 2.3. 安装源码到本地maven仓库 + + +```bash +``` +# Step 1: Install JDK 11+ for compilation + step 1: 安装 jdk-11.0.x +# Step 2: Clone the repository + step 2: git clone https://github.com/leonchen83/redis-replicator.git +git clone https://github.com/leonchen83/redis-replicator.git + step 3: $cd ./redis-replicator +# Step 3: Navigate to the project directory + step 4: $mvn clean install package -DskipTests +cd redis-replicator +``` +# Step 4: Build the project + +mvn clean install package -DskipTests +## 2.4. 选择一个版本 +``` + + +| **Redis Version** | **redis-replicator Version** | +## 2.4. Select a Version +|-------------------|------------------------------| + +| [2.6, 8.4.x] | [3.11.0, ] | +| **Redis Version** | **redis-replicator Version** | +| [2.6, 8.2.x] | [3.10.0,3.10.0] | +|-------------------|------------------------------| +| [2.6, 8.0.x] | [3.9.0, 3.9.0] | +| [2.6, 8.4.x] | [3.11.0, ] | +| [2.6, 7.2.x] | [3.8.0, 3.8.1] | +| [2.6, 8.2.x] | [3.10.0,3.10.0] | +| [2.6, 7.0.x] | [3.6.4, 3.7.0] | +| [2.6, 8.0.x] | [3.9.0, 3.9.0] | +| [2.6, 7.0.x-RC2] | [3.6.2, 3.6.3] | +| [2.6, 7.2.x] | [3.8.0, 3.8.1] | +| [2.6, 7.0.0-RC1] | [3.6.0, 3.6.1] | +| [2.6, 7.0.x] | [3.6.4, 3.7.0] | +| [2.6, 6.2.x] | [3.5.2, 3.5.5] | +| [2.6, 7.0.x-RC2] | [3.6.2, 3.6.3] | +| [2.6, 6.2.0-RC1] | [3.5.0, 3.5.1] | +| [2.6, 7.0.0-RC1] | [3.6.0, 3.6.1] | +| [2.6, 6.0.x] | [3.4.0, 3.4.4] | +| [2.6, 6.2.x] | [3.5.2, 3.5.5] | +| [2.6, 5.0.x] | [2.6.1, 3.3.3] | +| [2.6, 6.2.0-RC1] | [3.5.0, 3.5.1] | +| [2.6, 4.0.x] | [2.3.0, 2.5.0] | +| [2.6, 6.0.x] | [3.4.0, 3.4.4] | +| [2.6, 4.0-RC3] | [2.1.0, 2.2.0] | +| [2.6, 5.0.x] | [2.6.1, 3.3.3] | +| [2.6, 3.2.x] | [1.0.18] (not supported) | +| [2.6, 4.0.x] | [2.3.0, 2.5.0] | + +| [2.6, 4.0-RC3] | [2.1.0, 2.2.0] | + +| [2.6, 3.2.x] | [1.0.18] (not supported) | +# 3. 简要用法 + + + +## 3.1. 用法 +# 3. Simple Usage + + +```java +## 3.1. Basic Usage + Replicator replicator = new RedisReplicator("redis://127.0.0.1:6379"); + + replicator.addEventListener(new EventListener() { +```java + @Override +Replicator replicator = new RedisReplicator("redis://127.0.0.1:6379"); + public void onEvent(Replicator replicator, Event event) { +replicator.addEventListener(new EventListener() { + if (event instanceof KeyStringValueString) { + @Override + KeyStringValueString kv = (KeyStringValueString) event; + public void onEvent(Replicator replicator, Event event) { + System.out.println(new String(kv.getKey())); + if (event instanceof KeyStringValueString) { + System.out.println(new String(kv.getValue())); + KeyStringValueString kv = (KeyStringValueString) event; + } else { + System.out.println(new String(kv.getKey())); + .... + System.out.println(new String(kv.getValue())); + } + } else { + } + // ... + }); + } + replicator.open(); + } +``` +}); + +replicator.open(); +## 3.2. 备份远程redis的rdb文件 +``` + + +参阅 [RdbBackupExample.java](./examples/com/moilioncircle/examples/backup/RdbBackupExample.java) +## 3.2. Backup Remote RDB Snapshot + + +## 3.3. 备份远程redis的实时命令 +See [RdbBackupExample.java](./examples/com/moilioncircle/examples/backup/RdbBackupExample.java) + + +参阅 [CommandBackupExample.java](./examples/com/moilioncircle/examples/backup/CommandBackupExample.java) +## 3.3. Backup Remote Commands + + +## 3.4. 将rdb转换成dump格式 +See [CommandBackupExample.java](./examples/com/moilioncircle/examples/backup/CommandBackupExample.java) + + +我们可以用 `DumpRdbVisitor` 来将 rdb 转换成 redis [DUMP](https://redis.io/commands/dump) 格式。 +## 3.4. Convert RDB to Dump Format + + +```java +You can use `DumpRdbVisitor` to convert an RDB file to the Redis [DUMP](https://redis.io/commands/dump) format. + + + Replicator r = new RedisReplicator("redis:///path/to/dump.rdb"); +```java + r.setRdbVisitor(new DumpRdbVisitor(r)); +Replicator r = new RedisReplicator("redis:///path/to/dump.rdb"); + r.addEventListener(new EventListener() { +r.setRdbVisitor(new DumpRdbVisitor(r)); + @Override +r.addEventListener(new EventListener() { + public void onEvent(Replicator replicator, Event event) { + @Override + if (!(event instanceof DumpKeyValuePair)) return; + public void onEvent(Replicator replicator, Event event) { + DumpKeyValuePair dkv = (DumpKeyValuePair) event; + if (!(event instanceof DumpKeyValuePair)) return; + byte[] serialized = dkv.getValue(); + DumpKeyValuePair dkv = (DumpKeyValuePair) event; + // we can use redis RESTORE command to migrate this serialized value to another redis. + byte[] serialized = dkv.getValue(); + } + // We can use the Redis RESTORE command to migrate this serialized value to another Redis instance. + }); + } + r.open(); +}); + +r.open(); +``` +``` + + +## 3.5. 检查Rdb的正确性 +## 3.5. RDB Check + + +我们可以用 `SkipRdbVisitor` 来检查 rdb 的正确性. +You can use `SkipRdbVisitor` to check the correctness of an RDB file. + + +```java +```java + +Replicator r = new RedisReplicator("redis:///path/to/dump.rdb"); + Replicator r = new RedisReplicator("redis:///path/to/dump.rdb"); +r.setRdbVisitor(new SkipRdbVisitor(r)); + r.setRdbVisitor(new SkipRdbVisitor(r)); +r.open(); + r.open(); +``` + + +``` +## 3.6. Scan and PSYNC + + +## 3.6. Scan与PSYNC +By default, redis-replicator uses the `PSYNC` command, pretending to be a replica, to receive commands. An example is as follows: + +```java +默认情况下, redis-replicator 使用 PSYNC 命令伪装成slave接收命令, 如下所示 +Replicator r = new RedisReplicator("redis://127.0.0.1:6379"); +```java +r.addEventListener(new EventListener() { + Replicator r = new RedisReplicator("redis://127.0.0.1:6379"); + @Override + r.addEventListener(new EventListener() { + public void onEvent(Replicator replicator, Event event) { + @Override + System.out.println(event); + public void onEvent(Replicator replicator, Event event) { + } + System.out.println(event); +}); + } + + }); +r.open(); + +``` + r.open(); + +However, on some cloud services, the `PSYNC` command is prohibited. In such cases, you can use the `SCAN` command instead: +``` +```java + +Replicator r = new RedisReplicator("redis://127.0.0.1:6379?enableScan=yes&scanStep=256"); +然而, 在某些云服务中, PSYNC 是被禁止使用的, 因此我们使用 Scan 命令来替换PSYNC命令扫描全库, 如下所示 +r.addEventListener(new EventListener() { +```java + @Override + + public void onEvent(Replicator replicator, Event event) { + Replicator r = new RedisReplicator("redis://127.0.0.1:6379?enableScan=yes&scanStep=256"); + System.out.println(event); + r.addEventListener(new EventListener() { + } + @Override +}); + public void onEvent(Replicator replicator, Event event) { + + System.out.println(event); +r.open(); + } +``` + }); + + +## 3.7. Other Examples + r.open(); + +See [examples](./examples/com/moilioncircle/examples/README.md) +``` + +# 4. Advanced Topics +## 3.7. 其他示例 + +## 4.1. Command Extension +参阅 [examples](./examples/com/moilioncircle/examples/README.md) + +### 4.1.1. Write a Command +# 4. 高级主题 +```java + +@CommandSpec(command = "APPEND") +## 4.1. 命令扩展 +public static class YourAppendCommand extends AbstractCommand { + + private final String key; +### 4.1.1. 首先写一个command类 + private final String value; +```java + + @CommandSpec(command = "APPEND") + public YourAppendCommand(String key, String value) { + public static class YourAppendCommand extends AbstractCommand { + this.key = key; + private final String key; + this.value = value; + private final String value; + } + + + public YourAppendCommand(String key, String value) { + public String getKey() { + this.key = key; + return key; + this.value = value; + } + } + + + public String getValue() { + public String getKey() { + return value; + return key; + } + } +} + +``` + public String getValue() { + + return value; +### 4.1.2. Write a Command Parser + } +```java + } +public class YourAppendParser implements CommandParser { +``` + @Override + + public YourAppendCommand parse(Object[] command) { +### 4.1.2. 然后写一个command parser + return new YourAppendCommand(new String((byte[]) command[1], UTF_8), new String((byte[]) command[2], UTF_8)); +```java + } + public class YourAppendParser implements CommandParser { +} + +``` + @Override + + public YourAppendCommand parse(Object[] command) { +### 4.1.3. Register the Parser + return new YourAppendCommand(new String((byte[]) command[1], UTF_8), new String((byte[]) command[2], UTF_8)); +```java + } +Replicator replicator = new RedisReplicator("redis://127.0.0.1:6379"); + } +replicator.addCommandParser(CommandName.name("APPEND"), new YourAppendParser()); + +``` + + +### 4.1.4. Handle Command Event +### 4.1.3. 注册这个command parser到replicator +```java +```java +replicator.addEventListener(new EventListener() { + Replicator replicator = new RedisReplicator("redis://127.0.0.1:6379"); + @Override + replicator.addCommandParser(CommandName.name("APPEND"),new YourAppendParser()); + public void onEvent(Replicator replicator, Event event) { +``` + if(event instanceof YourAppendCommand){ + + YourAppendCommand appendCommand = (YourAppendCommand)event; +### 4.1.4. 处理这个注册的command事件 + // Your code goes here +```java + } + replicator.addEventListener(new EventListener() { + } + @Override +}); + public void onEvent(Replicator replicator, Event event) { +``` + if(event instanceof YourAppendCommand){ + + YourAppendCommand appendCommand = (YourAppendCommand)event; +### 4.1.5. Putting It All Together + // your code goes here + + } +See [CommandExtensionExample.java](./examples/com/moilioncircle/examples/extension/CommandExtensionExample.java) + } + + }); +## 4.2. Module Extension +``` +### 4.2.1. Compile Redis Test Modules + +```bash +### 4.1.5. 结合到一起 +cd /path/to/redis-4.0-rc2/src/modules + +make +参阅 [CommandExtensionExample.java](./examples/com/moilioncircle/examples/extension/CommandExtensionExample.java) +``` + +### 4.2.2. Uncomment in redis.conf +## 4.2. Module扩展(redis-4.0及以上) + +### 4.2.1. 编译redis源码中的测试modules +``` +```java +loadmodule /path/to/redis-4.0-rc2/src/modules/hellotype.so + $cd /path/to/redis-4.0-rc2/src/modules +``` + $make +### 4.2.3. Write a Module Parser +``` +```java +### 4.2.2. 打开redis配置文件redis.conf中相关注释 +public class HelloTypeModuleParser implements ModuleParser { + + @Override +```java + public HelloTypeModule parse(RedisInputStream in, int version) throws IOException { + loadmodule /path/to/redis-4.0-rc2/src/modules/hellotype.so + DefaultRdbModuleParser parser = new DefaultRdbModuleParser(in); +``` + int elements = parser.loadUnsigned(version).intValue(); +### 4.2.3. 写一个module parser + long[] ary = new long[elements]; +```java + int i = 0; + public class HelloTypeModuleParser implements ModuleParser { + while (elements-- > 0) { + + ary[i++] = parser.loadSigned(version); + @Override + } + public HelloTypeModule parse(RedisInputStream in, int version) throws IOException { + return new HelloTypeModule(ary); + DefaultRdbModuleParser parser = new DefaultRdbModuleParser(in); + } + int elements = parser.loadUnsigned(version).intValue(); +} + long[] ary = new long[elements]; + + int i = 0; +public class HelloTypeModule implements Module { + while (elements-- > 0) { + private final long[] value; + ary[i++] = parser.loadSigned(version); + + } + public HelloTypeModule(long[] value) { + return new HelloTypeModule(ary); + this.value = value; + } + } + + public long[] getValue() { + public class HelloTypeModule implements Module { + return value; + private final long[] value; + } + +} + public HelloTypeModule(long[] value) { +``` + this.value = value; +### 4.2.4. Write a Command Parser + } +```java + +public class HelloTypeParser implements CommandParser { + public long[] getValue() { + @Override + return value; + public HelloTypeCommand parse(Object[] command) { + } + String key = new String((byte[]) command[1], Constants.UTF_8); + } + long value = Long.parseLong(new String((byte[]) command[2], Constants.UTF_8)); +``` + return new HelloTypeCommand(key, value); +### 4.2.4. 再写一个command parser + } +```java +} + public class HelloTypeParser implements CommandParser { + + @Override +@CommandSpec(command = "hellotype.insert") + public HelloTypeCommand parse(Object[] command) { +public class HelloTypeCommand extends AbstractCommand { + String key = new String((byte[])command[1],Constants.UTF_8); + private final String key; + long value = Long.parseLong(new String((byte[])command[2],Constants.UTF_8)); + private final long value; + return new HelloTypeCommand(key, value); + + } + public long getValue() { + } + return value; + + } + @CommandSpec(command = "hellotype.insert") + + public class HelloTypeCommand extends AbstractCommand { + public String getKey() { + private final String key; + return key; + private final long value; + } + + + public long getValue() { + public HelloTypeCommand(String key, long value) { + return value; + this.key = key; + } + this.value = value; + + } + public String getKey() { +} + return key; +``` + } +### 4.2.5. Register Parsers and Handle Events + + + public HelloTypeCommand(String key, long value) { +```java + this.key = key; +public static void main(String[] args) throws IOException { + this.value = value; + Replicator replicator = new RedisReplicator("redis://127.0.0.1:6379"); + } + replicator.addCommandParser(CommandName.name("hellotype.insert"), new HelloTypeParser()); + } + replicator.addModuleParser("hellotype", 0, new HelloTypeModuleParser()); +``` + replicator.addEventListener(new EventListener() { +### 4.2.5. 注册module parser和command parser并处理相关事件 + @Override + + public void onEvent(Replicator replicator, Event event) { +```java + if (event instanceof KeyStringValueModule) { + public static void main(String[] args) throws IOException { + System.out.println(event); + Replicator replicator = new RedisReplicator("redis://127.0.0.1:6379"); + } + replicator.addCommandParser(CommandName.name("hellotype.insert"), new HelloTypeParser()); + + replicator.addModuleParser("hellotype", 0, new HelloTypeModuleParser()); + if (event instanceof HelloTypeCommand) { + replicator.addEventListener(new EventListener() { + System.out.println(event); + @Override + } + public void onEvent(Replicator replicator, Event event) { + } + if (event instanceof KeyStringValueModule) { + }); + System.out.println(event); + replicator.open(); + } +} + +``` + if (event instanceof HelloTypeCommand) { + + System.out.println(event); +### 4.2.6. Putting It All Together + } + + } +See [ModuleExtensionExample.java](./examples/com/moilioncircle/examples/extension/ModuleExtensionExample.java) + }); + + replicator.open(); +## 4.3. Stream + } + +``` +Since Redis 5.0, a new data structure called `STREAM` has been added. Redis-replicator parses `STREAM` data as follows: + + +### 4.2.6. 结合到一起 +```java + +Replicator r = new RedisReplicator("redis://127.0.0.1:6379"); +参阅 [ModuleExtensionExample.java](./examples/com/moilioncircle/examples/extension/ModuleExtensionExample.java) +r.addEventListener(new EventListener() { + + @Override +## 4.3. Stream + public void onEvent(Replicator replicator, Event event) { + + if (event instanceof KeyStringValueStream) { +Redis-5.0+ 增加了一个新的数据结构 `STREAM`. Redis-replicator 用下述代码解析 `STREAM` + KeyStringValueStream kv = (KeyStringValueStream)event; + + // Key + + String key = kv.getKey(); +```java + + + // Stream + Replicator r = new RedisReplicator("redis://127.0.0.1:6379"); + Stream stream = kv.getValueAsStream(); + r.addEventListener(new EventListener() { + // Last stream ID + @Override + stream.getLastId(); + public void onEvent(Replicator replicator, Event event) { + + if (event instanceof KeyStringValueStream) { + // Entries + KeyStringValueStream kv = (KeyStringValueStream)event; + NavigableMap entries = stream.getEntries(); + // key + + String key = kv.getKey(); + // Optional: Groups + + for (Stream.Group group : stream.getGroups()) { + // stream + // Group PEL (Pending Entries List) + Stream stream = kv.getValueAsStream(); + NavigableMap gpel = group.getPendingEntries(); + // last stream id + + stream.getLastId(); + // Consumers + + for (Stream.Consumer consumer : group.getConsumers()) { + // entries + // Consumer PEL (Pending Entries List) + NavigableMap entries = stream.getEntries(); + NavigableMap cpel = consumer.getPendingEntries(); + + } + // optional : group + } + for (Stream.Group group : stream.getGroups()) { + } + // group PEL(pending entries list) + } + NavigableMap gpel = group.getPendingEntries(); +}); + +r.open(); + // consumer +``` + for (Stream.Consumer consumer : group.getConsumers()) { + + // consumer PEL(pending entries list) +## 4.4. Write Your Own RDB Parser + NavigableMap cpel = consumer.getPendingEntries(); + + } +* Write a `YourRdbVisitor` that extends `RdbVisitor`. + } +* Register your `RdbVisitor` with the `Replicator` using the `setRdbVisitor` method. + } + + } +## 4.5. Redis URI + }); + + r.open(); +Before version 2.4.0, `RedisReplicator` was constructed as follows: + + +``` +```java + +Replicator replicator = new RedisReplicator("127.0.0.1", 6379, Configuration.defaultSetting()); +## 4.4. 编写你自己的rdb解析器 +Replicator replicator = new RedisReplicator(new File("/path/to/dump.rdb"), FileType.RDB, Configuration.defaultSetting()); + +Replicator replicator = new RedisReplicator(new File("/path/to/appendonly.aof"), FileType.AOF, Configuration.defaultSetting()); +* 写一个类继承 `RdbVisitor` 抽象类 +Replicator replicator = new RedisReplicator(new File("/path/to/appendonly.aof"), FileType.MIXED, Configuration.defaultSetting()); +* 通过 `Replicator` 的 `setRdbVisitor` 方法注册你自己的 `RdbVisitor`. +``` + + +## 4.5. Redis URI +Since version 2.4.0, we have introduced the Redis URI concept to simplify the `RedisReplicator` construction process: + + +在 redis-replicator-2.4.0 版之前, 我们按如下方式构造 `RedisReplicator` : +```java + +Replicator replicator = new RedisReplicator("redis://127.0.0.1:6379"); +```java +Replicator replicator = new RedisReplicator("redis:///path/to/dump.rdb"); +Replicator replicator = new RedisReplicator("127.0.0.1", 6379, Configuration.defaultSetting()); +Replicator replicator = new RedisReplicator("redis:///path/to/appendonly.aof"); +Replicator replicator = new RedisReplicator(new File("/path/to/dump.rdb", FileType.RDB, Configuration.defaultSetting()); + +Replicator replicator = new RedisReplicator(new File("/path/to/appendonly.aof", FileType.AOF, Configuration.defaultSetting()); +// Configuration setting example +Replicator replicator = new RedisReplicator(new File("/path/to/appendonly.aof", FileType.MIXED, Configuration.defaultSetting()); +Replicator replicator = new RedisReplicator("redis://127.0.0.1:6379?authPassword=foobared&readTimeout=10000&ssl=yes"); +``` +Replicator replicator = new RedisReplicator("redis:///path/to/dump.rdb?rateLimit=1000000"); + +Replicator replicator = new RedisReplicator("rediss://user:pass@127.0.0.1:6379?rateLimit=1000000"); +在 redis-replicator-2.4.0 版之后, 我们引入了一个新的概念(Redis URI) 来简化 `RedisReplicator` 的构造, 以便提供一致的API. +``` + + +```java +# 5. Other Topics +Replicator replicator = new RedisReplicator("redis://127.0.0.1:6379"); + +Replicator replicator = new RedisReplicator("redis:///path/to/dump.rdb"); +## 5.1. Built-in Command Parsers +Replicator replicator = new RedisReplicator("redis:///path/to/appendonly.aof"); + +| **Command** | **Command** | **Command** | **Command** | **Command** | **Command** | +// 配置的例子 +|--------------|----------------|--------------------|--------------|---------------|----------------------| +Replicator replicator = new RedisReplicator("redis://127.0.0.1:6379?authPassword=foobared&readTimeout=10000&ssl=yes"); +| **PING** | **APPEND** | **SET** | **SETEX** | **MSET** | **DEL** | +Replicator replicator = new RedisReplicator("redis:///path/to/dump.rdb?rateLimit=1000000"); +| **SADD** | **HMSET** | **HSET** | **LSET** | **EXPIRE** | **EXPIREAT** | +Replicator replicator = new RedisReplicator("rediss://user:pass@127.0.0.1:6379?rateLimit=1000000"); +| **GETSET** | **HSETNX** | **MSETNX** | **PSETEX** | **SETNX** | **SETRANGE** | +``` +| **HDEL** | **UNLINK** | **SREM** | **LPOP** | **LPUSH** | **LPUSHX** | + +| **LREM** | **RPOP** | **RPUSH** | **RPUSHX** | **ZREM** | **ZINTERSTORE** | + +| **INCR** | **DECR** | **INCRBY** | **PERSIST** | **SELECT** | **FLUSHALL** | +# 5. 其他主题 +| **FLUSHDB** | **HINCRBY** | **ZINCRBY** | **MOVE** | **SMOVE** | **BRPOPLPUSH** | + +| **PFCOUNT** | **PFMERGE** | **SDIFFSTORE** | **RENAMENX** | **PEXPIREAT** | **SINTERSTORE** | +## 5.1. 内置的Command Parser +| **ZADD** | **BITFIELD** | **SUNIONSTORE** | **RESTORE** | **LINSERT** | **ZREMRANGEBYLEX** | + +| **GEOADD** | **PEXPIRE** | **ZUNIONSTORE** | **EVAL** | **SCRIPT** | **ZREMRANGEBYRANK** | +| **命令** |**命令** | **命令** |**命令** |**命令** | **命令** | +| **PUBLISH** | **BITOP** | **SETBIT** | **SWAPDB** | **PFADD** | **ZREMRANGEBYSCORE** | +|--------------| -------------- |--------------------| ------------ | ------------ | ------------------ | +| **RENAME** | **MULTI** | **EXEC** | **LTRIM** | **RPOPLPUSH** | **SORT** | +| **PING** | **APPEND** | **SET** | **SETEX** | **MSET** | **DEL** | +| **EVALSHA** | **ZPOPMAX** | **ZPOPMIN** | **XACK** | **XADD** | **XCLAIM** | +| **SADD** | **HMSET** | **HSET** | **LSET** | **EXPIRE** | **EXPIREAT** | +| **XDEL** | **XGROUP** | **XTRIM** | **XSETID** | **COPY** | **LMOVE** | +| **GETSET** | **HSETNX** | **MSETNX** | **PSETEX** | **SETNX** | **SETRANGE** | +| **BLMOVE** | **ZDIFFSTORE** | **GEOSEARCHSTORE** | **FUNCTION** | **SPUBLISH** | **HPERSIST** | +| **HDEL** | **UNLINK** | **SREM** | **LPOP** | **LPUSH** | **LPUSHX** | +| **HSETEX** | **HPEXPIREAT** | **XACKDEL** | **XDELEX** | **MSETEX** | | +| **LRem** | **RPOP** | **RPUSH** | **RPUSHX** | **ZREM** | **ZINTERSTORE** | + +| **INCR** | **DECR** | **INCRBY** |**PERSIST** | **SELECT** | **FLUSHALL** | +## 5.2. EOFException +| **FLUSHDB** | **HINCRBY** | **ZINCRBY** | **MOVE** | **SMOVE** |**BRPOPLPUSH** | + +| **PFCOUNT** | **PFMERGE** | **SDIFFSTORE** |**RENAMENX** | **PEXPIREAT**|**SINTERSTORE** | +When event consumption is too slow and the backlog of events exceeds the Redis backlog limit, Redis will actively disconnect from the slave. When Redis-replicator reconnects, it will perform a full synchronization. To avoid this situation, you need to set the parameter `client-output-buffer-limit slave 0 0 0`. +| **ZADD** | **BITFIELD** | **SUNIONSTORE** |**RESTORE** | **LINSERT** |**ZREMRANGEBYLEX** | + +| **GEOADD** | **PEXPIRE** | **ZUNIONSTORE** |**EVAL** | **SCRIPT** |**ZREMRANGEBYRANK** | +For more details, please refer to [redis.conf](https://raw.githubusercontent.com/antirez/redis/3.0/redis.conf). +| **PUBLISH** | **BITOP** | **SETBIT** | **SWAPDB** | **PFADD** |**ZREMRANGEBYSCORE**| + +| **RENAME** | **MULTI** | **EXEC** | **LTRIM** |**RPOPLPUSH** | **SORT** | +``` +| **EVALSHA** | **ZPOPMAX** | **ZPOPMIN** | **XACK** | **XADD** | **XCLAIM** | +client-output-buffer-limit slave 0 0 0 +| **XDEL** | **XGROUP** | **XTRIM** | **XSETID** | **COPY** | **LMOVE** | +``` +| **BLMOVE** | **ZDIFFSTORE** | **GEOSEARCHSTORE** | **FUNCTION** | **SPUBLISH** | **HPERSIST** | +**WARNING: This setting may cause the Redis server to run out of memory in some cases.** +| **HSETEX** | **HPEXPIREAT** | **XACKDEL** | **XDELEX** | **MSETEX** | | + + +## 5.3. Trace Event Log +## 5.2. 当出现EOFException + +* Set the log level to **debug**. +当消费事件过慢积压事件超过redis backlog限制时，redis会主动断开与slave的连接，Redis-replicator再重连时会走全量同步，如果想避免这一情况，需要设置参数`client-output-buffer-limit slave 0 0 0` +* If you are using Log4j2, add a logger as shown below: + + +相关配置请参考 [redis.conf](https://raw.githubusercontent.com/antirez/redis/3.0/redis.conf) +```xml + + +```java + +client-output-buffer-limit slave 0 0 0 + +``` +``` +**警告: 这个配置可能会使redis-server中的内存溢出** + + +```java +## 5.3. 跟踪事件日志log +Configuration.defaultSetting().setVerbose(true); + +// As a Redis URI parameter +* 日志级别调整成 **debug** +"redis://127.0.0.1:6379?verbose=yes" +* 如果你项目中使用log4j2,请加入如下Logger到配置文件: +``` + + +```xml +## 5.4. SSL Connection + + + +```java + +System.setProperty("javax.net.ssl.keyStore", "/path/to/keystore"); +``` +System.setProperty("javax.net.ssl.keyStorePassword", "password"); + +System.setProperty("javax.net.ssl.keyStoreType", "your_type"); +```java + + Configuration.defaultSetting().setVerbose(true); +System.setProperty("javax.net.ssl.trustStore", "/path/to/truststore"); + // redis uri +System.setProperty("javax.net.ssl.trustStorePassword", "password"); + "redis://127.0.0.1?verbose=yes" +System.setProperty("javax.net.ssl.trustStoreType", "your_type"); +``` + + +Configuration.defaultSetting().setSsl(true); +## 5.4. SSL安全链接 + + +// Optional settings +```java +Configuration.defaultSetting().setSslSocketFactory(sslSocketFactory); + System.setProperty("javax.net.ssl.keyStore", "/path/to/keystore"); +Configuration.defaultSetting().setSslParameters(sslParameters); + System.setProperty("javax.net.ssl.keyStorePassword", "password"); +Configuration.defaultSetting().setHostnameVerifier(hostnameVerifier); + System.setProperty("javax.net.ssl.keyStoreType", "your_type"); + +// As a Redis URI parameter + System.setProperty("javax.net.ssl.trustStore", "/path/to/truststore"); +"redis://127.0.0.1:6379?ssl=yes" + System.setProperty("javax.net.ssl.trustStorePassword", "password"); +"rediss://127.0.0.1:6379" + System.setProperty("javax.net.ssl.trustStoreType", "your_type"); +``` + + + Configuration.defaultSetting().setSsl(true); +If you prefer not to use `System.setProperty`, you can configure it programmatically as follows: + + + // 可选设置 +```java + Configuration.defaultSetting().setSslSocketFactory(sslSocketFactory); +RedisSslContextFactory factory = new RedisSslContextFactory(); + Configuration.defaultSetting().setSslParameters(sslParameters); +factory.setKeyStorePath("/path/to/redis/tests/tls/redis.p12"); + Configuration.defaultSetting().setHostnameVerifier(hostnameVerifier); +factory.setKeyStoreType("pkcs12"); + // redis uri +factory.setKeyStorePassword("password"); + "redis://127.0.0.1:6379?ssl=yes" + + "rediss://127.0.0.1:6379" +factory.setTrustStorePath("/path/to/redis/tests/tls/redis.p12"); +``` +factory.setTrustStoreType("pkcs12"); + +factory.setTrustStorePassword("password"); +## 5.5. redis认证 + + +SslConfiguration ssl = SslConfiguration.defaultSetting().setSslContextFactory(factory); +```java +Replicator replicator = new RedisReplicator("rediss://127.0.0.1:6379", ssl); + Configuration.defaultSetting().setAuthUser("default"); +``` + Configuration.defaultSetting().setAuthPassword("foobared"); + + // redis uri +## 5.5. Authentication + "redis://127.0.0.1:6379?authPassword=foobared&authUser=default" + + "redis://default:foobared@127.0.0.1:6379" +```java +``` +Configuration.defaultSetting().setAuthUser("default"); + +Configuration.defaultSetting().setAuthPassword("foobared"); +## 5.6. 避免全量同步 + + +// As a Redis URI parameter +* 调整redis-server中的如下配置 +"redis://127.0.0.1:6379?authPassword=foobared&authUser=default" + +"redis://default:foobared@127.0.0.1:6379" +```java +``` + repl-backlog-size + + repl-backlog-ttl +## 5.6. Avoid Full Sync + repl-ping-slave-period + +``` +Adjust the Redis server settings as follows: +`repl-ping-slave-period` **必须** 小于 `Configuration.getReadTimeout()`, 默认的 `Configuration.getReadTimeout()` 是60秒. + + +``` +## 5.7. 生命周期事件 +repl-backlog-size + +repl-backlog-ttl +```java +repl-ping-slave-period + Replicator replicator = new RedisReplicator("redis://127.0.0.1:6379"); +``` + final long start = System.currentTimeMillis(); +The `repl-ping-slave-period` **MUST** be less than `Configuration.getReadTimeout()`. The default `Configuration.getReadTimeout()` is 60 seconds. + final AtomicInteger acc = new AtomicInteger(0); + + replicator.addEventListener(new EventListener() { +## 5.7. Lifecycle Events + @Override + + public void onEvent(Replicator replicator, Event event) { +```java + if(event instanceof PreRdbSyncEvent) { +Replicator replicator = new RedisReplicator("redis://127.0.0.1:6379"); + System.out.println("pre rdb sync"); +final long start = System.currentTimeMillis(); + } else if(event instanceof PostRdbSyncEvent) { +final AtomicInteger acc = new AtomicInteger(0); + long end = System.currentTimeMillis(); +replicator.addEventListener(new EventListener() { + System.out.println("time elapsed:" + (end - start)); + @Override + System.out.println("rdb event count:" + acc.get()); + public void onEvent(Replicator replicator, Event event) { + } else { + if(event instanceof PreRdbSyncEvent) { + acc.incrementAndGet(); + System.out.println("pre rdb sync"); + } + } else if(event instanceof PostRdbSyncEvent) { + } + long end = System.currentTimeMillis(); + }); + System.out.println("time elapsed:" + (end - start)); + replicator.open(); + System.out.println("rdb event count:" + acc.get()); +``` + } else { + + acc.incrementAndGet(); +## 5.8. 处理巨大的KV + } + + } +根据 [4.3. 编写你自己的rdb解析器](#43-编写你自己的rdb解析器), 这个工具内嵌了一个[迭代方式的rdb解析器](./src/main/java/com/moilioncircle/redis/replicator/rdb/iterable/ValueIterableRdbVisitor.java), 以便处理巨大的KV. +}); +详细的例子参阅: +replicator.open(); +[1] [HugeKVFileExample.java](./examples/com/moilioncircle/examples/huge/HugeKVFileExample.java) +``` +[2] [HugeKVSocketExample.java](./examples/com/moilioncircle/examples/huge/HugeKVSocketExample.java) + + +## 5.8. Handle Huge Key-Value Pairs +## 5.9. Redis6支持 + +As mentioned in [4.4. Write Your Own RDB Parser](#44-write-your-own-rdb-parser), this tool has a built-in [Iterable Rdb Parser](./src/main/java/com/moilioncircle/redis/replicator/rdb/iterable/ValueIterableRdbVisitor.java) to handle huge key-value pairs. +### 5.9.1. SSL支持 +For more details, please refer to: + +[1] [HugeKVFileExample.java](./examples/com/moilioncircle/examples/huge/HugeKVFileExample.java) +``` +[2] [HugeKVSocketExample.java](./examples/com/moilioncircle/examples/huge/HugeKVSocketExample.java) + $cd /path/to/redis + + $./utils/gen-test-certs.sh +## 5.9. Redis 6 Support + $cd tests/tls + + $openssl pkcs12 -export -CAfile ca.crt -in redis.crt -inkey redis.key -out redis.p12 +### 5.9.1. SSL Support + $cd /path/to/redis + + $./src/redis-server --tls-port 6379 --port 0 --tls-cert-file ./tests/tls/redis.crt \ +```bash + --tls-key-file ./tests/tls/redis.key --tls-ca-cert-file ./tests/tls/ca.crt \ +cd /path/to/redis + --tls-replication yes --bind 0.0.0.0 --protected-mode no +./utils/gen-test-certs.sh + +cd tests/tls + System.setProperty("javax.net.ssl.keyStore", "/path/to/redis/tests/tls/redis.p12"); +openssl pkcs12 -export -CAfile ca.crt -in redis.crt -inkey redis.key -out redis.p12 + System.setProperty("javax.net.ssl.keyStorePassword", "password"); +cd /path/to/redis + System.setProperty("javax.net.ssl.keyStoreType", "pkcs12"); +./src/redis-server --tls-port 6379 --port 0 --tls-cert-file ./tests/tls/redis.crt \ + + --tls-key-file ./tests/tls/redis.key --tls-ca-cert-file ./tests/tls/ca.crt \ + System.setProperty("javax.net.ssl.trustStore", "/path/to/redis/tests/tls/redis.p12"); + --tls-replication yes --bind 0.0.0.0 --protected-mode no + System.setProperty("javax.net.ssl.trustStorePassword", "password"); +``` + System.setProperty("javax.net.ssl.trustStoreType", "pkcs12"); + +```java + Replicator replicator = new RedisReplicator("rediss://127.0.0.1:6379"); +System.setProperty("javax.net.ssl.keyStore", "/path/to/redis/tests/tls/redis.p12"); + +System.setProperty("javax.net.ssl.keyStorePassword", "password"); +``` +System.setProperty("javax.net.ssl.keyStoreType", "pkcs12"); + + +如果你不想设置 `System.setProperty` 可以使用下面的方式 +System.setProperty("javax.net.ssl.trustStore", "/path/to/redis/tests/tls/redis.p12"); + +System.setProperty("javax.net.ssl.trustStorePassword", "password"); +```java +System.setProperty("javax.net.ssl.trustStoreType", "pkcs12"); + + + RedisSslContextFactory factory = new RedisSslContextFactory(); +Replicator replicator = new RedisReplicator("rediss://127.0.0.1:6379"); + factory.setKeyStorePath("/path/to/redis/tests/tls/redis.p12"); +``` + factory.setKeyStoreType("pkcs12"); + + factory.setKeyStorePassword("password"); +### 5.9.2. ACL Support + + + factory.setTrustStorePath("/path/to/redis/tests/tls/redis.p12"); +```java + factory.setTrustStoreType("pkcs12"); +Replicator replicator = new RedisReplicator("redis://user:pass@127.0.0.1:6379"); + factory.setTrustStorePassword("password"); +``` + + + SslConfiguration ssl = SslConfiguration.defaultSetting().setSslContextFactory(factory); +## 5.10. Redis 7 Support + Replicator replicator = new RedisReplicator("rediss://127.0.0.1:6379", ssl); + +### 5.10.1. Function +``` + +Since Redis 7.0, `FUNCTION` is supported, and its structure is stored in the RDB file. You can use the following method to parse a `FUNCTION`. +### 5.9.2. ACL支持 + +```java +```java +Replicator replicator = new RedisReplicator("redis://127.0.0.1:6379"); + +replicator.addEventListener(new EventListener() { + Replicator replicator = new RedisReplicator("redis://user:pass@127.0.0.1:6379"); + @Override + + public void onEvent(Replicator replicator, Event event) { +``` + if (event instanceof Function) { + + Function function = (Function) event; +## 5.10. Redis7支持 + function.getCode(); + + +### 5.10.1. Function + // Your code goes here + + } +Redis 7.0 添加了 `function` 的支持. `function` 的结构存储在rdb文件中. 因此我们能用如下方式解析`function`. + } + +}); +```java +replicator.open(); + +``` + Replicator replicator = new RedisReplicator("redis://127.0.0.1:6379"); + + replicator.addEventListener(new EventListener() { +You can also parse a `FUNCTION` into serialized data and use `FUNCTION RESTORE` to restore it to a target Redis instance. + @Override + + public void onEvent(Replicator replicator, Event event) { +```java + if (event instanceof Function) { +Replicator replicator = new RedisReplicator("redis://127.0.0.1:6379"); + Function function = (Function) event; +replicator.setRdbVisitor(new DumpRdbVisitor(replicator)); + function.getCode(); +replicator.addEventListener(new EventListener() { + + @Override + // your code goes here + public void onEvent(Replicator replicator, Event event) { + } + if (event instanceof DumpFunction) { + } + DumpFunction function = (DumpFunction) event; + }); + byte[] serialized = function.getSerialized(); + replicator.open(); + // Your code goes here +``` + // You can use FUNCTION RESTORE to restore the serialized data to a target Redis instance + + } +也可以把 `function` 解析成 `serialized` 格式. 这样接下来我们可以用 `FUNCTION RESTORE` 命令把 `serialized` 数据迁移到目标redis + } + +}); +```java +replicator.open(); + +``` + Replicator replicator = new RedisReplicator("redis://127.0.0.1:6379"); + + replicator.setRdbVisitor(new DumpRdbVisitor(replicator)); +## 5.11. Redis 7.4 Support + replicator.addEventListener(new EventListener() { + + @Override +### 5.11.1. TTL Hash + public void onEvent(Replicator replicator, Event event) { + + if (event instanceof DumpFunction) { +Since Redis 7.4, `TTL HASH` is supported, and its structure is stored in the RDB file. You can use the following method to parse a `TTL HASH`. + DumpFunction function = (DumpFunction) event; + + byte[] serialized = function.getSerialized(); +```java + // your code goes here +Replicator replicator = new RedisReplicator("redis://127.0.0.1:6379"); + // you can use FUNCTION RESTORE to restore above serialized data to target redis +replicator.addEventListener(new EventListener() { + } + @Override + } + public void onEvent(Replicator replicator, Event event) { + }); + if (event instanceof KeyStringValueTTLHash) { + replicator.open(); + KeyStringValueTTLHash skv = (KeyStringValueTTLHash) event; +``` + // Key + + byte[] key = skv.getKey(); +## 5.11. Redis7.4支持 + + + // TTL Hash +### 5.11.1. TTL Hash + Map ttlHash = skv.getValue(); + + for (Map.Entry entry : ttlHash.entrySet()) { +Redis 7.4 添加了 `ttl hash` 的支持. `ttl hash` 的结构存储在rdb文件中. 因此我们能用如下方式解析`ttl hash`. + System.out.println("field: " + Strings.toString(entry.getKey())); + + System.out.println("value: " + Strings.toString(entry.getValue().getValue())); +```java + System.out.println("field ttl: " + entry.getValue().getExpires()); + + } + Replicator replicator = new RedisReplicator("redis://127.0.0.1:6379"); + } + replicator.addEventListener(new EventListener() { + } + @Override +}); + public void onEvent(Replicator replicator, Event event) { +replicator.open(); + if (event instanceof KeyStringValueTTLHash) { +``` + KeyStringValueTTLHash skv = (KeyStringValueTTLHash) event; + + // key +# 6. Contributors + byte[] key = skv.getKey(); +* [Leon Chen](https://github.com/leonchen83) + +* [Adrian Yao](https://github.com/adrianyao89) + // ttl hash +* [Trydofor](https://github.com/trydofor) + Map ttlHash = skv.getValue(); +* [Argun](https://github.com/Argun) + for (Map.Entry entry : ttlHash.entrySet()) { +* [Sean Pan](https://github.com/XinYang-Pan) + System.out.println("field:" + Strings.toString(entry.getKey())); +* [René Kerner](https://github.com/rk3rn3r) + System.out.println("value:" + Strings.toString(entry.getValue().getValue())); +* [Maplestoria](https://github.com/maplestoria) + System.out.println("field ttl:" + entry.getValue().getExpires()); +* Special thanks to [Kevin Zheng](https://github.com/KevinZheng001) + } + + } +# 7. Consulting + } + + }); +Commercial support for `redis-replicator` is available. The following services are currently offered: + replicator.open(); +* Onsite consulting: $10,000 per day +``` +* Onsite training: $10,000 per day + + +# 6. 贡献者 +You may also contact Baoyi Chen directly at [chen.bao.yi@gmail.com](mailto:chen.bao.yi@gmail.com). + + +* [Leon Chen](https://github.com/leonchen83) +# 8. References +* [Adrian Yao](https://github.com/adrianyao89) +* [rdb.c](https://github.com/antirez/redis/blob/unstable/src/rdb.c) +* [Trydofor](https://github.com/trydofor) +* [Redis RDB File Format](https://github.com/leonchen83/redis-replicator/wiki/RDB-dump-data-format) +* [Argun](https://github.com/Argun) +* [Redis Protocol specification](http://redis.io/topics/protocol) +* [Sean Pan](https://github.com/XinYang-Pan) +* [Redis Replication](http://redis.io/topics/replication) +* [René Kerner](https://github.com/rk3rn3r) +* [Redis-replicator Design and Implementation](https://github.com/leonchen83/mycode/blob/master/redis/redis-share/Redis-replicator%E8%AE%BE%E8%AE%A1%E4%B8%8E%E5%AE%9E%E7%8E%B0.md) +* [Maplestoria](https://github.com/maplestoria) + +* 特别感谢 [Kevin Zheng](https://github.com/KevinZheng001) +# 9. Supported By + + +# 7. 商业咨询 +## 9.1. 宁文君 + + +`redis-replicator` 支持如下的商业咨询服务: +January 27, 2023, was a sad day as I lost my mother, 宁文君. She was always encouraging and supportive of my work on this tool. Every time a company started using it, she would get as excited as a child and motivate me to continue. Without her, I could not have maintained this tool for so many years. Even though I haven't achieved much, she was always proud of me. R.I.P, and may God bless her. +* 现场咨询. 50,000元/天 + +* 现场培训. 50,000元/天 +## 9.2. YourKit + + +可以直接联系`陈宝仪`, 发送邮件至 [chen.bao.yi@gmail.com](mailto:chen.bao.yi@qq.com). +![YourKit](https://www.yourkit.com/images/yklogo.png) + +YourKit is kindly supporting this open source project with its full-featured Java Profiler. +# 8. 相关引用 +YourKit, LLC is the creator of innovative and intelligent tools for profiling + * [rdb.c](https://github.com/antirez/redis/blob/unstable/src/rdb.c) +Java and .NET applications. Take a look at YourKit's leading software products: + * [Redis RDB文件格式](https://github.com/leonchen83/redis-replicator/wiki/RDB-dump-data-format) +YourKit Java Profiler and + * [Redis 协议指南](http://redis.io/topics/protocol) +YourKit .NET Profiler. + * [Redis 同步协议](http://redis.io/topics/replication) + + * [Redis-replicator 设计与实现](https://github.com/leonchen83/mycode/blob/master/redis/redis-share/Redis-replicator%E8%AE%BE%E8%AE%A1%E4%B8%8E%E5%AE%9E%E7%8E%B0.md) +## 9.3. IntelliJ IDEA + + +# 9. 致谢 +[IntelliJ IDEA](https://www.jetbrains.com/?from=redis-replicator) is a Java Integrated Development Environment (IDE) for developing computer software. + +It is developed by JetBrains (formerly known as IntelliJ), and is available as an Apache 2 Licensed community edition, +## 9.1. 宁文君 +and in a proprietary commercial edition. Both can be used for commercial development. + + +2023年1月27日，在这一天我的妈妈宁文君（1953-2023）离世了。她是一个慈祥严格又乐于助人的老太太，自己的退休金虽然不多，但每年也会给贫困山区捐衣物现金。她是支撑我写下这个工具的最大动力，每当我跟她说又有新的公司在用这个工具时，她都和我一样高兴并鼓励我继续维护下去，也一直鼓励我参加各种技术分享活动。虽然我并没有取得多少成就，但她一直为我自豪。可能很多年后宁文君这个名字会被遗忘，但我希望 Github 会再有将数据备份到北极的活动，这样这个名字就会保存一千年。愿逝者安息。 +## 9.4. Redisson + + +## 9.2. YourKit +[Redisson](https://github.com/redisson/redisson), a Redis-based In-Memory Data Grid for Java, offers distributed objects and services (`BitSet`, `Set`, `Multimap`, `SortedSet`, `Map`, `List`, `Queue`, `BlockingQueue`, `Deque`, `BlockingDeque`, `Semaphore`, `Lock`, `AtomicLong`, `CountDownLatch`, `Publish / Subscribe`, `Bloom filter`, `Remote service`, `Spring cache`, `Executor service`, `Live Object service`, `Scheduler service`) backed by a Redis server. Redisson provides a more convenient and easier way to work with Redis. Redisson objects provide a separation of concerns, allowing you to focus on data modeling and application logic. + +![YourKit](https://www.yourkit.com/images/yklogo.png) +YourKit is kindly supporting this open source project with its full-featured Java Profiler. +YourKit, LLC is the creator of innovative and intelligent tools for profiling +Java and .NET applications. Take a look at YourKit's leading software products: +YourKit Java Profiler and +YourKit .NET Profiler. + +## 9.3. IntelliJ IDEA + +[IntelliJ IDEA](https://www.jetbrains.com/?from=redis-replicator) is a Java integrated development environment (IDE) for developing computer software. +It is developed by JetBrains (formerly known as IntelliJ), and is available as an Apache 2 Licensed community edition, +and in a proprietary commercial edition. Both can be used for commercial development. + +## 9.4. Redisson + +[Redisson](https://github.com/redisson/redisson) is Redis based In-Memory Data Grid for Java offers distributed objects and services (`BitSet`, `Set`, `Multimap`, `SortedSet`, `Map`, `List`, `Queue`, `BlockingQueue`, `Deque`, `BlockingDeque`, `Semaphore`, `Lock`, `AtomicLong`, `CountDownLatch`, `Publish / Subscribe`, `Bloom filter`, `Remote service`, `Spring cache`, `Executor service`, `Live Object service`, `Scheduler service`) backed by Redis server. Redisson provides more convenient and easiest way to work with Redis. Redisson objects provides a separation of concern, which allows you to keep focus on the data modeling and application logic. diff --git a/README.zh_CN.md b/README.zh_CN.md index 52a8aec8..bd0c2959 100644 --- a/README.zh_CN.md +++ b/README.zh_CN.md @@ -1,3 +1,6 @@ + + + 内容索引([Table of Contents](./README.md)) =================