猿创征文｜体验新一代分布式数据库—OceanBase

文章目录

• • 一、国产数据库兴起的背景
  • 二、全球数据库排行榜
  • 三、国产数据库排行榜
  • 四、OceanBase介绍
  • 五、OceanBase优势
  • 六、使用 OBD 部署 OceanBase 数据库
  • • 前期准备
    • 步骤1：下载并安装OBD
    • 步骤 2：修改配置文件
    • 步骤 3：部署 OceanBase 数据库
    • 步骤 4：启动 OceanBase 数据库
    • 步骤 5：检查集群运行状态
  • 七、通过OBclient 连接 OceanBase 数据库
  • 八、安装 OBAgent
  • 九、通过 Prometheus 监控数据库
  • 十、参考链接

一、国产数据库兴起的背景

现如今，当我们谈到关键核心技术，自主创新等话题，很多人都会非常关注芯片，但是还有一种IT的核心技术，它的重要性丝毫不在芯片之下，那就是数据库。

中国有十四亿人口，每秒钟都产生海量的数据，从日常的水电费的记录缴纳， 到日常的网络购物，再到时下流行的数字货币交易，乃至物联网技术应用等方面，都跟数据库有着密切的联系，息息相关。由此可见，数据库的处理能力、安全及可靠性等等要求之高可想而知。

数据库的任务就是确保交易过程要高效准确的被记录。尤其是移动支付的普及，金融支付手段的移动化和碎片化，使数据量激增。大量的交易记录和金额，不仅关乎国计民生，还涉及到国家安全和社会稳定。

银行的业务系统经常会在短时间内出现海量高并发的交易需求，那这样的存钱、取钱、贷款等交易过程。属于银行核心交易系统。因此支撑核心交易数据库非常的重要。

近年来，在中国金融科技创新的大环境，给了一批敢于坚持自主创新，探索分布式数据库发展道路的先行者探索的机会。其中，新一代分布式数据库OceanBase就是其中的代表。

近年来，我们已经看到了芯片和操作系统等关键基础技术的创新成果。一批批未来之星正在冉冉升起。正在不断夯实底层技术的创新实力，也给中国其他企业带来丰富的成功经验。

面对未来，中国自主研发仍然需要厚积薄发，更多的IT企业正在走自主创新之路。相信国产数据库的明天会更加美好。

二、全球数据库排行榜

参考链接🔗：https://hellogithub.com/report/db-engines

三、国产数据库排行榜

参考链接🔊：中国数据库排行 - 墨天轮

四、OceanBase介绍

五、OceanBase优势

• 稳定可信赖：经历双 11 共 9 年的验证，蚂蚁集团百万规模使用，单服务器故障自愈，跨城多机房容灾，数据多副本存储，创新推出“三地五中心”城市级容灾新标准，最高可达金融 6 级标准（RPO=0，RTO<=30 秒）。

• 高性能数据：海量数据下， 支持在线事务处理 OLTP 和在线分析处理 OLAP 业务线性扩展， 并相互隔离。

• 更低的存储和运维成本：支持部署运行在 PC 服务器和低端 SSD，高存储压缩率降低存储成本，无中心化设计和多租户混部大幅提升计算资源利用率。

• 兼容MySQL 开源生态：与 MySQL生态高度兼容，提供数据库全生命周期的工具产品，组件化架构全面开放生态，从开发调试到生产运维及数据传输全方位护航。

六、使用 OBD 部署 OceanBase 数据库

实验说明：本实验是基于Anolis OS 8.6 版本系统搭建 OceanBase 数据库。

前期准备

注：若选择使用 Docker 部署 OceanBase 数据库，则最低要求 10G。

步骤1：下载并安装OBD

依次执行以下命令：

# 添加第三方YUM软件源
sudo yum install -y yum-utils

# 添加 OceanBase 的官方软件源
sudo yum-config-manager --add-repo https://mirrors.aliyun.com/oceanbase/OceanBase.repo

# 执行命令安装 OBD
sudo yum install -y ob-deploy

步骤 2：修改配置文件

2.1 下载 OceanBase 快速体验示例配置文件quick_start_demo.yaml

链接：OceanBase 快速体验示例配置文件

# This is a quick start demo config file for ob-deploy
# Please refer to https://github.com/oceanbase/obdeploy/tree/master/example for more examples
# You must change the config item `home_path` to deploy an oceanbase instance
# You can use the shell commands below to deploy an oceanbase instance:
# yum install -y yum-utils
# yum-config-manager --add-repo https://mirrors.aliyun.com/oceanbase/OceanBase.repo
# yum install -y ob-deploy
# obd cluster autodeploy obcluster -c quick_start.yaml

oceanbase-ce:
  servers:
    # Please don't use hostname, only IP can be supported
    - name: 127.0.0.1
      # Please don't use hostname, only IP can be supported
      ip: 127.0.0.1
  global:
    #  The working directory for OceanBase Database. OceanBase Database is started under this directory. This is a required field.
    home_path: /home/xyb/observer
    # The directory for data storage. The default value is $home_path/store.
    # data_dir: /data
    # The directory for clog, ilog, and slog. The default value is the same as the data_dir value.
    # redo_dir: /redo
    # Please set devname as the network adaptor's name whose ip is  in the setting of severs.
    # if set severs as "127.0.0.1", please set devname as "lo"
    # if current ip is 192.168.1.10, and the ip's network adaptor's name is "eth0", please use "eth0"
    devname: lo
    mysql_port: 2881 # External port for OceanBase Database. The default value is 2881. DO NOT change this value after the cluster is started.
    rpc_port: 2882 # Internal port for OceanBase Database. The default value is 2882. DO NOT change this value after the cluster is started.
    zone: zone1
    cluster_id: 1
    # please set memory limit to a suitable value which is matching resource. 
    memory_limit: 8G # The maximum running memory for an observer
    system_memory: 4G # The reserved system memory. system_memory is reserved for general tenants. The default value is 30G.
    stack_size: 512K
    cpu_count: 16
    cache_wash_threshold: 1G
    __min_full_resource_pool_memory: 268435456
    workers_per_cpu_quota: 10
    schema_history_expire_time: 1d
    # The value of net_thread_count had better be same as cpu's core number. 
    net_thread_count: 4
    sys_bkgd_migration_retry_num: 3
    minor_freeze_times: 10
    enable_separate_sys_clog: 0
    enable_merge_by_turn: false
    datafile_size: 5G
    # datafile_disk_percentage: 20 # The percentage of the data_dir space to the total disk space. This value takes effect only when datafile_size is 0. The default value is 90.
    syslog_level: INFO # System log level. The default value is INFO.
    enable_syslog_wf: false # Print system logs whose levels are higher than WARNING to a separate log file. The default value is true.
    enable_syslog_recycle: true # Enable auto system log recycling or not. The default value is false.
    max_syslog_file_count: 4 # The maximum number of reserved log files before enabling auto recycling. The default value is 0.
    # root_password: # root user password, can be empty
    appname: XYBOB

2.2 在配置文件中添加 home_path

oceanbase-ce:
global:
    #  The working directory for OceanBase Database. OceanBase Database is started under this directory. This is a required field.
    home_path: /xyb/observer

步骤 3：部署 OceanBase 数据库

3.1 运行以下命令部署集群：

obd cluster autodeploy XYBOB -c quick_start_demo.yaml -A

[root@Anolis ~]# obd cluster autodeploy XYBOB -c quick_start_demo.yaml -A
oceanbase-ce-3.1.4 already installed.
Cluster param config check ok
Open ssh connection ok
Generate observer configuration ok
oceanbase-ce-3.1.4 already installed.
+-------------------------------------------------------------------------------------------+
|                                          Packages                                         |
+--------------+---------+-----------------------+------------------------------------------+
| Repository   | Version | Release               | Md5                                      |
+--------------+---------+-----------------------+------------------------------------------+
| oceanbase-ce | 3.1.4   | 10000092022071511.el8 | 6e37b476af77b883861e2e334bc0fcb5080a2290 |
+--------------+---------+-----------------------+------------------------------------------+
Repository integrity check ok
Parameter check ok
Open ssh connection ok
Cluster status check ok
Initializes observer work home ok
Remote oceanbase-ce-3.1.4-10000092022071511.el8-6e37b476af77b883861e2e334bc0fcb5080a2290 repository install ok
Remote oceanbase-ce-3.1.4-10000092022071511.el8-6e37b476af77b883861e2e334bc0fcb5080a2290 repository lib check !!
Try to get lib-repository
oceanbase-ce-libs-3.1.4 already installed.
Remote oceanbase-ce-libs-3.1.4-10000092022071511.el8-b2b744a809402059b38ce99265ced33db09860d4 repository install ok
Remote oceanbase-ce-3.1.4-10000092022071511.el8-6e37b476af77b883861e2e334bc0fcb5080a2290 repository lib check ok
XYBOB deployed
Get local repositories ok
Search plugins ok
Open ssh connection ok
Load cluster param plugin ok
Check before start observer ok
[WARN] (127.0.0.1) The recommended value of fs.aio-max-nr is 1048576 (Current value: 65536)
[WARN] (127.0.0.1) clog and data use the same disk (/home)

Start observer ok
observer program health check ok
Connect to observer ok
Initialize cluster
Cluster bootstrap ok
Create tenant test ok
Wait for observer init ok
+---------------------------------------------+
|                   observer                  |
+-----------+---------+------+-------+--------+
| ip        | version | port | zone  | status |
+-----------+---------+------+-------+--------+
| 127.0.0.1 | 3.1.4   | 2881 | zone1 | active |
+-----------+---------+------+-------+--------+

XYBOB running
[root@Anolis ~]#

3.2、运行以下命令查看集群状态：

obd cluster display XYBOB

[root@Anolis ~]# obd cluster display XYBOB
Get local repositories and plugins ok
Open ssh connection ok
Cluster status check ok
Connect to observer ok
Wait for observer init ok
+---------------------------------------------+
|                   observer                  |
+-----------+---------+------+-------+--------+
| ip        | version | port | zone  | status |
+-----------+---------+------+-------+--------+
| 127.0.0.1 | 3.1.4   | 2881 | zone1 | active |
+-----------+---------+------+-------+--------+

[root@Anolis ~]#

步骤 4：启动 OceanBase 数据库

4.1 运行以下命令启动XYBOB集群：

obd cluster start XYBOB

❌报错❌

[root@Anolis ~]# obd cluster start XYBOB
Get local repositories ok
Search plugins ok
Open ssh connection ok
Load cluster param plugin ok
Check before start observer x
[WARN] (127.0.0.1) The recommended value of fs.aio-max-nr is 1048576 (Current value: 65536)
[ERROR] OBD-1007: (127.0.0.1) open files must not be less than 20000 (Current value: 1024)
[WARN] (127.0.0.1) clog and data use the same disk (/)

See https://open.oceanbase.com/docs/obd-cn/V1.4.0/10000000000436999 .

✅解决✅

[root@Anolis ~]# echo 1048576 > /proc/sys/fs/file-max
[root@Anolis ~]# ulimit -HSn 1048576

启动XYBOB集群：

[root@Anolis ~]# obd cluster start XYBOB
Get local repositories ok
Search plugins ok
Open ssh connection ok
Load cluster param plugin ok
Check before start observer ok
[WARN] (127.0.0.1) The recommended value of fs.aio-max-nr is 1048576 (Current value: 65536)
[WARN] (127.0.0.1) clog and data use the same disk (/)

Start observer ok
observer program health check ok
Connect to observer ok
Initialize cluster
Cluster bootstrap ok
Create tenant test ok
Wait for observer init ok
+---------------------------------------------+
|                   observer                  |
+-----------+---------+------+-------+--------+
| ip        | version | port | zone  | status |
+-----------+---------+------+-------+--------+
| 127.0.0.1 | 3.1.4   | 2881 | zone1 | active |
+-----------+---------+------+-------+--------+

XYBOB running
[root@Anolis ~]# yum install -y obclient

[root@Anolis ~]# obd cluster start XYBOB
Get local repositories ok
Search plugins ok
Open ssh connection ok
Load cluster param plugin ok
Cluster status check ok
Deploy "XYBOB" is running

步骤 5：检查集群运行状态

5.1 通过宿主机查看 OBServer 状态

# 登录 OBServer 所在的宿主机。在命令行工具中运行以下命令查看 observer 进程。
[root@Anolis ~]# ps -ef | grep observer
root        1520       1 99 08:25 ?        00:20:14 /home/xyb/observer/bin/observer -r 127.0.0.1:2882:2881 -p 2881 -P 2882 -z zone1 -n XYBOB -c 1 -d /home/xyb/observer/store -i lo -l INFO -o __min_full_resource_pool_memory=268435456,memory_limit=8G,system_memory=4G,stack_size=512K,cpu_count=16,cache_wash_threshold=1G,workers_per_cpu_quota=10,schema_history_expire_time=1d,net_thread_count=4,sys_bkgd_migration_retry_num=3,minor_freeze_times=10,enable_separate_sys_clog=0,enable_merge_by_turn=False,datafile_size=5G,enable_syslog_wf=True,enable_syslog_recycle=True,max_syslog_file_count=4
root        2189    1429  0 08:44 pts/0    00:00:00 grep --color=auto observer

5.2 通过视图查看 OBServer 租户信息

use oceanbase;

select * from gv$tenant;

七、通过OBclient 连接 OceanBase 数据库

7.1 安装 OceanBase 数据库客户端 OBClient：

sudo yum install -y obclient

7.2 运行以下命令，使用 OBClient 客户端连接 OceanBase 数据库：

obclient -h127.0.0.1 -P2881 -uroot@test

SELECT SYSDATE();

八、安装 OBAgent

8.1 安装Go环境

• 下载Go环境Linux版本安装包

[root@Anolis ~]# go version
go version go1.19.2 linux/amd64

安装Go环境具体过程参考此文章： 采用『Prometheus+Grafana』安装部署方式实现对Linux系统主机监控管理

8.2 安装OBAgent

• 下载 obagent-1.1.2-9.el7.x86_64.rpm 包

• 通过RPM包安装 OBAgent

rpm -ivh obagent-1.1.2-9.el7.x86_64.rpm 

[root@Anolis ~]# rpm -ivh obagent-1.1.2-9.el7.x86_64.rpm 
Verifying...                          ################################# [100%]
准备中...                          ################################# [100%]
        软件包 obagent-1.1.2-9.el7.x86_64 已经安装
[root@Anolis ~]# 

九、通过 Prometheus 监控数据库

9.1 下载 Prometheus 软件。

9.2 解压并安装 Prometheus 软件。

tar -zxvf Prometheus-2.30.3.linux-amd64.tar.gz -C /usr/local/

9.3 复制 OBAgent 生成的 prometheus 配置文件到 prometheus 安装目录中。

 mv /home/admin/obagent/conf/prometheus_config/ /usr/local/prometheus-2.39.1.linux-amd64/

9.4 新建 Prometheus 服务文件。

sudo mkdir /var/lib/Prometheus
sudo vim /etc/systemd/system/prometheus.service

[Unit]
Description=Prometheus Monitoring System
Documentation=Prometheus Monitoring System
 
[Service]
ExecStart=/usr/local/prometheus/prometheus --config.file=/usr/local/prometheus/prometheus.yml --storage.tsdb.path=/var/lib/Prometheus --web.enable-lifecycle --web.external-url=http://192.168.200.55:9090
Restart=on-failure

[Install]
WantedBy=multi-user.target

9.5 启动 Prometheus 服务。

sudo systemctl daemon-reload

sudo systemctl start prometheus.service

sudo systemctl status prometheus.service

9.6 查看 prometheus 是否启动

[root@Anolis ~]# netstat -ntlp | grep 9090
tcp6       0      0 :::9090                 :::*                    LISTEN      3772/prometheus

9.7 访问WEB界面，查看数据库告警

十、参考链接

OceanBase-分布式数据库 海量记录 笔笔算数

数据库排行 | HelloGitHub

中国数据库排行 - 墨天轮