学大数据需要java学到什么程度

参考文档：
http://postgres.cn/docs/14/app-pgverifybackup.html
通过pg_verifybackup来验证备份的文件是否正常。
pg_verifybackup用于根据备份时服务器生成的backup_manifest检查使用pg_basebackup进行的数据库群集备份的完整性。备份必须以“普通”格式存储；“tar”格式的备份可以在解压缩后进行检查。

无自建立的表空间的情况
--对数据库进行备份，输出为平面文件，也就是使用和源服务器数据目录和表空间相同的布局

postgres@pg15:/postgres/pg15/backups$ pg_basebackup -Fp -Pv -Xf  -p 5432 -U postgres -D /postgres/pg15/backups/
pg_basebackup: initiating base backup, waiting for checkpoint to complete
pg_basebackup: checkpoint completed
pg_basebackup: write-ahead log start point: 0/60000028 on timeline 2
91684/91684 kB (100%), 1/1 tablespace                                         
pg_basebackup: write-ahead log end point: 0/60000100
pg_basebackup: syncing data to disk ...
pg_basebackup: renaming backup_manifest.tmp to backup_manifest
pg_basebackup: base backup completed

-- 生成的备份文件，和源服务器上的文件一样 

postgres@pg15:/postgres/pg15/backups$ ls
backup_label      base.tar          pg_commit_ts          pg_ident.conf  pg_replslot   pg_stat_tmp     pg_twophase  postgresql.auto.conf
backup_label.old  current_logfiles  pg_dynshmem           pg_logical     pg_serial     pg_subtrans     PG_VERSION   postgresql.conf
backup_manifest   global            pg_hba.conf           pg_multixact   pg_snapshots  pg_tablespaces  pg_wal       postgresql.conf20230328
base              log               pg_hba.conf_20230328  pg_notify      pg_stat       pg_tblspc       pg_xact      tablespace_map.old
postgres@pg15:/postgres/pg15/backups$ pwd
/postgres/pg15/backups
postgres@pg15:/postgres/pg15/backups$ 

-- 使用pg_verifybackup 进行验证，验证成功

pg_verifybackup -m /postgres/pg15/backups/backup_manifest /postgres/pg15/backups/

postgres@pg15:/postgres/pg15/backups$ pg_verifybackup -m /postgres/pg15/backups/backup_manifest /postgres/pg15/backups/
backup successfully verified
postgres@pg15:/postgres/pg15/backups$

-- 存在自建立的表空间的情况，新建的表空间为test_tbs 
--在备份的时候，提示表空间所在的路径不为空 ，查看发现确实有新建立的表空间

postgres@pg15:~$ pg_basebackup -Fp -Pv -Xf  -p 5432 -U postgres -D /postgres/pg15/backups/
pg_basebackup: initiating base backup, waiting for checkpoint to complete
pg_basebackup: checkpoint completed
pg_basebackup: write-ahead log start point: 0/51000028 on timeline 1
pg_basebackup: error: directory "/postgres/pg15/data/pg_tablespaces" exists but is not empty   
pg_basebackup: removing contents of data directory "/postgres/pg15/backups/"
postgres@pg15:~$ 

postgres=# \db+
                                               List of tablespaces
    Name    |  Owner   |              Location              | Access privileges | Options |  Size  | Description 
------------+----------+------------------------------------+-------------------+---------+--------+-------------
 pg_default | postgres |                                    |                   |         | 36 MB  | 
 pg_global  | postgres |                                    |                   |         | 531 kB | 
 test_tbs   | postgres | /postgres/pg15/data/pg_tablespaces |                   |         | 42 MB  | 
(3 rows)

postgres=# 

-- 解决该问题，可以使用以下参数
-T olddir=newdir
--tablespace-mapping=olddir=newdir

-- 带上参数-T 来进行备份，发现提示backup的目录不为空（把newdir建立在了backups目录下）

pg_basebackup -Fp -Pv -Xf  -p 5432 -U postgres -D /postgres/pg15/backups/ -T /postgres/pg15/data/pg_tablespaces=/postgres/pg15/backups/new_tbs

postgres@pg15:/postgres/pg15/backups/new_tbs$ pg_basebackup -Fp -Pv -Xf  -p 5432 -U postgres -D /postgres/pg15/backups/ -T /postgres/pg15/data/pg_tablespaces=/postgres/pg15/backups/new_tbs
pg_basebackup: error: directory "/postgres/pg15/backups/" exists but is not empty   <<<< 备份目录不能有内容 
postgres@pg15:/postgres/pg15/backups/new_tbs$ 

-- 重新指定newdir的目录，使其不在backups目录下，可以备份

pg_basebackup -Fp -Pv -Xf  -p 5432 -U postgres -D /postgres/pg15/backups/ -T /postgres/pg15/data/pg_tablespaces=/postgres/pg15/new_tbs

postgres@pg15:/postgres/pg15/new_tbs$ pg_basebackup -Fp -Pv -Xf  -p 5432 -U postgres -D /postgres/pg15/backups/ -T /postgres/pg15/data/pg_tablespaces=/postgres/pg15/new_tbs
pg_basebackup: initiating base backup, waiting for checkpoint to complete
pg_basebackup: checkpoint completed
pg_basebackup: write-ahead log start point: 0/52000028 on timeline 1
97298/97298 kB (100%), 2/2 tablespaces                                         
pg_basebackup: write-ahead log end point: 0/52000100
pg_basebackup: syncing data to disk ...
pg_basebackup: renaming backup_manifest.tmp to backup_manifest
pg_basebackup: base backup completed
postgres@pg15:/postgres/pg15/new_tbs$ 


postgres@pg15:/postgres/pg15/new_tbs$ ls
PG_15_202209061
postgres@pg15:/postgres/pg15/new_tbs$ pwd
/postgres/pg15/new_tbs
postgres@pg15:/postgres/pg15/new_tbs$ 

postgres@pg15:/postgres/pg15/backups$ ls
backup_label      global        pg_hba.conf           pg_multixact  pg_snapshots  pg_tablespaces  pg_wal                postgresql.conf20230328
backup_manifest   log           pg_hba.conf_20230328  pg_notify     pg_stat       pg_tblspc       pg_xact
base              pg_commit_ts  pg_ident.conf         pg_replslot   pg_stat_tmp   pg_twophase     postgresql.auto.conf
current_logfiles  pg_dynshmem   pg_logical            pg_serial     pg_subtrans   PG_VERSION      postgresql.conf
postgres@pg15:/postgres/pg15/backups$ pwd
/postgres/pg15/backups
postgres@pg15:/postgres/pg15/backups$ 

-- 使用pg_verifybackup验证备份，验证成功 

pg_verifybackup -m /postgres/pg15/backups/backup_manifest /postgres/pg15/backups/

postgres@pg15:/postgres/pg15/backups$ pg_verifybackup -m /postgres/pg15/backups/backup_manifest /postgres/pg15/backups/
backup successfully verified
postgres@pg15:/postgres/pg15/backups$ 

-- 测试，当使用tar的方式备份的时候，如何使用pg_verifybackup来验证

pg_basebackup -D /postgres/pg15/backups/ -Fp -Pv -U postgres -h 192.168.2.101 -p5432 -R

postgres@pg15:/postgres$ sudo mkdir -p backup20230506
[sudo] password for postgres:         
postgres@pg15:/postgres$ ls
backup20230506  pg15
postgres@pg15:/postgres$ 

-- 开始备份，以tar的方式。注意下面两个语句的区别，
第二个语句需要修改pg_hba.conf，生成的tar是3个（base、表空间、wal）。
第一个语句加了Xf参数，不需要修改pg_hba.conf，生成的tar是2个（base和表空间，其中wal包含在base目录中）

pg_basebackup -Fp -Pv -Xf  -p 5432 -U postgres -D /postgres/pg15/backups/ 
pg_basebackup -D /postgres/backup20230506 -Ft -Pv -U postgres -h 192.168.2.101 -p5432 -R

pg_basebackup -Ft -Pv -Xf  -p 5432 -U postgres -D /postgres/backup20230506

postgres@pg15:/postgres$ pg_basebackup -Ft -Pv -Xf  -p 5432 -U postgres -D /postgres/backup20230506
pg_basebackup: initiating base backup, waiting for checkpoint to complete
pg_basebackup: checkpoint completed
pg_basebackup: write-ahead log start point: 0/54000028 on timeline 1
97299/97299 kB (100%), 2/2 tablespaces                                         
pg_basebackup: write-ahead log end point: 0/54000100
pg_basebackup: syncing data to disk ...
pg_basebackup: renaming backup_manifest.tmp to backup_manifest
pg_basebackup: base backup completed
postgres@pg15:/postgres$ 

postgres@pg15:/postgres/backup20230506$ ls
16389.tar  backup_manifest  base.tar
postgres@pg15:/postgres/backup20230506$ 

--解压后，开始pg_verifybackup验证 ，测试过多次，失败，报错。
报错信息：
pg_verifybackup: error: "PG_15_202209061/16388/2754" is present on disk but not in the manifest
pg_verifybackup: error: "pg_tablespaces/PG_15_202209061/16388/24628_fsm" is present on disk but not in the manifest

解决方法1 ： 设置软链接，问题仍然存在 

ln -s /postgres/backup20230506/pg_tablespaces  /postgres/backup20230506/pg_tblspc/16389 

postgres@pg15:/postgres/backup20230506/pg_tblspc$ ls -l /postgres/backup20230506/pg_tblspc/16389
lrwxrwxrwx 1 postgres postgres 39 May  6 14:58 /postgres/backup20230506/pg_tblspc/16389 -> /postgres/backup20230506/pg_tablespaces
postgres@pg15:/postgres/backup20230506/pg_tblspc$ 

查看backup_manifest的内容，发现是扫描pg_tblspc下的内容

"Path": "pg_tblspc/16389/PG_15_202209061/16388/2608_fsm", "Size": 24576, "Last-Modified": "2023-04-21 03:02:42 GMT", "Checksum-Algorithm": "CRC32C", "Checksum": "5ae3c854" },

而报错信息，是pg_tablespaces/PG_15_202209061/16388/24628_fsm" is present on disk but not in the manifest
而运行的数据库上pg_tblspc下面是个软链接，而不是实实在在的文件。所以这个是问题所在

解决方法2 
将备份的表空间的内容，放在pg_tblspc下，而不是放在pg_tablespaces。 即，pg_tblspc下有数据，pg_tablespaces下清空。刚好和数据库相反。
数据库是pg_tblspc下是软连接，有数据，，pg_tablespaces下有实实在在的数据。

---通过方法2处理后，问题解决。 或者，在解压表空间的备份文件的时候，直接解压在pg_tblspc，而不是解压在pg_tablespaces。

pg_verifybackup -m /home/postgres/backup_manifest /postgres/backup20230506


postgres@pg15:~$ pg_verifybackup -m /home/postgres/backup_manifest /postgres/backup20230506/
backup successfully verified
postgres@pg15:~$ 

END