0
点赞
收藏
分享

微信扫一扫

分库分表之_分库分表 + 复杂查询


前言

Github:​​https://github.com/HealerJean​​

博客:​​http://blog.healerjean.com​​

代码配置暂时和和分库分表之_分库分表相同。但是为了测试下面的join查询将user表的数量调整到了和company表的数量一致,以及给company添加了一个新的字段ref_user_id

1、开始Demo

1.1、hlj-08-sharding_db_table-range_group.sql

drop database if exists ds_0;
create database ds_0 character set 'utf8' collate 'utf8_general_ci';
use ds_0;

drop table if exists user_0;
create table `user_0`
(
id bigint(20) unsigned not null,
city varchar(20) not null default '',
name varchar(20) not null default '',
age int(11) not null default 0,
status int(10) not null default '0' comment '状态',
create_time datetime not null default current_timestamp comment '创建时间',
update_time datetime not null default current_timestamp on update current_timestamp comment '修改时间',
primary key (id)
) engine = innodb
default charset = utf8;

drop table if exists user_1;
create table `user_1`
(
id bigint(20) unsigned not null,
city varchar(20) not null default '',
name varchar(20) not null default '',
age int(11) not null default 0,
status int(10) not null default '0' comment '状态',
create_time datetime not null default current_timestamp comment '创建时间',
update_time datetime not null default current_timestamp on update current_timestamp comment '修改时间',
primary key (id)
) engine = innodb
default charset = utf8;




drop table if exists company_0;
create table `company_0`
(
id bigint(20) unsigned not null comment '主键',
ref_user_id bigint(20) unsigned not null comment '主键',
name varchar(20) not null default '' comment '企业名称',
company_name_english varchar(128) not null default '' comment '企业英文名称',
status int(10) not null default '0' comment '状态',
create_time datetime not null default current_timestamp comment '创建时间',
update_time datetime not null default current_timestamp on update current_timestamp comment '修改时间',

primary key (id)
) engine = innodb
default charset = utf8;

drop table if exists company_1;
create table `company_1`
(
id bigint(20) unsigned not null comment '主键',
ref_user_id bigint(20) unsigned not null comment '主键',
name varchar(20) not null default '' comment '企业名称',
company_name_english varchar(128) not null default '' comment '企业英文名称',
status int(10) not null default '0' comment '状态',
create_time datetime not null default current_timestamp comment '创建时间',
update_time datetime not null default current_timestamp on update current_timestamp comment '修改时间',

primary key (id)
) engine = innodb
default charset = utf8;



CREATE TABLE `demo_entity`
(
id bigint(20) unsigned NOT NULL COMMENT '主键',
`name` varchar(64) NOT NULL,
`phone` varchar(20) DEFAULT '' COMMENT '手机号',
`email` varchar(64) DEFAULT '' COMMENT '邮箱',
`age` int(10) DEFAULT NULL,
`status` varchar(8) NOT NULL COMMENT '状态',
`create_user` bigint(16) unsigned DEFAULT NULL COMMENT '创建人',
`create_name` varchar(64) DEFAULT '' COMMENT '创建人名称',
`create_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
`update_user` bigint(16) unsigned DEFAULT NULL COMMENT '更新人',
`update_name` varchar(64) DEFAULT '' COMMENT '更新人名称',
`update_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '更新时间',
PRIMARY KEY (id)
) ENGINE = InnoDB
DEFAULT CHARSET = utf8;



drop database if exists ds_1;
create database ds_1 character set 'utf8' collate 'utf8_general_ci';
use ds_1;

drop table if exists user_0;
create table `user_0`
(
id bigint(20) unsigned not null,
city varchar(20) not null default '',
name varchar(20) not null default '',
age int(11) not null default 0,
status int(10) not null default '0' comment '状态',
create_time datetime not null default current_timestamp comment '创建时间',
update_time datetime not null default current_timestamp on update current_timestamp comment '修改时间',
primary key (id)
) engine = innodb
default charset = utf8;

drop table if exists user_1;
create table `user_1`
(
id bigint(20) unsigned not null,
city varchar(20) not null default '',
name varchar(20) not null default '',
age int(11) not null default 0,
status int(10) not null default '0' comment '状态',
create_time datetime not null default current_timestamp comment '创建时间',
update_time datetime not null default current_timestamp on update current_timestamp comment '修改时间',
primary key (id)
) engine = innodb
default charset = utf8;



drop table if exists company_0;
create table `company_0`
(
id bigint(20) unsigned not null comment '主键',
ref_user_id bigint(20) unsigned not null comment '主键',
name varchar(20) not null default '' comment '企业名称',
company_name_english varchar(128) not null default '' comment '企业英文名称',
status int(10) not null default '0' comment '状态',
create_time datetime not null default current_timestamp comment '创建时间',
update_time datetime not null default current_timestamp on update current_timestamp comment '修改时间',

primary key (id)
) engine = innodb
default charset = utf8;

drop table if exists company_1;
create table `company_1`
(
id bigint(20) unsigned not null comment '主键',
ref_user_id bigint(20) unsigned not null comment '主键',
name varchar(20) not null default '' comment '企业名称',
company_name_english varchar(128) not null default '' comment '企业英文名称',
status int(10) not null default '0' comment '状态',
create_time datetime not null default current_timestamp comment '创建时间',
update_time datetime not null default current_timestamp on update current_timestamp comment '修改时间',

primary key (id)
) engine = innodb
default charset = utf8;

1.1.1、数据库图文

分库分表之_分库分表 + 复杂查询_主键

1.2、依赖

<!--shardingsphere-->
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>sharding-jdbc-spring-boot-starter</artifactId>
<version>4.0.0-RC1</version>
</dependency>

1.3、配置文件:application.properties

server.port=8888


# 配置 mybatis的一些配置,也可以在 application.properties 中配置,如果配置了就不需要了mybatis.xml
#mybatis-plus.config-location=classpath:mybatis.xml
#Maven 多模块项目的扫描路径需以 classpath*: 开头 (即加载多个 jar 包下的 XML 文件)
mybatis-plus.mapper-locations=classpath*:mapper/*.xml
mybatis-plus.type-aliases-package=com.healerjean.proj.pojo
##主键类型 0:"数据库ID自增,非常大", 1:"用户输入ID(如果用户不输入,则默认是0)",2:"全局唯一ID (数字类型唯一ID)", 3:"全局唯一ID UUID";
mybatis-plus.id-type: 0
#字段策略 0:"忽略判断",1:"非 NULL 判断"),2:"非空判断"
mybatis-plus.field-strategy: 2
#数据库大写下划线转换
mybatis-plus.capital-mode: true
mybatis-plus.refresh-mapper: true


# #当遇到同样名字的时候,是否允许覆盖注册
spring.main.allow-bean-definition-overriding=true
# 显示SQL
spring.shardingsphere.props.sql.show=true



##############################
## 分库分表
#############################
spring.shardingsphere.datasource.names=ds0,ds1

# 数据源
spring.shardingsphere.datasource.ds0.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.ds0.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.ds0.url=jdbc:mysql://localhost:3306/ds_0?serverTimezone=CTT&useUnicode=true&characterEncoding=utf-8&allowMultiQueries=true
spring.shardingsphere.datasource.ds0.username=root
spring.shardingsphere.datasource.ds0.password=123456

spring.shardingsphere.datasource.ds1.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.ds1.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.ds1.url=jdbc:mysql://localhost:3306/ds_1?serverTimezone=CTT&useUnicode=true&characterEncoding=utf-8&allowMultiQueries=true
spring.shardingsphere.datasource.ds1.username=root
spring.shardingsphere.datasource.ds1.password=123456


# 分库配置
#spring.shardingsphere.sharding.default-database-strategy.inline.sharding-column=id
#spring.shardingsphere.sharding.default-database-strategy.inline.algorithm-expression=ds$->{id % 2}
spring.shardingsphere.sharding.default-database-strategy.standard.sharding-column=id
spring.shardingsphere.sharding.default-database-strategy.standard.precise-algorithm-class-name=com.healerjean.proj.config.datasource.CustomShardingDBAlgorithm


# user company 分表
# user_0,user_1,user_2(自定义分表算法)
spring.shardingsphere.sharding.tables.user.actual-data-nodes=ds$->{0..1}.user_$->{0..1}
spring.shardingsphere.sharding.tables.user.table-strategy.standard.sharding-column=id
spring.shardingsphere.sharding.tables.user.table-strategy.standard.precise-algorithm-class-name=com.healerjean.proj.config.datasource.CustomShardingTableAlgorithm
# company_0,company_1 (inline分表策略 表达式 id%2)
spring.shardingsphere.sharding.tables.company.actual-data-nodes=ds$->{0..1}.company_$->{0..1}
spring.shardingsphere.sharding.tables.company.database-strategy.inline.sharding-column=ref_user_id
spring.shardingsphere.sharding.tables.company.database-strategy.inline.algorithm-expression=ds${ref_user_id.longValue() % 2}
spring.shardingsphere.sharding.tables.company.table-strategy.inline.sharding-column=ref_user_id
spring.shardingsphere.sharding.tables.company.table-strategy.inline.algorithm-expression=company_${ref_user_id.longValue() % 2}


## 默认数据源指定(不分库的表)
spring.shardingsphere.sharding.default-data-source-name=ds0

1.4、具体测试方法和类

1.4.1、实体类

1.4.1.1、User.java

@Data
@Accessors(chain = true)
public class User implements Serializable {
private static final long serialVersionUID = 1L;

/** 主键 */
private Long id;
private String name;
private String city;
private Integer age;
private String status;
private Date createTime;
private Date updateTime;
}

1.4.1.2、Company.java

@Data
public class Company {

private Long id;
private Long refUserId;
private String name;
private String companyNameEnglish;
private String status;
private Date createTime;
private Date updateTime;
}

1.4.1.3、DemoEntity.java

@Data
@Accessors(chain = true)
public class DemoEntity implements Serializable {
private static final long serialVersionUID = 1L;

private Long id;
private String name;
private String phone;
private String email;
private Integer age;
private String status;
private Long createUser;
private String createName;
private java.util.Date createTime;
private Long updateUser;
private String updateName;
private java.util.Date updateTime;

}

1.4.1.4、新对象 UserRefCompany

@Data
public class UserRefCompany {

/** 主键 */
private Long userId;
private String name;
private String city;
private String status;

private Long companyId;
private String companyName;
private String companyNameEnglish;


private Integer avgAge;
private Integer sumAge;


}

1.4.2、DTO数据

1.4.2.1、UserDTO.java

@Data
@Accessors(chain = true)
@ApiModel(value = "demo实体类")
@JsonInclude(JsonInclude.Include.NON_NULL)
public class UserDTO {


@ApiModelProperty(value = "主键", hidden = true)
@JsonSerialize(using = JsonLongSerializer.class )
private Long id;

@ApiModelProperty(value = "姓名")
@NotBlank(message = "姓名不能为空", groups = ValidateGroup.HealerJean.class)
private String name;

@ApiModelProperty(value = "城市")
private String city;

@ApiModelProperty(value = "年龄")
private Integer age;


@ApiModelProperty(value = "状态", hidden = true)
private String status;


@ApiModelProperty(value = "创建时间", hidden = true)
@JsonFormat(pattern = DateUtils.YYYY_MM_dd_HH_mm_ss, timezone = "GMT+8")
private Date createTime;

@ApiModelProperty(value = "修改时间", hidden = true)
@JsonFormat(pattern = DateUtils.YYYY_MM_dd_HH_mm_ss, timezone = "GMT+8")
private Date updateTime;

private Integer pageNow ;
private Integer pageSize ;

}

1.4.2.2、CompanyDTO.java

@Data
public class CompanyDTO {

@JsonSerialize(using = JsonLongSerializer.class)
private Long id;

private Long refUserId;

private String name;
private String companyNameEnglish;
private String status;


@ApiModelProperty(value = "创建时间", hidden = true)
@JsonFormat(pattern = DateUtils.YYYY_MM_dd_HH_mm_ss, timezone = "GMT+8")
private Date createTime;

@ApiModelProperty(value = "修改时间", hidden = true)
@JsonFormat(pattern = DateUtils.YYYY_MM_dd_HH_mm_ss, timezone = "GMT+8")
private Date updateTime;
}

1.4.2.3、DemoDTO.java

@Data
@Accessors(chain = true)
@ApiModel(value = "demo实体类")
@JsonInclude(JsonInclude.Include.NON_NULL)
public class DemoDTO extends PageQuery {

@JsonSerialize(using = JsonLongSerializer.class )
private Long id;

@ApiModelProperty(value = "姓名")
@NotBlank(message = "姓名不能为空", groups = ValidateGroup.HealerJean.class)
private String name;

@ApiModelProperty(value = "年龄")
private Integer age;

@ApiModelProperty(value = "手机号")
private String phone;

@ApiModelProperty(value = "邮箱")
private String email;

@ApiModelProperty(value = "是否删除,10可用,99删除 ", hidden = true)
private String status;

@ApiModelProperty(value = "创建人", hidden = true)
private Long createUser;

@ApiModelProperty(value = "创建人名字", hidden = true)
private String createName;

@ApiModelProperty(value = "创建时间", hidden = true)
private java.util.Date createTime;

@ApiModelProperty(value = "更新人", hidden = true)
private Long updateUser;

@ApiModelProperty(value = "更新人名称", hidden = true)
private String updateName;

@ApiModelProperty(hidden = true)
private java.util.Date updateTime;

}

1.4.3、Mapper

1.4.3.1、UserMapper.java

public interface UserMapper extends BaseMapper<User> {


}

1.4.3.1、CompanyMapper.java

public interface CompanyMapper  extends BaseMapper<Company> {


}

1.4.3.1、DemoEntityMapper.java

public interface DemoEntityMapper extends BaseMapper<DemoEntity> {

}

1.4.4、Service

1.4.4.1、 UserService.java

public interface UserService {


UserDTO insert(UserDTO userDTO);

UserDTO findById(Long id);

List<UserDTO> list();

}

1.4.4.2、 CompanyService.java

public interface CompanyService {


CompanyDTO insert(CompanyDTO companyDTO);

CompanyDTO findById(Long id);

List<CompanyDTO> list();
}

1.4.4.3、 DemoEntityService.java

public interface DemoEntityService {


DemoDTO insert(DemoDTO demoEntity);

DemoDTO findById(Long id);

List<DemoDTO> list();

}

1.4.5、ServiceImpl.java

1.4.5.1、UserServiceImpl.java

@Service
@Slf4j
public class UserServiceImpl implements UserService {

@Resource
private UserMapper userMapper;


@Override
public UserDTO insert(UserDTO userDTO) {
User user = BeanUtils.dtoToUserDTO(userDTO);
user.setStatus(StatusEnum.生效.code);
userMapper.insert(user);
userDTO.setId(user.getId());
return userDTO;
}

@Override
public UserDTO findById(Long id) {
User user = userMapper.selectById(id);
return user == null ? null : BeanUtils.userToDTO(user);
}

@Override
public List<UserDTO> list() {
List<User> users = userMapper.selectList(null);
List<UserDTO> list = null;
if (!EmptyUtil.isEmpty(users)) {
list = users.stream().map(BeanUtils::userToDTO).collect(Collectors.toList());
}
return list;
}

}

1.4.5.2、CompanyServiceImpl.java

@Service
public class CompanyServiceImpl implements CompanyService {

@Resource
private CompanyMapper companyMapper;

@Override
public CompanyDTO insert(CompanyDTO companyDTO) {
Company company = BeanUtils.dtoToCompany(companyDTO);
company.setStatus(StatusEnum.生效.code);
companyMapper.insert(company);
companyDTO.setId(company.getId());
return companyDTO;
}

@Override
public CompanyDTO findById(Long id) {
Company company = companyMapper.selectById(id);
return company == null ? null : BeanUtils.companyToDTO(company);
}

@Override
public List<CompanyDTO> list() {
List<Company> companys = companyMapper.selectList(null);
List<CompanyDTO> list = null;
if (!EmptyUtil.isEmpty(companys)) {
list = companys.stream().map(BeanUtils::companyToDTO).collect(Collectors.toList());
}
return list;
}
}

1.4.5.3、DemoEntityServiceImpl.java

@Service
@Slf4j
public class DemoEntityServiceImpl implements DemoEntityService {


@Resource
private DemoEntityMapper demoEntityMapper;

@Resource
private CompanyService companyService;
@Resource
private UserService userService;

@Override
public DemoDTO insert(DemoDTO demoDTO) {
DemoEntity demoEntity = BeanUtils.dtoToDemo(demoDTO);
demoEntity.setStatus(StatusEnum.生效.code);
demoEntityMapper.insert(demoEntity);
demoDTO.setId(demoEntity.getId());
return demoDTO;
}


@Override
public DemoDTO findById(Long id) {
DemoEntity demoEntity = demoEntityMapper.selectById(id);
return demoEntity == null ? null : BeanUtils.demoToDTO(demoEntity);
}

@Override
public List<DemoDTO> list() {
List<DemoDTO> collect = null;
List<DemoEntity> list = demoEntityMapper.selectList(null);
if (!EmptyUtil.isEmpty(list)) {
collect = list.stream().map(BeanUtils::demoToDTO).collect(Collectors.toList());
}
return collect;
}


}

1.4.6、Controller

1.4.6.1、UserController.java

@ApiResponses(value = {
@ApiResponse(code = 200, message = "访问正常"),
@ApiResponse(code = 301, message = "逻辑错误"),
@ApiResponse(code = 500, message = "系统错误"),
@ApiResponse(code = 401, message = "未认证"),
@ApiResponse(code = 403, message = "禁止访问"),
@ApiResponse(code = 404, message = "url错误")
})
@Api(description = "demo控制器")
@Controller
@RequestMapping("hlj/demo")
@Slf4j
public class UserController {



@Autowired
private UserService userService;

@ApiOperation(value = "insert",
notes = "insert",
consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE,
produces = MediaType.APPLICATION_JSON_VALUE,
response = UserDTO.class)
@PostMapping(value = "insert", produces = "application/json; charset=utf-8")
@ResponseBody
public ResponseBean insert(UserDTO userDTO) {
log.info("样例--------mybaits-plus添加demo实体------数据信息{}", userDTO);
String validate = ValidateUtils.validate(userDTO, ValidateGroup.HealerJean.class);
if (!validate.equals(CommonConstants.COMMON_SUCCESS)) {
throw new BusinessException(ResponseEnum.参数错误, validate);
}
return ResponseBean.buildSuccess(userService.insert(userDTO));
}


@ApiOperation(notes = "findById",
value = "findById",
consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE,
produces = MediaType.APPLICATION_JSON_VALUE,
response = UserDTO.class)
@ApiImplicitParams({
@ApiImplicitParam(name = "id", value = "demo主键", required = true, paramType = "path", dataType = "long"),
})
@GetMapping("findById/{id}")
@ResponseBody
public ResponseBean findById(@PathVariable Long id) {
log.info("样例--------findById------数据:id:{}", id);
return ResponseBean.buildSuccess(userService.findById(id));
}

@ApiOperation(notes = "list",
value = "list",
consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE,
produces = MediaType.APPLICATION_JSON_VALUE,
response = UserDTO.class)
@GetMapping("list")
@ResponseBody
public ResponseBean list() {
log.info("样例--------list------");
return ResponseBean.buildSuccess(userService.list());
}


}

1.4.6.2、CompanyController.java

@ApiResponses(value = {
@ApiResponse(code = 200, message = "访问正常"),
@ApiResponse(code = 301, message = "逻辑错误"),
@ApiResponse(code = 500, message = "系统错误"),
@ApiResponse(code = 401, message = "未认证"),
@ApiResponse(code = 403, message = "禁止访问"),
@ApiResponse(code = 404, message = "url错误")
})
@Api(description = "demo控制器")
@Controller
@RequestMapping("hlj/company")
@Slf4j
public class CompanyController {


@Autowired
private CompanyService companyService;

@ApiOperation(value = "insert",
notes = "insert",
consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE,
produces = MediaType.APPLICATION_JSON_VALUE,
response = UserDTO.class)
@PostMapping(value = "insert", produces = "application/json; charset=utf-8")
@ResponseBody
public ResponseBean insert(CompanyDTO companyDTO) {
log.info("user--------insert------请求参数:{}", companyDTO);
return ResponseBean.buildSuccess(companyService.insert(companyDTO));
}


@ApiOperation(notes = "findById",
value = "findById",
consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE,
produces = MediaType.APPLICATION_JSON_VALUE,
response = UserDTO.class)
@ApiImplicitParams({
@ApiImplicitParam(name = "id", value = "demo主键", required = true, paramType = "path", dataType = "long"),
})
@GetMapping("findById/{id}")
@ResponseBody
public ResponseBean findById(@PathVariable Long id) {
log.info("company--------findById------id:{}", id);
return ResponseBean.buildSuccess(companyService.findById(id));
}



@ApiOperation(notes = "list",
value = "list",
consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE,
produces = MediaType.APPLICATION_JSON_VALUE,
response = UserDTO.class)
@GetMapping("list")
@ResponseBody
public ResponseBean list() {
log.info("company--------list------");
return ResponseBean.buildSuccess(companyService.list());
}

}

1.4.6.3、DemoController.java

@ApiResponses(value = {
@ApiResponse(code = 200, message = "访问正常"),
@ApiResponse(code = 301, message = "逻辑错误"),
@ApiResponse(code = 500, message = "系统错误"),
@ApiResponse(code = 401, message = "未认证"),
@ApiResponse(code = 403, message = "禁止访问"),
@ApiResponse(code = 404, message = "url错误")
})
@Api(description = "demo控制器")
@Controller
@RequestMapping("hlj/demo")
@Slf4j
public class DemoController {

@Autowired
private DemoEntityService demoEntityService;


@ApiOperation(value = "insert",
notes = "insert",
consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE,
produces = MediaType.APPLICATION_JSON_VALUE,
response = UserDTO.class)
@PostMapping(value = "insert", produces = "application/json; charset=utf-8")
@ResponseBody
public ResponseBean insert(DemoDTO demoDTO) {
log.info("demo--------insert------请求参数:{}", demoDTO);
return ResponseBean.buildSuccess(demoEntityService.insert(demoDTO));
}


@ApiOperation(notes = "findById",
value = "findById",
consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE,
produces = MediaType.APPLICATION_JSON_VALUE,
response = UserDTO.class)
@ApiImplicitParams({
@ApiImplicitParam(name = "id", value = "demo主键", required = true, paramType = "path", dataType = "long"),
})
@GetMapping("findById/{id}")
@ResponseBody
public ResponseBean findById(@PathVariable Long id) {
log.info("demo--------findById------id:{}", id);
return ResponseBean.buildSuccess(demoEntityService.findById(id));
}



@ApiOperation(notes = "list",
value = "list",
consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE,
produces = MediaType.APPLICATION_JSON_VALUE,
response = UserDTO.class)
@GetMapping("list")
@ResponseBody
public ResponseBean list() {
log.info("demo--------list------");
return ResponseBean.buildSuccess(demoEntityService.list());
}

}

1.4.6、自定义分表算法 CustomShardingTableAlgorithm

@Slf4j
public class CustomShardingTableAlgorithm implements PreciseShardingAlgorithm<Long> {

@Override
public String doSharding(Collection<String> availableTargetNames, PreciseShardingValue<Long> shardingValue) {
for (String tableName : availableTargetNames) {
if (tableName.endsWith(shardingValue.getValue() % 3 + "")) {
log.info("表为:{}, 主键为:{}, 最终被分到的表为:{}", availableTargetNames, shardingValue, tableName);
return tableName;
}
}
throw new IllegalArgumentException();
}
}

1.4.7、自定义分库算法:CustomShardingDBAlgorithm

@Slf4j
public class CustomShardingDBAlgorithm implements PreciseShardingAlgorithm<Long> {


@Override
public String doSharding(Collection<String> availableTargetNames, PreciseShardingValue<Long> shardingValue) {
for (String dbName : availableTargetNames) {
if (dbName.endsWith(shardingValue.getValue() % 2 + "")) {
log.info("库为:{}, 主键为:{}, 最终被分到的库为:【{}】", availableTargetNames, shardingValue, dbName);
return dbName;
}
}
throw new IllegalArgumentException();
}
}

2、开始测试

基本的增删改成就不测试了,我们这里主要是测试一些分库后可能出问题的一些语句

2.1、limit查询:成功

2.1.1、代码如下

2.1.1.1、​​UserService.java​

public interface UserService {

List<UserDTO> limit(UserDTO userDTO);

}

2.1.1.2、​​UserServiceImpl.java​

@Service
@Slf4j
public class UserServiceImpl implements UserService {


/**
* limit 成功
*/
@Override
public List<UserDTO> limit(UserDTO userDTO) {
Wrapper<User> userWrapper = new QueryWrapper<User>().lambda()
.orderByDesc(User::getCreateTime)
.last("limit " + userDTO.getPageNow() + ", " + userDTO.getPageSize());
List<User> users = userMapper.selectList(userWrapper);
List<UserDTO> list = null;
if (!EmptyUtil.isEmpty(users)) {
list = users.stream().map(BeanUtils::userToDTO).collect(Collectors.toList());
}
return list;
}


}

2.1.1.3、​​UserController.java​

@Controller
@RequestMapping("hlj/user")
@Slf4j
public class UserController {

@ApiOperation(notes = "limit",
value = "limit",
consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE,
produces = MediaType.APPLICATION_JSON_VALUE,
response = UserDTO.class)
@GetMapping("limit")
@ResponseBody
public ResponseBean limit(UserDTO userDTO) {
log.info("demo--------limit------");
return ResponseBean.buildSuccess(userService.limit(userDTO));
}


}

2.1.2、访问测试

现在所有的User表数据汇总:

name

city

age

createTime

a

北京

1

2020-03-30 18:15:14

b

北京

2

2020-03-30 18:16:14

c

山西

3

2020-03-30 18:16:20

d

山西

4

2020-03-30 18:16:27

e

河北

5

2020-03-30 18:16:38

f

江苏

6

2020-03-30 18:16:46

2.1.2.1、访问:pageNow:0,pageSize:2

{
"success": true,
"result": [
{
"id": "1244569246752800769",
"name": "f",
"city": "江苏",
"age": 6,
"status": "10",
"createTime": "2020-03-30 18:16:46",
"updateTime": "2020-03-30 18:16:46"
},
{
"id": "1244569214813175810",
"name": "e",
"city": "河北",
"age": 5,
"status": "10",
"createTime": "2020-03-30 18:16:38",
"updateTime": "2020-03-30 18:16:38"
}
],
"msg": "",
"code": 200,
"date": "1585563469572"
}

控制台sql日志

# 日志汇总
Logic SQL:
SELECT id,name,city,age,status,create_time,update_time FROM user
ORDER BY create_time DESC limit 0, 2



# 数据库日志
Actual SQL: ds0 :::
SELECT id,name,city,age,status,create_time,update_time FROM user_0
ORDER BY create_time DESC limit 0, 2 ShardingSphere-SQL.log[89]

Actual SQL: ds0 :::
SELECT id,name,city,age,status,create_time,update_time FROM user_1
ORDER BY create_time DESC limit 0, 2

Actual SQL: ds1 :::
SELECT id,name,city,age,status,create_time,update_time FROM user_0
ORDER BY create_time DESC limit 0, 2

Actual SQL: ds1 :::
SELECT id,name,city,age,status,create_time,update_time FROM user_1
ORDER BY create_time DESC limit 0, 2

2.1.2.2、访问:pageNow:3,pageSize:1

{
"success": true,
"result": [
{
"id": "1244569141396078594",
"name": "c",
"city": "山西",
"age": 3,
"status": "10",
"createTime": "2020-03-30 18:16:20",
"updateTime": "2020-03-30 18:16:20"
}
],
"msg": "",
"code": 200,
"date": "1585563506891"
}

控制台sql日志

# 日志汇总
Actual SQL:
SELECT id,name,city,age,status,create_time,update_time FROM user ORDER BY create_time DESC limit 3, 1


# 数据库日志
Actual SQL: ds0 :::
SELECT id,name,city,age,status,create_time,update_time FROM user_0 ORDER BY create_time DESC limit 0, 4

Actual SQL: ds0 :::
SELECT id,name,city,age,status,create_time,update_time FROM user_1 ORDER BY create_time DESC limit 0, 4

Actual SQL: ds1 :::
SELECT id,name,city,age,status,create_time,update_time FROM user_0 ORDER BY create_time DESC limit 0, 4

Actual SQL: ds1 :::
SELECT id,name,city,age,status,create_time,update_time FROM user_1 ORDER BY create_time DESC limit 0, 4

2.1.3、归纳总结:

成功,可以看到分表向每个数据库的每个表中发送了sql语句

2.2、groupBy + 函数 :成功

2.2.1、代码如下

2.2.1.1、​​UserMapper.java​

public interface UserMapper extends BaseMapper<User> {


List<UserRefCompany> groupByCity();
}

2.2.1.2、​​UserMapper.xml​

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd" >
<mapper namespace="com.healerjean.proj.dao.mapper.UserMapper">


<select id="groupByCity" resultType="com.healerjean.proj.pojo.UserRefCompany">
select city , sum(age) as sumAge, avg(age) as avgAge from user group by city
</select>


</mapper>

2.2.1.3、​​UserService.java​

@Service
@Slf4j
public class UserServiceImpl implements UserService {

/**
* groupBy 成功
*/
@Override
public List<UserRefCompany> group() {
List<UserRefCompany> list = userMapper.groupByCity();
return list;
}

}

2.2.1.4、​​UserController.java​

@Controller
@RequestMapping("hlj/user")
@Slf4j
public class UserController {

@ApiOperation(notes = "group",
value = "group",
consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE,
produces = MediaType.APPLICATION_JSON_VALUE,
response = UserDTO.class)
@GetMapping("group")
@ResponseBody
public ResponseBean group() {
log.info("demo--------group------");
return ResponseBean.buildSuccess(userService.group());
}


}

2.1.2、测试

现在所有的User表数据汇总:

name

city

age

a

北京

1

b

北京

2

c

山西

3

d

山西

4

e

河北

5

f

江苏

6

2.1.2.1、访问测试

{
"success": true,
"result": [
{
"userId": null,
"name": null,
"city": "北京",
"status": null,
"companyId": null,
"companyName": null,
"companyNameEnglish": null,
"avgAge": 1,
"sumAge": 3
},
{
"userId": null,
"name": null,
"city": "山西",
"status": null,
"companyId": null,
"companyName": null,
"companyNameEnglish": null,
"avgAge": 3,
"sumAge": 7
},
{
"userId": null,
"name": null,
"city": "江苏",
"status": null,
"companyId": null,
"companyName": null,
"companyNameEnglish": null,
"avgAge": 6,
"sumAge": 6
},
{
"userId": null,
"name": null,
"city": "河北",
"status": null,
"companyId": null,
"companyName": null,
"companyNameEnglish": null,
"avgAge": 5,
"sumAge": 5
}
],
"msg": "",
"code": 200,
"date": "1585563964870"
}

控制台日志

Logic SQL: 
select city , sum(age) as sumAge, avg(age) as avgAge from user group by city






Actual SQL: ds0 :::
select city , sum(age) as sumAge, avg(age) as avgAge , COUNT(age) AS AVG_DERIVED_COUNT_0 , SUM(age) AS AVG_DERIVED_SUM_0 from user_0 group by city ORDER BY city ASC ShardingSphere-SQL.log[89]

Actual SQL: ds0 :::
select city , sum(age) as sumAge, avg(age) as avgAge , COUNT(age) AS AVG_DERIVED_COUNT_0 , SUM(age) AS AVG_DERIVED_SUM_0 from user_1 group by city ORDER BY city ASC

Actual SQL: ds1 :::
select city , sum(age) as sumAge, avg(age) as avgAge , COUNT(age) AS AVG_DERIVED_COUNT_0 , SUM(age) AS AVG_DERIVED_SUM_0 from user_0 group by city ORDER BY city ASC

Actual SQL: ds1 :::
select city , sum(age) as sumAge, avg(age) as avgAge , COUNT(age) AS AVG_DERIVED_COUNT_0 , SUM(age) AS AVG_DERIVED_SUM_0 from user_1 group by city ORDER BY city ASC

2.1.3、归纳总结:

成功,可以看到分表向每个数据库的每个表中发送了sql语句 (取出数据后,自动讲里面获取的数字进行计算)

2.3、​​between and​​ :成功

2.3.1、代码如下

2.3.1.1、​​UserService.java​

public interface UserService {

List<UserDTO> between();

}

2.3.1.2、​​UserServiceImpl.java​

@Service
@Slf4j
public class UserServiceImpl implements UserService {


/**
* between 成功
*/
@Override
public List<UserDTO> between() {
Wrapper<User> userWrapper = new QueryWrapper<User>().lambda()
.between(User::getAge, 1, 3)
.orderByDesc(User::getCreateTime);
List<User> users = userMapper.selectList(userWrapper);
List<UserDTO> list = null;
if (!EmptyUtil.isEmpty(users)) {
list = users.stream().map(BeanUtils::userToDTO).collect(Collectors.toList());
}
return list;
}


}

2.3.1.3、​​UserController.java​

@Controller
@RequestMapping("hlj/user")
@Slf4j
public class UserController {


@ApiOperation(notes = "between",
value = "group",
consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE,
produces = MediaType.APPLICATION_JSON_VALUE,
response = UserDTO.class)
@GetMapping("between")
@ResponseBody
public ResponseBean between() {
log.info("demo--------between------");
return ResponseBean.buildSuccess(userService.between());
}



}

2.3.2、测试

现在所有的User表数据汇总:

name

city

age

createTime

a

北京

1

2020-03-30 18:15:14

b

北京

2

2020-03-30 18:16:14

c

山西

3

2020-03-30 18:16:20

d

山西

4

2020-03-30 18:16:27

e

河北

5

2020-03-30 18:16:38

f

江苏

6

2020-03-30 18:16:46

2.3.2.1、访问测试 :between 1 and 3 orderby createTime desc

{
"success": true,
"result": [
{
"id": "1244569141396078594",
"name": "c",
"city": "山西",
"age": 3,
"status": "10",
"createTime": "2020-03-30 18:16:20",
"updateTime": "2020-03-30 18:16:20"
},
{
"id": "1244569114653196290",
"name": "b",
"city": "北京",
"age": 2,
"status": "10",
"createTime": "2020-03-30 18:16:14",
"updateTime": "2020-03-30 18:16:14"
},
{
"id": "1244568862747492353",
"name": "a",
"city": "北京",
"age": 1,
"status": "10",
"createTime": "2020-03-30 18:15:14",
"updateTime": "2020-03-30 18:15:14"
}
],
"msg": "",
"code": 200,
"date": "1585564394319"
}

2.3.3、归纳总结:

成功,可以看到分表向每个数据库的每个表中发送了sql语句

2.2、​​left join​​: 连接查询

2.2.1、代码如下

2.2.1.1、​​UserMapper.java​

public interface UserMapper extends BaseMapper<User> {


List<UserRefCompany> leftJoin();

}

2.2.1.2、​​UserMapper.xml​

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd" >
<mapper namespace="com.healerjean.proj.dao.mapper.UserMapper">


<select id="leftJoin" resultType="com.healerjean.proj.pojo.UserRefCompany">
select u.id,
u.name ,
c.id as companyId,
c.name as companyName,
c.ref_user_id as refUserId,
c.company_name_english as companyNameEnglish
from user u left join company c on u.id = c .ref_user_id
order by u.create_time
</select>


</mapper>

2.2.1.3、​​UserService.java​

@Service
@Slf4j
public class UserServiceImpl implements UserService {

/**
* left join 设置了绑定关系后成功
*/
@Override
public List<UserRefCompany> leftJoin() {
List<UserRefCompany> userRefCompanies = userMapper.leftJoin();
return userRefCompanies;
}

}

2.2.1.4、​​UserController.java​

@Controller
@RequestMapping("hlj/user")
@Slf4j
public class UserController {

@ApiOperation(notes = "leftJoin",
value = "leftJoin",
consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE,
produces = MediaType.APPLICATION_JSON_VALUE,
response = UserDTO.class)
@GetMapping("leftJoin")
@ResponseBody
public ResponseBean leftJoin() {
log.info("demo--------leftJoin------");
return ResponseBean.buildSuccess( userService.leftJoin());
}


}

2.2.2、访问测试:

现在所有的User表数据汇总:

id

name

city

age

1244568862747492353

a

北京

1

1244569114653196290

b

北京

2

1244569141396078594

c

山西

3

1244569167870525442

d

山西

4

1244569214813175810

e

河北

5

1244569246752800769

f

江苏

6

现在所有的company表数据汇总、添加这个数据的时候,其实我们已经做了一些小动作了,company表的分库分表是按照ref_user_id进行分库的(和user表中id的分库策略一致),这样就保证了后面绑定关系之后查询的成功

id

ref_user_id

name

company_name_english

create_time

1244575632182181889

1244568862747492353

北京小米有限公司

xiaomi

2020-03-30 18:42:08

1244575692508856321

1244569114653196290

北京字节跳动

zijie

2020-03-30 18:42:22

1244575754915905538

1244569141396078594

山西爱酷科技

Iku

2020-03-30 18:42:37

1244575817167765506

1244569214813175810

河北腾讯

tecent

2020-03-30 18:42:52

2.2.2.1、默认失败

{
"success": true,
"result": [
{
"userId": null,
"name": "a",
"city": null,
"status": null,
"companyId": 1244585691742093300,
"companyName": "北京小米有限公司",
"companyNameEnglish": "xiaomi",
"avgAge": null,
"sumAge": null
},
{
"userId": null,
"name": "a",
"city": null,
"status": null,
"companyId": null,
"companyName": null,
"companyNameEnglish": null,
"avgAge": null,
"sumAge": null
},
{
"userId": null,
"name": "b",
"city": null,
"status": null,
"companyId": 1244585869291176000,
"companyName": "北京字节跳动",
"companyNameEnglish": "zijie",
"avgAge": null,
"sumAge": null
},
{
"userId": null,
"name": "b",
"city": null,
"status": null,
"companyId": null,
"companyName": null,
"companyNameEnglish": null,
"avgAge": null,
"sumAge": null
},
{
"userId": null,
"name": "c",
"city": null,
"status": null,
"companyId": 1244585918465196000,
"companyName": "山西爱酷科技",
"companyNameEnglish": "Iku",
"avgAge": null,
"sumAge": null
},
{
"userId": null,
"name": "c",
"city": null,
"status": null,
"companyId": null,
"companyName": null,
"companyNameEnglish": null,
"avgAge": null,
"sumAge": null
},
{
"userId": null,
"name": "d",
"city": null,
"status": null,
"companyId": null,
"companyName": null,
"companyNameEnglish": null,
"avgAge": null,
"sumAge": null
},
{
"userId": null,
"name": "d",
"city": null,
"status": null,
"companyId": null,
"companyName": null,
"companyNameEnglish": null,
"avgAge": null,
"sumAge": null
},
{
"userId": null,
"name": "e",
"city": null,
"status": null,
"companyId": 1244585970768167000,
"companyName": "河北腾讯",
"companyNameEnglish": "tecent",
"avgAge": null,
"sumAge": null
},
{
"userId": null,
"name": "e",
"city": null,
"status": null,
"companyId": null,
"companyName": null,
"companyNameEnglish": null,
"avgAge": null,
"sumAge": null
},
{
"userId": null,
"name": "f",
"city": null,
"status": null,
"companyId": null,
"companyName": null,
"companyNameEnglish": null,
"avgAge": null,
"sumAge": null
},
{
"userId": null,
"name": "f",
"city": null,
"status": null,
"companyId": null,
"companyName": null,
"companyNameEnglish": null,
"avgAge": null,
"sumAge": null
}
],
"msg": "",
"code": 200,
"date": "1585567537634"
}

控制台日志(整理了下),可以看到下面的查询类似于卡迪尔积,每个x和每个y数组里面的分别join了一次,所以导致了错误的出现,

要查询的结果
Logic SQL: select u.id,
u.name ,
c.id as companyId,
c.name as companyName,
c.ref_user_id as refUserId,
c.company_name_english as companyNameEnglish
from user u
left join company c on u.id = c .ref_user_id ShardingSphere-SQL.log[89]





分库查询
Actual SQL: ds0 :::
select ^^^ from user_1 u left join company_1 c on u.id = c .ref_user_id
select ^^^ from user_1 u left join company_0 c on u.id = c .ref_user_id
select ^^^ from user_0 u left join company_1 c on u.id = c .ref_user_id
select ^^^ from user_0 u left join company_0 c on u.id = c .ref_user_id


Actual SQL: ds1 :::
select ^^^ from user_1 u left join company_1 c on u.id = c .ref_user_id
select ^^^ from user_1 u left join company_0 c on u.id = c .ref_user_id
select ^^^ from user_0 u left join company_1 c on u.id = c .ref_user_id
select ^^^ from user_0 u left join company_0 c on u.id = c .ref_user_id

2.2.3、配置user和company的绑定关系 :成功

如果要保证一对一的表关系,必须保证2点

1、user和company的表数量以及所在节点要完全一模一样

2、从表的的外键规则和主表的外键分库分表策略要一直

# user  company 分表
# user_0,user_1,user_2(自定义分表算法)
spring.shardingsphere.sharding.tables.user.actual-data-nodes=ds$->{0..1}.user_$->{0..1}
spring.shardingsphere.sharding.tables.user.table-strategy.standard.sharding-column=id
spring.shardingsphere.sharding.tables.user.table-strategy.standard.precise-algorithm-class-name=com.healerjean.proj.config.datasource.CustomShardingTableAlgorithm
# company_0,company_1 (inline分表策略 表达式 id%2)
spring.shardingsphere.sharding.tables.company.actual-data-nodes=ds$->{0..1}.company_$->{0..1}
spring.shardingsphere.sharding.tables.company.database-strategy.inline.sharding-column=ref_user_id
spring.shardingsphere.sharding.tables.company.database-strategy.inline.algorithm-expression=ds${ref_user_id.longValue() % 2}
spring.shardingsphere.sharding.tables.company.table-strategy.inline.sharding-column=ref_user_id
spring.shardingsphere.sharding.tables.company.table-strategy.inline.algorithm-expression=company_${ref_user_id.longValue() % 2}


# 绑定表规则列表
spring.shardingsphere.sharding.binding-tables[0]=user,company

2.2.3.1、访问测试

{
"success": true,
"result": [
{
"userId": null,
"name": "a",
"city": null,
"status": null,
"companyId": 1244585691742093300,
"companyName": "北京小米有限公司",
"companyNameEnglish": "xiaomi",
"avgAge": null,
"sumAge": null
},
{
"userId": null,
"name": "b",
"city": null,
"status": null,
"companyId": 1244585869291176000,
"companyName": "北京字节跳动",
"companyNameEnglish": "zijie",
"avgAge": null,
"sumAge": null
},
{
"userId": null,
"name": "c",
"city": null,
"status": null,
"companyId": 1244585918465196000,
"companyName": "山西爱酷科技",
"companyNameEnglish": "Iku",
"avgAge": null,
"sumAge": null
},
{
"userId": null,
"name": "d",
"city": null,
"status": null,
"companyId": null,
"companyName": null,
"companyNameEnglish": null,
"avgAge": null,
"sumAge": null
},
{
"userId": null,
"name": "e",
"city": null,
"status": null,
"companyId": 1244585970768167000,
"companyName": "河北腾讯",
"companyNameEnglish": "tecent",
"avgAge": null,
"sumAge": null
},
{
"userId": null,
"name": "f",
"city": null,
"status": null,
"companyId": null,
"companyName": null,
"companyNameEnglish": null,
"avgAge": null,
"sumAge": null
}
],
"msg": "",
"code": 200,
"date": "1585567592799"
}

控制台日志

要查询的结果
Logic SQL: select u.id,
u.name ,
c.id as companyId,
c.name as companyName,
c.ref_user_id as refUserId,
c.company_name_english as companyNameEnglish
from user u
left join company c on u.id = c .ref_user_id ShardingSphere-SQL.log[89]





分库查询
Actual SQL: ds0 :::
select ^^^ from user_1 u left join company_1 c on u.id = c .ref_user_id
select ^^^ from user_0 u left join company_0 c on u.id = c .ref_user_id


Actual SQL: ds1 :::
select ^^^ from user_1 u left join company_1 c on u.id = c .ref_user_id
select ^^^ from user_0 u left join company_0 c on u.id = c .ref_user_id

2.3、默认数据源,配合数据源看

默认数据源指定(不分库的表)

默认数据源指定(不分库的表)
spring.shardingsphere.sharding.default-data-source-name=ds0

2.4、配置广播表

为了防止跨库查,可以将那些永远不会改变的表。在每个库中复制一份。一般适用于配置类的数据,(如果插入数据,会向所有的数据库同时进行插入)

# 配置广播表(为了防止跨库查,可以将那些永远不会改变的表。在每个库中复制一份。一般适用于配置类的数据,(如果插入数据,会向所有的数据库同时进行插入)
spring.shardingsphere.sharding.broadcast-tables=demo_entity

分库分表之_分库分表 + 复杂查询_spring_02


举报

相关推荐

0 条评论