棘手问题
项目交接
- 从乙方SVN上将代码和文档拷贝下来，发现项目有三个，暂且称为ABC三个项目，还有部分原型图片和第三方服务配置信息。
- 项目刚开始以为是springboot+jpa+angularjs项目，AB两个项目虽然都是maven项目并且功能点类似，对接的三方服务不同而已，C项目是提供的额外的API项目，可以理解为只有springMVC的接口没有页面的项目。
- 启动项目的时候发现AB虽然代码有90%类似，但是并未复用代码，这就为今后的维护带来很大的隐患了，没办法保证两个项目的公共部分同步，每次都依靠手工去修改。
- C项目是一个普通的JavaEE项目，没用maven做jar包依赖管理，而且编译的时候还不通过，后来检查发现，在项目build中依赖了target（target原本为项目编译目标文件夹，而此文件夹目前除了项目编译的class文件等还有一个额外的文件夹可以理解为是部分的main文件夹里的java）目录下的一个文件夹，这种配置方式让我无比头疼，最终为了项目好交接和维护，给改成maven项目。
项目启动
- 项目传到测试环境，首先就是无法启动了，或者启动的环境配置不对。
Exception in thread "main" java.lang.UnsupportedClassVersionError: picard/cmdline/PicardCommandLine : Unsupported major.minor version 52.0
项目版本及上线
-
再来说说项目中静态文件的问题（灰色敏感信息，你懂的）
线上问题
- 日志首先暴露了一个"Broken pipe"的问题，乙方解答是打开的文件说超过服务器的默认值1024，修改服务器设置，增加进程可打开的文件句柄数量。问题仍然存在。
- 文件的事情发现了原因，但是项目运行半天后又发生了连接被拒绝的问题，log提示线程池队列已满，拒绝入队请求。
- 寻找JPA连接数据库死锁原因,使用如下语句查找数据库死锁的源头
SELECT DISTINCT b.trx_id blocking_trx_id,
b.trx_mysql_thread_id 源头锁thread_id,
SUBSTRING(p. HOST, 1, INSTR(p. HOST, ':') - 1) blocking_host,
SUBSTRING(p. HOST, INSTR(p. HOST, ':') + 1) blocking_port,
IF(p.COMMAND = 'Sleep', p.TIME, 0) idel_in_trx,
b.trx_query blocking_query,
r.trx_id waiting_trx_id,
r.trx_mysql_thread_id waiting_thread,
TIMESTAMPDIFF(SECOND, r.trx_wait_started, CURRENT_TIMESTAMP) wait_time,
r.trx_query waiting_query,
l.lock_table waiting_table_lock
FROM information_schema.INNODB_LOCKS l
LEFT JOIN information_schema.INNODB_LOCK_WAITS w
ON w.requested_lock_id = l.lock_id
LEFT JOIN information_schema.INNODB_TRX b
ON b.trx_id = w.blocking_trx_id
LEFT JOIN information_schema.INNODB_TRX r
ON r.trx_id = w.requesting_trx_id
LEFT JOIN information_schema. PROCESSLIST p
ON p.ID = b.trx_mysql_thread_id
JOIN (SELECT blocking_trx_id -- 查找最源头的trx_id
FROM information_schema.INNODB_LOCK_WAITS ilw
WHERE blocking_trx_id NOT IN
(SELECT requesting_trx_id
FROM information_schema.INNODB_LOCK_WAITS)) c
ON c.blocking_trx_id = b.trx_id
ORDER BY wait_time DESC;
=====================================
2018-04-26 18:34:07 7f8ac91fb700 INNODB MONITOR OUTPUT
=====================================
Per second averages calculated from the last 17 seconds
-----------------
BACKGROUND THREAD
-----------------
srv_master_thread loops: 13501269 srv_active, 0 srv_shutdown, 15670842 srv_idle
srv_master_thread log flush and writes: 29170711
----------
SEMAPHORES
----------
OS WAIT ARRAY INFO: reservation count 7603696
OS WAIT ARRAY INFO: signal count 16033072
Mutex spin waits 117932426, rounds 141446289, OS waits 979385
RW-shared spins 8219797, rounds 187794144, OS waits 5726533
RW-excl spins 1287414, rounds 41555159, OS waits 839372
Spin rounds per wait: 1.20 mutex, 22.85 RW-shared, 32.28 RW-excl
------------------------
LATEST FOREIGN KEY ERROR
------------------------
2018-04-25 21:26:22 7f8aa8ae7700 Transaction:
TRANSACTION 71107923, ACTIVE 0 sec updating or deleting
mysql tables in use 1, locked 1
9 lock struct(s), heap size 2936, 5 row lock(s), undo log entries 1
MySQL thread id 1700944, OS thread handle 0x7f8aa8ae7700, query id 1293657675 111.230.128.98 cdb_outerroot updating
delete from TUser where id=1875151
Foreign key constraint fails for table `dawanka`.`Event`:
,
CONSTRAINT `FK6wrxw84b767pp74tpxwjy8ky7` FOREIGN KEY (`user_id`) REFERENCES `TUser` (`id`)
Trying to delete or update in parent table, in index `PRIMARY` tuple:
DATA TUPLE: 45 fields;
0: len 8; hex 80000000001c9ccf; asc ;;
1: len 6; hex 0000043d0553; asc = S;;
2: len 7; hex 2b00001295110d; asc + ;;
3: len 1; hex 00; asc ;;
4: len 1; hex 00; asc ;;
5: len 1; hex 01; asc ;;
6: len 1; hex 00; asc ;;
7: len 4; hex 80000000; asc ;;
8: len 11; hex 41204d616e2773204d616e; asc A Man's Man;;
9: SQL NULL;
10: len 1; hex 00; asc ;;
11: SQL NULL;
12: SQL NULL;
13: SQL NULL;
14: len 11; hex 3135363139383031333536; asc 15619801356;;
15: SQL NULL;
16: SQL NULL;
17: SQL NULL;
18: len 1; hex 00; asc ;;
19: len 5; hex 999ef480dd; asc ;;
20: len 1; hex 00; asc ;;
21: len 1; hex 01; asc ;;
22: len 39; hex 77785f35303838616665632d323234642d343635612d616634342d333636663436333531353865; asc wx_5088afec-224d-465a-af44-366f4635158e;;
23: len 131; hex 687474703a2f2f77782e716c6f676f2e636e2f6d6d6f70656e2f58654f4c516e55374b74545637674e744970704e335269636c38415744774673307431666b32534d7242445456476963503577496c71616b63543264726172744f61527178584a5a696264397170547569615a6963666a6b354f724959315655365945484a2f313332; asc http://wx.qlogo.cn/mmopen/XeOLQnU7KtTV7gNtIppN3Ricl8AWDwFs0t1fk2SMrBDTVGicP5wIlqakcT2drartOaRqxXJZibd9qpTuiaZicfjk5OrIY1VU6YEHJ/132;;
24: len 0; hex ; asc ;;
25: SQL NULL;
26: SQL NULL;
27: SQL NULL;
28: SQL NULL;
29: SQL NULL;
30: len 8; hex 8000000000000000; asc ;;
31: len 9; hex 800000000000000000; asc ;;
32: SQL NULL;
33: SQL NULL;
34: SQL NULL;
35: SQL NULL;
36: SQL NULL;
37: SQL NULL;
38: SQL NULL;
39: SQL NULL;
40: SQL NULL;
41: SQL NULL;
42: SQL NULL;
43: SQL NULL;
44: SQL NULL;
*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 1043 page no 76968 n bits 128 index `PRIMARY` of table `dawanka`.`TUser` trx id 71385898 lock_mode X locks rec but not gap waiting
Record lock, heap no 45 PHYSICAL RECORD: n_fields 45; compact format; info bits 0
*** (2) TRANSACTION:
TRANSACTION 71385898, ACTIVE 6838 sec starting index read
mysql tables in use 1, locked 1
10 lock struct(s), heap size 2936, 9 row lock(s), undo log entries 2
MySQL thread id 1707165, OS thread handle 0x7f8aa2196700, query id 1295789031 111.230.128.98 cdb_outerroot updating
update TUser set ***(隐私信息) where id=2128399
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 1043 page no 76968 n bits 120 index `PRIMARY` of table `dawanka`.`TUser` trx id 71385898 lock mode S locks rec but not gap
Record lock, heap no 45 PHYSICAL RECORD: n_fields 45; compact format; info bits 0
- 优化方案：
(1) 删掉数据库所有外键，并由程序控制数据逻辑
(2) 修改hibernate配置文件，防止项目启动，自动被加外键" spring.jpa.hibernate.ddl-auto=disable "
总结问题
- 代码交接不全面，文档未整理，存在较多的滞后信息和遗漏信息；
- 项目代码SVN版本管理未做；
- 各环境变量分离未做到；
- 各环境的环境变量保持一致未做到；
- 前后端分离的项目硬生生的合并到tomcat中，未研究讨论前者的设计用途擅自更改架构设计；
- 并发问题根本原因在于数据库死锁，而需要将外键删除，并修改hibernate配置文件，不适用外键的数据库设计；
PS:其实我们团队是这整个项目中的丙方，替乙方填坑的团队。
后续问题，还在排查中... ...
