上面是我们一个应用的基本架构,域名在CloudFlare上解析,部分域名启用了proxied,部分则没有启用。
直接访问域名,看看两个Nginx上的日志里的IP情况呢
首先确保两个Nginx的http下server里加了如下配置:
http下添加日志格式定义:
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
作为proxy的Nginx配置:
set $andytest "andytest-backend-999.lb-2.us-east";
location / {
proxy_pass http://$andytest:9001;
proxy_redirect off;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header HTTP_X_REAL_IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
server下添加日志:
access_log /var/log/nginx/andytest.access.log main;
最早在http下添加的如下不生效:
access_log /var/log/nginx/access.log;
应该是没用上定义的main的缘故,我对Nginx的配置其实不是很懂哈。
Nginx A的日志:
172.28.3.230 - - [06/Feb/2024:00:57:30 +0000] "GET /index.html9001 HTTP/1.1" 404 187 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" "103.198.119.149"
172.28.3.230 - - [06/Feb/2024:00:57:54 +0000] "GET /index.html2095 HTTP/1.1" 404 187 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" "103.198.119.149, 172.68.225.94"
103.198.119.149是我的公司出口IP,172.28.3.230应该是aws elb-1的实例IP
9001没有CloudFlare的proxied,但是经过了aws的lb-1
2095经过了CloudFlare的proxy,同时还经过了aws的lb-1,所以最后的IP多了一个CloudFlare的IP:172.68.225.94
Nginx B的日志:
172.31.23.162 - - [06/Feb/2024:00:57:30 +0000] "GET /index.html9001 HTTP/1.1" 404 555 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/5
37.36" "103.198.119.149, 172.28.3.230, 53.132.220.196"
172.31.23.162 - - [06/Feb/2024:00:57:54 +0000] "GET /index.html2095 HTTP/1.1" 404 555 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/5
37.36" "103.198.119.149, 172.68.225.94, 172.28.3.230, 53.132.220.196"
53.132.220.196是Nginx A的ec2 EIP,172.31.23.162应该是aws elb-2的实例IP
这次的IP在原来的IP后面加上了aws elb-1的IP和Nginx A的EIP
如果在CloudFlare上新建A记录直接指向 53.132.220.196 ,而不是cname到其上的aws elb-1,再看看访问日志:
Nginx A的日志:
103.198.119.149 - - [06/Feb/2024:01:35:11 +0000] "GET /index.html90010 HTTP/1.1" 404 187 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" "-"
172.71.218.44 - - [06/Feb/2024:01:35:47 +0000] "GET /index.html20950 HTTP/1.1" 404 187 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" "103.198.119.149"
172.71.218.44是CloudFlare的另外一个IP。但是这俩IP都不是bluxxxx.xyz启用proxied后解析出来的那俩CloudFlare IP。
Nginx B的日志:
172.31.23.162 - - [06/Feb/2024:01:35:11 +0000] "GET /index.html90010 HTTP/1.1" 404 555 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" "103.198.119.149, 53.132.220.196"
172.31.62.121 - - [06/Feb/2024:01:35:47 +0000] "GET /index.html20950 HTTP/1.1" 404 555 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" "103.198.119.149, 172.71.218.44, 53.132.220.196"
172.31.62.121应该是aws elb-2的另一个高可用实例的IP,这里可以看到没经过aws elb-1,后面的proxy_add_x_forwarded_for就没有172.28.xx.xx的IP了。
Nginx A和Nginx B都在us-east-1,但是在不同的aws账号下。
搞清楚这些IP信息后,对于后续想在ELB上绑定AWS WAF的rate limit时到底要选哪个IP就比较好理解了。