一、通过 keepalive+nginx 实现 2台服务器 节点高可用
1.配置 epel 源
在master1、master2

vi  /etc/yum.repos.d/epel.repo

[epel]

name=Extra Packages for Enterprise Linux 7 - $basearch #baseurl=http://download.fedoraproject.org/pub/epel/7/$basearch metalink=https://mirrors.fedoraproject.org/metalink?repo=epel-7&arch=$basearch&infra=$infra&content=$contentdir failovermethod=priority enabled=1 gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7

[epel-debuginfo]

name=Extra Packages for Enterprise Linux 7 - $basearch - Debug #baseurl=http://download.fedoraproject.org/pub/epel/7/$basearch/debug metalink=https://mirrors.fedoraproject.org/metalink?repo=epel-debug-7&arch=$basearch&infra=$infra&content=$contentdir failovermethod=priority enabled=0 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7 gpgcheck=1

[epel-source]

name=Extra Packages for Enterprise Linux 7 - $basearch - Source #baseurl=http://download.fedoraproject.org/pub/epel/7/SRPMS metalink=https://mirrors.fedoraproject.org/metalink?repo=epel-source-7&arch=$basearch&infra=$infra&content=$contentdir failovermethod=priority enabled=0 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7 gpgcheck=1

2.安装 nginx 主备
在master1、master2上执行如下

 yum install nginx keepalived -y

3.修改 nginx 配置文件。主备一样
在master1、master2上执行如下

vim /etc/nginx/nginx.conf
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;
include /usr/share/nginx/modules/*.conf;
events {
worker_connections 1024;
}
# 四层负载均衡，为两台 Master apiserver 组件提供负载均衡
stream {
log_format main '$remote_addr $upstream_addr - [$time_local] $status
$upstream_bytes_sent';
access_log /var/log/nginx/k8s-access.log main;
upstream k8s-apiserver {
server 10.10.1.11:6443; # Master1 APISERVER IP:PORT
server 10.10.1.12:6443; # Master2 APISERVER IP:PORT
}
server {
listen 16443; # 由于 nginx 与 master 节点复用，这个监听端口不能是 6443，否则会冲突
proxy_pass k8s-apiserver;
}
}
http {
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
include /etc/nginx/mime.types;
default_type application/octet-stream;
server {
listen 80 default_server;
server_name _;
location / {
}
}
}

重启全部master1、master2、node1 让selinux关闭生效
reboot

4.keepalive 配置
主 keepalived，在master1上操作

vim /etc/keepalived/keepalived.conf
global_defs {
notification_email {
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id NGINX_MASTER
}
vrrp_script check_nginx {
script "/etc/keepalived/check_nginx.sh"
}
vrrp_instance VI_1 {
state MASTER
interface ens32 # 修改为实际网卡名
virtual_router_id 51 # VRRP 路由 ID 实例，每个实例是唯一的
priority 100 # 优先级，备服务器设置 90
advert_int 1 # 指定 VRRP 心跳包通告间隔时间，默认 1 秒
authentication {
auth_type PASS
auth_pass 1111
}
# 虚拟 IP
virtual_ipaddress {
10.10.1.99/24
}
track_script {
check_nginx
}
}
#vrrp_script：指定检查 nginx 工作状态脚本（根据 nginx 状态判断是否故障转移）
#virtual_ipaddress：虚拟 IP（VIP）

编写监控脚本master1上操作，测试使用

vim /etc/keepalived/check_nginx.sh
#!/bin/bash
count=$(ps -ef |grep nginx | grep sbin | egrep -cv "grep|$$")
if [ "$count" -eq 0 ];then
systemctl stop keepalived
fi

赋予执行权限master1上操作

chmod +x /etc/keepalived/check_nginx.sh

备 keepalived，在master2上操作

vim /etc/keepalived/keepalived.conf
global_defs {
notification_email {
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id NGINX_BACKUP
}
vrrp_script check_nginx {
script "/etc/keepalived/check_nginx.sh"
}
vrrp_instance VI_1 {
state BACKUP
interface ens32
virtual_router_id 51 # VRRP 路由 ID 实例，每个实例是唯一的
priority 90
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.10.1.99/24
}
track_script {
check_nginx
}
}

编写监控脚本master2上操作，测试使用

vim /etc/keepalived/check_nginx.sh
#!/bin/bash
count=$(ps -ef |grep nginx | grep sbin | egrep -cv "grep|$$")
if [ "$count" -eq 0 ];then
systemctl stop keepalived
fi

赋予执行权限master2上操作

chmod +x /etc/keepalived/check_nginx.sh

注：keepalived 根据脚本返回状态码（0 为工作正常，非 0 不正常）判断是否故障转移。

5.启动服务
在master1、master2上执行如下

systemctl daemon-reload
yum install nginx-mod-stream -y
systemctl start nginx
systemctl start keepalived
systemctl enable nginx keepalived
systemctl status keepalived

6.测试 vip 是否绑定成功
master1上操作

ip addr

7.测试 keepalived
停掉 master1 上的 nginx。Vip 会漂移到 master2
在master1上操作

systemctl stop nginx
ip addr
#启动 xianchaomaster1 上的 nginx 和 keepalived，vip 又会漂移回来

systemctl daemon-reload
systemctl start nginx
systemctl start keepalived
ip addr