1. 环境准备与基础配置
在开始构建高可用Kubernetes集群之前，我们需要做好充分的环境准备。我建议使用3台及以上配置相同的CentOS 7/8或Ubuntu 20.04 LTS服务器，每台至少4核CPU、8GB内存和50GB磁盘空间。在实际项目中，我曾遇到过内存不足导致kubelet频繁崩溃的情况，所以资源预留很关键。
首先在所有节点上执行基础系统配置：
# 关闭防火墙 systemctl stop firewalld && systemctl disable firewalld # 关闭SELinux setenforce 0 sed -i 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config # 关闭swap swapoff -a sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab # 设置时区 timedatectl set-timezone Asia/Shanghai # 配置hosts cat >> /etc/hosts <<EOF 192.168.1.101 k8s-master1 192.168.1.102 k8s-master2 192.168.1.103 k8s-master3 192.168.1.201 k8s-node1 192.168.1.202 k8s-node2 EOF内核参数调优是很多教程会忽略的部分，但这些配置对集群稳定性至关重要：
cat > /etc/sysctl.d/k8s.conf <<EOF net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 net.ipv4.ip_forward = 1 vm.swappiness = 0 vm.overcommit_memory = 1 vm.panic_on_oom = 0 fs.inotify.max_user_instances = 8192 fs.inotify.max_user_watches = 1048576 fs.file-max = 52706963 fs.nr_open = 52706963 net.ipv6.conf.all.disable_ipv6 = 1 net.netfilter.nf_conntrack_max = 2310720 EOF sysctl -p /etc/sysctl.d/k8s.conf2. 证书体系设计与签发
Kubernetes重度依赖TLS证书进行组件间通信加密，手动部署时需要特别注意证书的合理规划。我采用Cloudflare的cfssl工具链来生成证书，相比openssl更易用。
首先创建CA证书（在第一个Master节点操作）：
mkdir -p /etc/kubernetes/pki cd /etc/kubernetes/pki cat > ca-config.json <<EOF { "signing": { "default": { "expiry": "87600h" }, "profiles": { "kubernetes": { "usages": ["signing", "key encipherment", "server auth", "client auth"], "expiry": "87600h" } } } } EOF cat > ca-csr.json <<EOF { "CN": "Kubernetes", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "ST": "Shanghai", "L": "Shanghai", "O": "Kubernetes", "OU": "CA" } ] } EOF cfssl gencert -initca ca-csr.json | cfssljson -bare ca接下来为API Server生成证书，特别注意要包含所有可能的访问方式：
cat > kube-apiserver-csr.json <<EOF { "CN": "kube-apiserver", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "ST": "Shanghai", "L": "Shanghai", "O": "Kubernetes", "OU": "Kubernetes" } ] } EOF cfssl gencert \ -ca=ca.pem \ -ca-key=ca-key.pem \ -config=ca-config.json \ -hostname=10.96.0.1,192.168.1.101,192.168.1.102,192.168.1.103,127.0.0.1,kubernetes,kubernetes.default,kubernetes.default.svc,kubernetes.default.svc.cluster.local \ -profile=kubernetes \ kube-apiserver-csr.json | cfssljson -bare kube-apiserver其他组件证书类似方式生成后，需要将CA证书和生成的证书同步到所有节点：
for node in k8s-master2 k8s-master3 k8s-node1 k8s-node2; do rsync -avz /etc/kubernetes/pki/ca* $node:/etc/kubernetes/pki/ rsync -avz /etc/kubernetes/pki/kube-apiserver* $node:/etc/kubernetes/pki/ # 同步其他必要证书... done3. 核心组件部署与配置
3.1 etcd集群部署
etcd作为Kubernetes的后端存储，必须确保高可用。我采用3节点集群部署：
# 下载etcd二进制包 ETCD_VER=v3.5.0 wget https://github.com/etcd-io/etcd/releases/download/${ETCD_VER}/etcd-${ETCD_VER}-linux-amd64.tar.gz tar xvf etcd-${ETCD_VER}-linux-amd64.tar.gz mv etcd-${ETCD_VER}-linux-amd64/{etcd,etcdctl} /usr/local/bin/ # 创建systemd服务文件 cat > /etc/systemd/system/etcd.service <<EOF [Unit] Description=etcd Documentation=https://github.com/coreos/etcd [Service] Type=notify ExecStart=/usr/local/bin/etcd \\ --name=k8s-master1 \\ --data-dir=/var/lib/etcd \\ --initial-advertise-peer-urls=https://192.168.1.101:2380 \\ --listen-peer-urls=https://192.168.1.101:2380 \\ --listen-client-urls=https://192.168.1.101:2379,https://127.0.0.1:2379 \\ --advertise-client-urls=https://192.168.1.101:2379 \\ --initial-cluster-token=etcd-cluster \\ --initial-cluster=k8s-master1=https://192.168.1.101:2380,k8s-master2=https://192.168.1.102:2380,k8s-master3=https://192.168.1.103:2380 \\ --initial-cluster-state=new \\ --client-cert-auth \\ --trusted-ca-file=/etc/kubernetes/pki/ca.pem \\ --cert-file=/etc/kubernetes/pki/etcd.pem \\ --key-file=/etc/kubernetes/pki/etcd-key.pem \\ --peer-client-cert-auth \\ --peer-trusted-ca-file=/etc/kubernetes/pki/ca.pem \\ --peer-cert-file=/etc/kubernetes/pki/etcd.pem \\ --peer-key-file=/etc/kubernetes/pki/etcd-key.pem Restart=on-failure RestartSec=5 [Install] WantedBy=multi-user.target EOF启动etcd后验证集群健康状态：
ETCDCTL_API=3 etcdctl \ --endpoints=https://192.168.1.101:2379 \ --cacert=/etc/kubernetes/pki/ca.pem \ --cert=/etc/kubernetes/pki/etcd.pem \ --key=/etc/kubernetes/pki/etcd-key.pem \ endpoint health3.2 控制平面组件部署
Kubernetes控制平面包含kube-apiserver、kube-controller-manager和kube-scheduler三个核心组件。我采用二进制方式部署：
# 下载Kubernetes二进制文件 K8S_VER=v1.25.0 wget https://dl.k8s.io/${K8S_VER}/kubernetes-server-linux-amd64.tar.gz tar xvf kubernetes-server-linux-amd64.tar.gz cd kubernetes/server/bin cp kube-apiserver kube-controller-manager kube-scheduler kubectl /usr/local/bin/ # 配置kube-apiserver cat > /etc/systemd/system/kube-apiserver.service <<EOF [Unit] Description=Kubernetes API Server Documentation=https://kubernetes.io/docs/concepts/overview/ [Service] ExecStart=/usr/local/bin/kube-apiserver \\ --advertise-address=192.168.1.101 \\ --allow-privileged=true \\ --apiserver-count=3 \\ --audit-log-maxage=30 \\ --audit-log-maxbackup=3 \\ --audit-log-maxsize=100 \\ --audit-log-path=/var/log/kubernetes/audit.log \\ --authorization-mode=Node,RBAC \\ --bind-address=0.0.0.0 \\ --client-ca-file=/etc/kubernetes/pki/ca.pem \\ --enable-admission-plugins=NodeRestriction \\ --enable-bootstrap-token-auth=true \\ --etcd-cafile=/etc/kubernetes/pki/ca.pem \\ --etcd-certfile=/etc/kubernetes/pki/apiserver-etcd-client.pem \\ --etcd-keyfile=/etc/kubernetes/pki/apiserver-etcd-client-key.pem \\ --etcd-servers=https://192.168.1.101:2379,https://192.168.1.102:2379,https://192.168.1.103:2379 \\ --kubelet-client-certificate=/etc/kubernetes/pki/apiserver-kubelet-client.pem \\ --kubelet-client-key=/etc/kubernetes/pki/apiserver-kubelet-client-key.pem \\ --service-account-key-file=/etc/kubernetes/pki/sa.pub \\ --service-account-signing-key-file=/etc/kubernetes/pki/sa.key \\ --service-account-issuer=https://kubernetes.default.svc.cluster.local \\ --service-cluster-ip-range=10.96.0.0/12 \\ --tls-cert-file=/etc/kubernetes/pki/kube-apiserver.pem \\ --tls-private-key-file=/etc/kubernetes/pki/kube-apiserver-key.pem \\ --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.pem \\ --proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.pem \\ --proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client-key.pem \\ --requestheader-allowed-names=front-proxy-client \\ --requestheader-extra-headers-prefix=X-Remote-Extra- \\ --requestheader-group-headers=X-Remote-Group \\ --requestheader-username-headers=X-Remote-User Restart=on-failure RestartSec=5 [Install] WantedBy=multi-user.target EOF其他控制平面组件配置类似，启动后可以通过kubectl检查状态：
kubectl get componentstatuses4. 高可用与负载均衡实现
生产环境必须确保控制平面的高可用性。我采用HAProxy+Keepalived方案实现API Server的负载均衡：
# 安装HAProxy yum install -y haproxy # 配置HAProxy cat > /etc/haproxy/haproxy.cfg <<EOF global log /dev/log local0 log /dev/log local1 notice chroot /var/lib/haproxy stats socket /run/haproxy/admin.sock mode 660 level admin stats timeout 30s user haproxy group haproxy daemon defaults log global mode http option httplog option dontlognull timeout connect 5000 timeout client 50000 timeout server 50000 frontend k8s-api bind *:6443 mode tcp default_backend k8s-api backend k8s-api mode tcp balance roundrobin server k8s-master1 192.168.1.101:6443 check server k8s-master2 192.168.1.102:6443 check server k8s-master3 192.168.1.103:6443 check EOF # 配置Keepalived cat > /etc/keepalived/keepalived.conf <<EOF vrrp_script chk_haproxy { script "killall -0 haproxy" interval 2 weight 2 } vrrp_instance VI_1 { interface eth0 state MASTER virtual_router_id 51 priority 101 virtual_ipaddress { 192.168.1.100/24 } track_script { chk_haproxy } } EOF5. 工作节点与网络插件部署
工作节点需要部署kubelet和kube-proxy，并安装容器运行时（这里以Docker为例）：
# 安装Docker yum install -y yum-utils yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo yum install -y docker-ce docker-ce-cli containerd.io systemctl enable --now docker # 配置kubelet cat > /etc/systemd/system/kubelet.service <<EOF [Unit] Description=Kubernetes Kubelet Documentation=https://kubernetes.io/docs/concepts/overview/ [Service] ExecStart=/usr/local/bin/kubelet \\ --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf \\ --kubeconfig=/etc/kubernetes/kubelet.conf \\ --config=/var/lib/kubelet/config.yaml \\ --container-runtime=remote \\ --container-runtime-endpoint=unix:///run/containerd/containerd.sock \\ --pod-infra-container-image=registry.aliyuncs.com/google_containers/pause:3.7 \\ --node-labels=node-role.kubernetes.io/node= Restart=on-failure RestartSec=5 [Install] WantedBy=multi-user.target EOF网络插件选择Calico，它支持网络策略和IP地址管理：
kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml6. 集群验证与排错
部署完成后需要进行全面验证：
# 检查节点状态 kubectl get nodes -o wide # 检查核心组件状态 kubectl get pods -n kube-system # 部署测试应用 kubectl create deployment nginx --image=nginx kubectl expose deployment nginx --port=80 --type=NodePort # 测试网络连通性 kubectl run busybox --rm -it --image=busybox -- sh wget -qO- nginx常见问题排查技巧：
- kubelet无法启动：检查证书路径和权限
- Pod网络不通：检查Calico日志和路由表
- API Server不可用：检查etcd集群状态和HAProxy日志
- 证书过期：使用cfssl重新生成并分发证书
