30.企业级反向代理haproxy（3）：haproxy编译安装

3.3 编译安装HAProxy

编译安装HAProxy 2.0 LTS版本，更多源码包下载地址：http://www.haproxy.org/download/

3.3.1 解决lua环境

HAProxy 支持基于lua实现功能扩展，lua是一种小巧的脚本语言，于1993年由巴西里约热内卢天主教大学(Pontifical Catholic University of Rio de Janeiro) 里的一个研究小组开发，其设计目的是为了嵌入应用程序中，从而为应用程序提供灵活的扩展和定制功能。

Lua官网：http://www.lua.org/

Lua应用场景

• 游戏开发
• 独立应用脚本
• Web应用脚本
• 扩展和数据库插件，如MySQL Proxy
• 安全系统，如入侵检测系统

3.3.1.1 CentOS基础环境

参考链接：http://www.lua.org/start.html

由于CentOS 7 之前版本自带的lua版本比较低并不符合HAProxy要求的lua最低版本(5.3)的要求，因此需要编译安装较新版本的lua环境，然后才能编译安装HAProxy，过程如下：

# 当前系统版本
[root@centos7 ~]# lua -v
Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio

# 安装基础命令及编译依赖环境
[root@centos7 ~]# yum install gcc readline-devel -y
[root@centos7 ~]# wget http://www.lua.org/ftp/lua-5.3.6.tar.gz
[root@centos7 ~]# tar xvf lua-5.3.6.tar.gz -C /usr/local/src/
[root@centos7 ~]# cd /usr/local/src/lua-5.3.6/
[root@centos7 lua-5.3.6]# make linux test

# 查看编译安装的版本
[root@centos7 lua-5.3.6]# src/lua -v
Lua 5.3.6  Copyright (C) 1994-2020 Lua.org, PUC-Rio

3.3.1.2 Ubuntu基础环境

# 安装基础命令及编译依赖环境
[root@ubuntu1804 ~]# apt install make gcc iproute2 ntpdate tcpdump telnet traceroute nfs-kernel-server nfs-common lrzsz tree openssl libssl-dev libpcre3 libpcre3-dev zlib1g-dev openssh-server libreadline-dev libsystemd-dev -y

[root@ubuntu1804 ~]# cd /usr/local/src/
[root@ubuntu1804 /usr/local/src]# wget http://www.lua.org/ftp/lua-5.3.6.tar.gz
[root@ubuntu1804 /usr/local/src]# tar xvf lua-5.3.6.tar.gz
[root@ubuntu1804 /usr/local/src]# cd lua-5.3.6/
[root@ubuntu1804 /usr/local/src/lua-5.3.6]# make linux test

[root@ubuntu1804 /usr/local/src/lua-5.3.6]# pwd
/usr/local/src/lua-5.3.6
[root@ubuntu1804 /usr/local/src/lua-5.3.6]# src/lua -v
Lua 5.3.6  Copyright (C) 1994-2020 Lua.org, PUC-Rio

或者安装系统自带的lua
[root@ubuntu1804 ~]# apt install lua5.3=5.3.3-1ubuntu0.18.04.1 -y
[root@ubuntu1804 ~]# lua5.3 -v
Lua 5.3.3  Copyright (C) 1994-2016 Lua.org, PUC-Rio

3.3.2 编译安装HAProxy

# HAProxy 1.8及1.9版本编译参数：
make ARCH=x86_64 TARGET=linux2628 USE_PCRE=1 USE_OPENSSL=1 USE_ZLIB=1 USE_SYSTEMD=1 USE_CPU_AFFINITY=1 PREFIX=/usr/local/haproxy

# HAProxy 2.0以上版本编译参数：
[root@centos7 ~]# yum install make gcc openssl-devel pcre-devel systemd-devel -y
[root@centos7 ~]# wget http://www.haproxy.org/download/2.4/src/haproxy-2.4.3.tar.gz # 需要科学上网

[root@centos7 ~]# tar xvf haproxy-2.4.3.tar.gz -C /usr/local/src/
[root@centos7 ~]# cd /usr/local/src/haproxy-2.4.3/

# 查看安装方法
[root@centos7 haproxy-2.4.3]# ll Makefile
-rw-rw-r-- 1 root root 45688 Aug 17 20:11 Makefile
[root@centos7 haproxy-2.4.3]# cat README
[root@centos7 haproxy-2.4.3]# cat INSTALL

# 参考INSTALL文件进行编译安装
[root@centos7 haproxy-2.4.3]# make ARCH=x86_64 TARGET=linux-glibc USE_PCRE=1 USE_OPENSSL=1 USE_ZLIB=1 USE_SYSTEMD=1 USE_LUA=1 LUA_INC=/usr/local/src/lua-5.3.6/src LUA_LIB=/usr/local/src/lua-5.3.6/src

[root@centos7 haproxy-2.4.3]# make install PREFIX=/apps/haproxy

[root@centos7 haproxy-2.4.3]# ln -s /apps/haproxy/sbin/haproxy /usr/sbin/

# 查看生成的文件
[root@centos7 haproxy-2.4.3]# tree /apps/haproxy/
/apps/haproxy/
├── doc
│   └── haproxy
│       ├── 51Degrees-device-detection.txt
│       ├── architecture.txt
│       ├── close-options.txt
│       ├── configuration.txt
│       ├── cookie-options.txt
│       ├── DeviceAtlas-device-detection.txt
│       ├── intro.txt
│       ├── linux-syn-cookies.txt
│       ├── lua.txt
│       ├── management.txt
│       ├── netscaler-client-ip-insertion-protocol.txt
│       ├── network-namespaces.txt
│       ├── peers.txt
│       ├── peers-v2.0.txt
│       ├── proxy-protocol.txt
│       ├── regression-testing.txt
│       ├── seamless_reload.txt
│       ├── SOCKS4.protocol.txt
│       ├── SPOE.txt
│       └── WURFL-device-detection.txt
├── sbin
│   └── haproxy
└── share
    └── man
        └── man1
            └── haproxy.1

6 directories, 22 files
[root@centos7 haproxy-2.4.3]# 

3.3.3 验证HAProxy 版本

# 验证HAProxy版本：
[root@centos7 ~]# which haproxy
/usr/sbin/haproxy
[root@centos7 ~]# haproxy -v
HAProxy version 2.4.3-4dd5a5a 2021/08/17 - https://haproxy.org/
Status: long-term supported branch - will stop receiving fixes around Q2 2026.
Known bugs: http://www.haproxy.org/bugs/bugs-2.4.3.html
Running on: Linux 3.10.0-1160.el7.x86_64 #1 SMP Mon Oct 19 16:18:59 UTC 2020 x86_64

# 大写-V选项显示版本和帮助用法
[root@centos7 ~]# haproxy -V
HAProxy version 2.4.3-4dd5a5a 2021/08/17 - https://haproxy.org/
Status: long-term supported branch - will stop receiving fixes around Q2 2026.
Known bugs: http://www.haproxy.org/bugs/bugs-2.4.3.html
Running on: Linux 3.10.0-1160.el7.x86_64 #1 SMP Mon Oct 19 16:18:59 UTC 2020 x86_64
Usage : haproxy [-f <cfgfile|cfgdir>]* [ -vdVD ] [ -n <maxconn> ] [ -N <maxpconn> ]
        [ -p <pidfile> ] [ -m <max megs> ] [ -C <dir> ] [-- <cfgfile>*]
        -v displays version ; -vv shows known build options.
        -d enters debug mode ; -db only disables background mode.
        -dM[<byte>] poisons memory with <byte> (defaults to 0x50)
        -V enters verbose mode (disables quiet mode)
        -D goes daemon ; -C changes to <dir> before loading files.
        -W master-worker mode.
        -Ws master-worker mode with systemd notify support.
        -q quiet mode : don't display messages
        -c check mode : only check config files and exit
        -n sets the maximum total # of connections (uses ulimit -n)
        -m limits the usable amount of memory (in MB)
        -N sets the default, per-proxy maximum # of connections (0)
        -L set local peer name (default to hostname)
        -p writes pids of all children to this file
        -de disables epoll() usage even when available
        -dp disables poll() usage even when available
        -dS disables splice usage (broken on old kernels)
        -dG disables getaddrinfo() usage
        -dR disables SO_REUSEPORT usage
        -dr ignores server address resolution failures
        -dV disables SSL verify on servers side
        -dW fails if any warning is emitted
        -dD diagnostic mode : warn about suspicious configuration statements
        -sf/-st [pid ]* finishes/terminates old pids.
        -x <unix_socket> get listening sockets from a unix socket
        -S <bind>[,<bind options>...] new master CLI

[root@centos7 ~]# haproxy -vv
HAProxy version 2.4.3-4dd5a5a 2021/08/17 - https://haproxy.org/
Status: long-term supported branch - will stop receiving fixes around Q2 2026.
Known bugs: http://www.haproxy.org/bugs/bugs-2.4.3.html
Running on: Linux 3.10.0-1160.el7.x86_64 #1 SMP Mon Oct 19 16:18:59 UTC 2020 x86_64
Build options :
  TARGET  = linux-glibc
  CPU     = generic
  CC      = cc
  CFLAGS  = -m64 -march=x86-64 -O2 -g -Wall -Wextra -Wdeclaration-after-statement -fwrapv -Wno-unused-label -Wno-sign-compare -Wno-unused-parameter -Wno-clobbered -Wno-missing-field-initializers -Wtype-limits
  OPTIONS = USE_PCRE=1 USE_OPENSSL=1 USE_LUA=1 USE_ZLIB=1 USE_SYSTEMD=1
  DEBUG   = 

Feature list : +EPOLL -KQUEUE +NETFILTER +PCRE -PCRE_JIT -PCRE2 -PCRE2_JIT +POLL -PRIVATE_CACHE +THREAD -PTHREAD_PSHARED +BACKTRACE -STATIC_PCRE -STATIC_PCRE2 +TPROXY +LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT +CRYPT_H +GETADDRINFO +OPENSSL +LUA +FUTEX +ACCEPT4 -CLOSEFROM +ZLIB -SLZ +CPU_AFFINITY +TFO +NS +DL +RT -DEVICEATLAS -51DEGREES -WURFL +SYSTEMD -OBSOLETE_LINKER +PRCTL +THREAD_DUMP -EVPORTS -OT -QUIC -PROMEX -MEMORY_PROFILING

Default settings :
  bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with multi-threading support (MAX_THREADS=64, default=16).
Built with OpenSSL version : OpenSSL 1.0.2k-fips  26 Jan 2017
Running on OpenSSL version : OpenSSL 1.0.2k-fips  26 Jan 2017
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : SSLv3 TLSv1.0 TLSv1.1 TLSv1.2
Built with Lua version : Lua 5.3.6
Built with network namespace support.
Built with zlib version : 1.2.7
Running on zlib version : 1.2.7
Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Built with PCRE version : 8.32 2012-11-30
Running on PCRE version : 8.32 2012-11-30
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Encrypted password support via crypt(3): yes
Built with gcc compiler version 4.8.5 20150623 (Red Hat 4.8.5-44)

Available polling systems :
      epoll : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available multiplexer protocols :
(protocols marked as <default> cannot be specified using 'proto' keyword)
              h2 : mode=HTTP       side=FE|BE     mux=H2       flags=HTX|CLEAN_ABRT|HOL_RISK|NO_UPG
            fcgi : mode=HTTP       side=BE        mux=FCGI     flags=HTX|HOL_RISK|NO_UPG
       <default> : mode=HTTP       side=FE|BE     mux=H1       flags=HTX
              h1 : mode=HTTP       side=FE|BE     mux=H1       flags=HTX|NO_UPG
       <default> : mode=TCP        side=FE|BE     mux=PASS     flags=
            none : mode=TCP        side=FE|BE     mux=PASS     flags=NO_UPG

Available services : none

Available filters :
        [SPOE] spoe
        [CACHE] cache
        [FCGI] fcgi-app
        [COMP] compression
        [TRACE] trace

[root@centos7 ~]# 

3.3.4 HAProxy启动文件

[root@centos7 ~]# vim /usr/lib/systemd/system/haproxy.service
[root@centos7 ~]# cat /usr/lib/systemd/system/haproxy.service
[Unit]
Description=HAProxy Load Balancer
After=syslog.target network.service

[Service]
ExecStartPre=/usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -c -q
ExecStart=/usr/sbin/haproxy -Ws -f /etc/haproxy/haproxy.cfg -p /var/lib/haproxy/haproxy.pid
ExecReload=/bin/kill -user2 $MAINPID

[Install]
WantedBy=multi-user.target

# 默认缺少配置文件，无法启动
[root@centos7 ~]# systemctl daemon-reload
[root@centos7 ~]# systemctl start haproxy.service
Job for haproxy.service failed because the control process exited with error code. See "systemctl status haproxy.service" and "journalctl -xe" for details.
[root@centos7 ~]# tail /var/log/messages
Aug 24 13:27:09 centos7 systemd: Started Update UTMP about System Runlevel Changes.
Aug 24 13:27:09 centos7 systemd: Startup finished in 1.034s (kernel) + 1.025s (initrd) + 5.374s (userspace) = 7.434s.
Aug 24 13:27:09 centos7 kernel: floppy0: no floppy controllers found
Aug 24 13:30:40 centos7 systemd: Reloading.
Aug 24 13:30:48 centos7 systemd: Starting HAProxy Load Balancer...
Aug 24 13:30:48 centos7 haproxy: [ALERT]    (1410) : Cannot open configuration file/directory /etc/haproxy/haproxy.cfg : No such file or directory
Aug 24 13:30:48 centos7 systemd: haproxy.service: control process exited, code=exited status=1
Aug 24 13:30:48 centos7 systemd: Failed to start HAProxy Load Balancer.
Aug 24 13:30:48 centos7 systemd: Unit haproxy.service entered failed state.
Aug 24 13:30:48 centos7 systemd: haproxy.service failed.
[root@centos7 ~]# 

3.3.5 配置文件

# 查看配置文件范例：
[root@centos7 ~]# tree /usr/local/src/haproxy-2.4.3/examples/
/usr/local/src/haproxy-2.4.3/examples/
├── acl-content-sw.cfg
├── basic-config-edge.cfg
├── content-sw-sample.cfg
├── errorfiles
│   ├── 400.http
│   ├── 403.http
│   ├── 408.http
│   ├── 500.http
│   ├── 502.http
│   ├── 503.http
│   ├── 504.http
│   └── README
├── haproxy.init
├── option-http_proxy.cfg
├── quick-test.cfg
├── socks4.cfg
├── transparent_proxy.cfg
└── wurfl-example.cfg

1 directory, 17 files

# 创建自定义的配置文件
[root@centos7 ~]# mkdir /etc/haproxy
[root@centos7 ~]# vim /etc/haproxy/haproxy.cfg
[root@centos7 ~]# cat /etc/haproxy/haproxy.cfg
global
    maxconn     100000
    chroot      /apps/haproxy
    stats socket /var/lib/haproxy/haproxy.sock mode 600 level admin
    #uid         200
    #gid         200
    user        haproxy
    group       haproxy
        #nbproc         4
    #cpu-map 1 0
    #cpu-map 2 1
    #cpu-map 3 2
    #cpu-mao 4 3
    daemon
    pidfile     /var/lib/haproxy/haproxy.pid
    log         127.0.0.1 local2 info

defaults
    option http-keep-alive
    option forwardfor
    maxconn 100000
    mode http
    timeout connect 300000ms
    timeout client  300000ms
    timeout server  300000ms

listen stats
    bind            0.0.0.0:9999
    mode            http
    stats           enable
    log             global
    stats uri  /haproxy-status
    stats auth haadmin:123456

listen web_port
    bind 10.0.0.7:80
    mode http
    log global
    server web1 127.0.0.1:8080 check inter 3000 fall 2 rise 5

3.3.6 启动haproxy

# 准备socket文件目录
[root@centos7 ~]# mkdir /var/lib/haproxy

# 设置用户和目录权限
[root@centos7 ~]# useradd -r -s /sbin/nologin -d /var/lib/haproxy haproxy
[root@centos7 ~]# systemctl enable --now haproxy.service
Created symlink from /etc/systemd/system/multi-user.target.wants/haproxy.service to /usr/lib/systemd/system/haproxy.service.

3.3.7 验证haproxy状态

haproxy.cfg文件中定义了chroot、pidfile、user、group等参数，如果系统没有相应的资源会导致haproxy无法启动，具体参考日志文件/var/log/messages

[root@centos7 ~]# systemctl status haproxy.service
● haproxy.service - HAProxy Load Balancer
   Loaded: loaded (/usr/lib/systemd/system/haproxy.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2021-08-24 13:54:57 CST; 9min ago
  Process: 1579 ExecStartPre=/usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -c -q (code=exited, status=0/SUCCESS)
 Main PID: 1584 (haproxy)
   CGroup: /system.slice/haproxy.service
           ├─1584 /usr/sbin/haproxy -Ws -f /etc/haproxy/haproxy.cfg -p /var/lib/hap...
           └─1586 /usr/sbin/haproxy -Ws -f /etc/haproxy/haproxy.cfg -p /var/lib/hap...

Aug 24 13:54:57 centos7.waluna.top systemd[1]: Starting HAProxy Load Balancer...
Aug 24 13:54:57 centos7.waluna.top systemd[1]: Started HAProxy Load Balancer.
Aug 24 13:54:57 centos7.waluna.top haproxy[1584]: [NOTICE]   (1584) : New worker #...d
Aug 24 13:54:57 centos7.waluna.top haproxy[1584]: [WARNING]  (1586) : Server web_p....
Aug 24 13:54:57 centos7.waluna.top haproxy[1584]: [NOTICE]   (1586) : haproxy vers...a
Aug 24 13:54:57 centos7.waluna.top haproxy[1584]: [NOTICE]   (1586) : path to exec...y
Aug 24 13:54:57 centos7.waluna.top haproxy[1584]: [ALERT]    (1586) : proxy 'web_p...!
Hint: Some lines were ellipsized, use -l to show in full.

[root@centos7 ~]# pstree -p |grep haproxy
           |-haproxy(1584)---haproxy(1586)-+-{haproxy}(1587)
           |                               |-{haproxy}(1588)
           |                               |-{haproxy}(1589)
           |                               |-{haproxy}(1590)
           |                               |-{haproxy}(1591)
           |                               |-{haproxy}(1592)
           |                               |-{haproxy}(1593)
           |                               |-{haproxy}(1594)
           |                               |-{haproxy}(1595)
           |                               |-{haproxy}(1596)
           |                               |-{haproxy}(1597)
           |                               |-{haproxy}(1598)
           |                               |-{haproxy}(1599)
           |                               |-{haproxy}(1600)
           |                               `-{haproxy}(1601)

3.3.8 查看haproxy的状态页面

浏览器访问：http://10.0.0.7:9999/haproxy-status