LAMP+haproxy+varnish实现网站访问的动静分离及静态资源缓存

LAMP+haproxy+varnish实现网站访问的动静分离及静态资源缓存

大家好,又见面了,我是全栈君。

wKiom1Yw10aDt_-tAARoZdsrL2k058.jpg

实验目标:
1.    LAMP节点提供用户动态请求访问,数据库单独有数据库节点提供;
2.    LAMP动态网站有两台服务器,提供负载均衡;
3.    静态网站服务器节点提供用户的静态资源请求访问;存在两台静态web服务器,其网站静态资源在静态服务器上存放;
4.    用户的静态请求访问后缓存在varnish服务器上,实现访问加速
5.    前端的haproxy提供反向代理功能,将用户的动态资源请求发送给后端LAMP节点,静态资源请求发往后端静态web服务器;
6.    该架构考虑还不健全,如静态内容的一致性,数据库的单点故障,只是一个不成熟的架构实现简单的动静分离及缓存服务器实现;

实现过程:

一. LAMP构建
================ ha.stu31.com : 172.16.31.10 ================
vim /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
172.16.31.10 ha.stu31.com ha
172.16.31.11 node1.stu31.com node1
172.16.31.12 node2.stu31.com node2
172.16.31.13 node3.stu31.com node3
172.16.31.14 node4.stu31.com node4
172.16.31.15 mysql.stu31.com mysql
172.16.31.16 node5.stu31.com node5
172.16.31.17 node6.stu31.com node6
=======
ssh-keygen -t rsa -P “” -f “/root/.ssh/id_rsa”
for i in {1..6}; do ssh-copy-id -i .ssh/id_rsa.pub node$i; done
for i in {1..6}; do scp /etc/hosts node$i:/etc/hosts; done
for i in {1..6}; do ssh node$i “ntpdate -u 202.112.10.60”; done
for i in {1..6}; do ssh node$i date; done
for i in {1..6}; do ssh node$i “mount /dev/cdrom /mnt/cdrom”; done
for i in {1..2}; do ssh node$i “yum -y install httpd php  mysql  php-mysql”; done
for i in {1..2}; do ssh node$i “service httpd start”; done
vim index.php
<?php
       phpinfo();
?>
///
for i in {1..2}; do scp index.php root@node$i:/var/www/html; done

====================== windows ======================
浏览器访问 http://172.16.31.11/index.php
浏览器访问 http://172.16.31.12/index.php

============== mysql.stu31.com : 172.16.31.15 ================
mount /dev/cdrom /mnt/cdrom/
yum -y install mysql-server mysql
service mysqld start
mysql
mysql> create schema bbsdb;
mysql> grant all on bbsdb.* to ‘bbsadmin’@’172.16.%.%’identified by ‘oracle’;
mysql> flush privileges;
mysql> quit

============== node1.stu31.com : 172.16.31.11 ================
wget http://download.comsenz.com/DiscuzX/3.2/Discuz_X3.2_SC_UTF8.zip
unzip -d /var/www/html/ Discuz_X3.2_SC_UTF8.zip
cd /var/www/html/upload
chmod -R go+w config/ data uc_*

====================== windows ======================
浏览器访问 http://172.16.31.11/upload/install
1. 检查安装环境:
2. 设置运行环境:点击选择全新安装
3. 安装数据库
填写数据库信息
数据库服务器: 172.16.31.15
数据库名: bbsdb
数据库用户名: bbsadmin
数据库密码: oracle
数据表前缀: pre_
系统邮箱: admin@stu31.com
填写管理员信息
管理员账号: admin
管理员密码: admin
重复密码: admin
管理员Email: admin@stu31.com
设置好数据库后点击安装即可

============== node2.stu31.com : 172.16.31.12 ===============

1. cd /var/www/html; scp -r upload/ readme/ utility/ node2:/var/www/html/
2. 重新解压软件包到node2,重新安装一遍,删除数据库重新安装(因为数据库中无重要内容)。

访问测试
====================== windows ======================
http://172.16.31.11/upload/forum.php
http://172.16.31.12/upload/forum.php
能正常访问证明LAMP平台构建成功

二. 静态web服务器构建

============== node5.stu31.com : 172.16.31.16 ===============
scp node1:/root/Discuz_X3.2_SC_UTF8.zip ./
(wget http://download.comsenz.com/DiscuzX/3.2/Discuz_X3.2_SC_UTF8.zip)
unzip -d /var/www/html/ Discuz_X3.2_SC_UTF8.zip
service httpd start

============== node6.stu31.com : 172.16.31.17 ===============
scp node1:/root/Discuz_X3.2_SC_UTF8.zip ./
(wget http://download.comsenz.com/DiscuzX/3.2/Discuz_X3.2_SC_UTF8.zip)
unzip -d /var/www/html/ Discuz_X3.2_SC_UTF8.zip
service httpd start

访问测试
====================== windows ======================
http://172.16.31.16/upload/forum.php
http://172.16.31.17/upload/forum.php
只能看到静态界面

三. Varnish缓存服务器构建

============== node3.stu31.com : 172.16.31.13 ===============

wget http://repo.varnish-cache.org/redhat/varnish-3.0/el6/x86_64/varnish/varnish-3.0.6-1.el6.x86_64.rpm
wget http://repo.varnish-cache.org/redhat/varnish-3.0/el6/x86_64/varnish/varnish-docs-3.0.6-1.el6.x86_64.rpm
wget http://repo.varnish-cache.org/redhat/varnish-3.0/el6/x86_64/varnish/varnish-libs-3.0.6-1.el6.x86_64.rpm
yum -y install *.rpm

vim /etc/sysconfig/varnish
NFILES=131072
MEMLOCK=82000
#NPROCS=”unlimited”            #加注释
DAEMON_COREFILE_LIMIT=”unlimited”        #去注释
RELOAD_VCL=1
VARNISH_VCL_CONF=/etc/varnish/default.vcl
VARNISH_LISTEN_PORT=80              #修改
VARNISH_ADMIN_LISTEN_ADDRESS=127.0.0.1
VARNISH_ADMIN_LISTEN_PORT=6082
VARNISH_SECRET_FILE=/etc/varnish/secret
VARNISH_MIN_THREADS=1            #修改
VARNISH_MAX_THREADS=1000
VARNISH_THREAD_TIMEOUT=120
VARNISH_STORAGE_FILE=/var/lib/varnish/varnish_storage.bin
VARNISH_STORAGE_SIZE=1G
VARNISH_STORAGE=”file,${VARNISH_STORAGE_FILE},${VARNISH_STORAGE_SIZE}”
VARNISH_TTL=120
DAEMON_OPTS=”-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} \
             -f ${VARNISH_VCL_CONF} \
             -T ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} \
             -t ${VARNISH_TTL} \
             -w ${VARNISH_MIN_THREADS},${VARNISH_MAX_THREADS},${VARNISH_THREAD_T
             -u varnish -g varnish \
             -S ${VARNISH_SECRET_FILE} \
             -s ${VARNISH_STORAGE}”
///   ( grep -v ^# /etc/sysconfig/varnish |sed ‘/^$/d’ )

cp /etc/varnish/default.vcl{,.bak}
vim /etc/varnish/default.vcl
:set nu
:15,$s@^#@@g

backend default {

  .host = “127.0.0.1”;
  .port = “80”;
}

# Below is a commented-out copy of the default VCL logic.  If you
# redefine any of these subroutines, the built-in logic will be
# appended to your code.
 sub vcl_recv {

     if (req.restarts == 0) {

        if (req.http.x-forwarded-for) {

            set req.http.X-Forwarded-For =
                req.http.X-Forwarded-For + “, ” + client.ip;
        } else {

            set req.http.X-Forwarded-For = client.ip;
        }
     }
     if (req.request != “GET” &&
       req.request != “HEAD” &&
       req.request != “PUT” &&
       req.request != “POST” &&
       req.request != “TRACE” &&
       req.request != “OPTIONS” &&
       req.request != “DELETE”) {

         /* Non-RFC2616 or CONNECT which is weird. */
         return (pipe);
     }
     if (req.request != “GET” && req.request != “HEAD”) {

         /* We only deal with GET and HEAD by default */
         return (pass);
     }
     if (req.http.Authorization || req.http.Cookie) {

         /* Not cacheable by default */
         return (pass);
     }
     return (lookup);
 }

 sub vcl_pipe {   
     # Note that only the first request to the backend will have
     # X-Forwarded-For set.  If you use X-Forwarded-For and want to
     # have it set for all requests, make sure to have:
     # set bereq.http.connection = “close”;
     # here.  It is not set by default as it might break some broken web
     # applications, like IIS with NTLM authentication.
     return (pipe);
 }

 sub vcl_pass {

     return (pass);
 }

 sub vcl_hash {

     hash_data(req.url);
     if (req.http.host) {

         hash_data(req.http.host);
     } else {

         hash_data(server.ip);
     }
     return (hash);
 }

 sub vcl_hit {

     return (deliver);
 }

 sub vcl_miss {

     return (fetch);
 }

sub vcl_fetch {

     if (beresp.ttl <= 0s ||
         beresp.http.Set-Cookie ||
         beresp.http.Vary == “*”) {

                /*
                 * Mark as “Hit-For-Pass” for the next 2 minutes
                 */
                set beresp.ttl = 120 s;
                return (hit_for_pass);
     }
     return (deliver);
 }

 sub vcl_deliver {

     return (deliver);
 }

 sub vcl_error {

     set obj.http.Content-Type = “text/html; charset=utf-8”;
     set obj.http.Retry-After = “5”;
     synthetic {“
 <?xml version=”1.0″ encoding=”utf-8″?>
 <!DOCTYPE html PUBLIC “-//W3C//DTD XHTML 1.0 Strict//EN”
  “http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd”>
 <html>
   <head>
     <title>”} + obj.status + ” ” + obj.response + {“</title>
   </head>
   <body>
     <h1>Error “} + obj.status + ” ” + obj.response + {“</h1>
     <p>”} + obj.response + {“</p>
     <h3>Guru Meditation:</h3>
     <p>XID: “} + req.xid + {“</p>
     <hr>
     <p>Varnish cache server</p>
   </body>
 </html>
 “};
     return (deliver);
 }

 sub vcl_init {

        return (ok);
 }

 sub vcl_fini {

        return (ok);
 }
///

vim /etc/varnish/web.vcl
:set paste
######定义ACL
acl purgers {                    #定义acl,实现IP地址过滤
   “127.0.0.1”;
   “172.16.0.0”/16;
}
######定义健康状态检测
probe dynamic {                  #设置动态网站服务器健康状态检测
   .url = “/index.html”;
   .interval = 5s;
   .timeout = 1s;
    .expected_response= 200;
}           #这里设置了两个健康状态检测主要是为了区分动、静网站
probe static {                   #设置动态网站服务器健康状态检测
   .url = “/index.html”;       #定义检测的页面
   .interval = 5s;              #探测请求的发送周期,默认为5秒
   .timeout = 1s;               #每次探测请求的过期时间
   .expected_response = 200;
}
######定义后端服务器
backend app1 {                  #定义一个后端服务器
   .host = “172.16.31.11”;     #服务器地址
   .port = “80”;              #服务器监听端口
   .probe = dynamic;           #健康状态检测
}
backend app2 {

   .host = “172.16.31.12”;
    .port = “80”;
   .probe = dynamic;
}
backend web1 {         
   .host = “172.16.31.16”;
   .port = “80”;
   .probe = static;
}
backend web2 {

   .host = “172.16.31.17”;
   .port = “80”;
   .probe = static;
}
######定义后端动态服务器组,实现负载均衡效果
director apps random {          #定义一个后端服务器组,实现负载均衡效果
    {

        .backend = app1;       #调用前面已定义过的后端主机
    .weight = 2;           #设置权重
    }
    {

    .backend = app2;
    .weight = 2;
    }
}
######定义后端静态服务器组,实现负载均衡效果
director webs random {

    {

       .backend = web1;
       .weight = 2 ;
    }
    {

       .backend = web2;
       .weight = 2 ;
    }
}
######定义vcl_recv函数,实现请求到达并成功接收后调用此函数中定义的规则
sub vcl_recv {

######定义动、静分离,以”.php”或”.php?后面跟所有文件”结尾的请求都发送到动态服务器,其他请求都发送到静态服务器
   if (req.url ~ “\.php(\?\.*|$)”) {

   set req.backend = apps;
    }else {

   set req.backend = webs;
    }
   return(lookup);
######定义允许清除缓存的IP地址,调用的是前面定义的ACL
   if (req.request == “PURGE”) {

       if (!client.ip ~ purgers) {

       error 405 “Method not allowed”;
    }
       return(lookup);
    }
######重新定义http请求首部,让后端服务器可以记录请求客户端的真实IP地址
       if (req.restarts == 0) {

           if (req.http.x-forwarded-for) {

               set req.http.X-Forwarded-For =
               req.http.X-Forwarded-For +”, ” + client.ip;
           } else {

                 set req.http.X-Forwarded-For =client.ip;
           }
        }
######除了定义的请求方法外,其他请求都到后端服务器
   if (req.request != “GET” &&
       req.request != “HEAD” &&
       req.request != “PUT” &&
       req.request != “POST” &&
       req.request != “TRACE” &&
       req.request != “OPTIONS” &&
       req.request != “DELETE”) {

       return (pipe);
    }
   if (req.request != “GET” && req.request !=”HEAD”) {

       return (pass);
    }
######定义不缓存认证与Cookie信息
   if (req.http.Authorization || req.http.Cookie) {

       return (pass);
    }
######定义压缩功能
   if (req.http.Accept-Enconding) {

      if (req.url ~ “\.(jpg|jpeg|gif|bmp|png|flv|gz|tgz|tbz|mp3)$”){

          remove req.http.Accept-Encoding;
      remove req.http.Cookie;
      } else if (req.http.Accept-Encoding ~ “gzip”) {

      set req.http.Accept-Encoding = “gzip”;
      } else if (req.http.Accept-Encoding ~ “deflate”) {

      set req.http.Accept-Encoding = “deflate”;
      } else { remove req.http.Accept-Encoding;
      }
    }
######定义指定格式结尾的文件去除Cookie信息
   if (req.request == “GET” && req.url ~”\.(jpeg|jpg|gif|png|bmp|swf)$”) {

   unset req.http.cookie;
    }
######定义防盗链设置
   if (req.http.referer ~ “http://.*”) {

       if (!(req.http.referer ~ “http://.*\.baidu\.com” ||req.http.referer ~ “http://.*\.google\.com.*”)) {

              set req.http.host =”www.stu31.com”;
         set req.url = “http://172.16.31.10/error.html”;
    }
    }
}
######定义vcl_hash函数
sub vcl_hash {

    hash_data(req.url);
   if (req.http.host) {

       hash_data(req.http.host);
    }else {

       hash_data(server.ip);
    }
   return(hash);
}
######定义vcl_hit函数
sub vcl_hit {

   if (req.request == “PURGE”) { #语法方法为”PURGE”
      purge;                     #清除缓存
      error 200 “Purged.”;      #返回错误状态码为”200″
    }
   return(deliver);
}
######定义vcl_miss函数
sub vcl_miss {

   if (req.request == “PURGE”) {

   purge;
   error 404 “Not In Cache.”;
    }
   return(fetch);
}
######定义vcl_psss函数
sub vcl_pass {

   if (req.request == “PURGE”) {

      error 502 “Purged On A Passed Object.”;
    }
   return(pass);
}
######定义vcl_fetch函数
sub vcl_fetch {

######定义缓存,如果匹配到已定义文件结尾的缓存1天,其他则缓存1小时
   if (req.request == “GET” && req.url ~”\.(html|jpg|png|bmp|jpeg|gif|js|ico|swf|css)$”) {

      set beresp.ttl = 1d;
      set beresp.http.expires = beresp.ttl;
    }else {

      set beresp.ttl = 1h;
    }
   return(deliver);
}
######定义在http首部中,如果请求命中显示”HIT”,未命中则显示”MISS”,通过F12可查看缓存命中状态及varnish服务器IP
sub vcl_deliver {

   if (obj.hits > 0) {

      set resp.http.X-Cache = “Hit from ” + server.ip;
    }else {

      set resp.http.X-Cache = “MISS”;
    }
}
end for /etc/varnish/web.vcl

/etc/rc.d/init.d/varnish  start
netstat -tnlp | grep varnish 

varnishadm -S/etc/varnish/secret -T 127.0.0.1:6082
varnish> vcl.load cache web.vcl
varnish> vcl.use cache
varnish> vcl.list
varnish> quit

============== node4.stu31.com : 172.16.31.14 ===============
scp node3:/root/*.rpm ./
yum -y install *.rpm
scp node3:/etc/sysconfig/varnish /etc/sysconfig/
scp node3:/etc/varnish/*.vcl /etc/varnish

/etc/rc.d/init.d/varnish  start
netstat -tnlp | grep varnish 

varnishadm -S/etc/varnish/secret -T 127.0.0.1:6082
varnish> vcl.load cache web.vcl
varnish> vcl.use cache        #立刻生效, 重启varnish失效
varnish> vcl.list
varnish> quit

如果我们希望后端的web服务器记录客户端访问的真实IP地址,我们需要配置httpd的配置文件中的日志格式:
node1, node2, node5, node6四个节点都要
vim /etc/httpd/conf/httpd.conf   
LogFormat “%{X-Forwarded-For}i %l %u%t \”%r\” %>s %b \”%{Referer}i\” \”%{User-Agent}i\””combined
///
service httpd reload
echo “ok” > /var/www/html/index.html    #此页面用于varnish做健康检测

四. HAProxy反向代理构建

=============== ha.stu31.com : 172.16.31.10 ================
mount /dev/cdrom /mnt/cdrom
yum -y install haproxy

vim /etc/rsyslog.conf
# Save boot messages also to boot.log
local7.*                                                /var/log/boot.log
local2.*                                                /var/log/haproxy.log     #添加此行
//
service rsyslog restart

cd /etc/haproxy/
cp haproxy.cfg{,.bak}
vim haproxy.cfg
global
    log         127.0.0.1 local2
    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     4000
    user        haproxy
    group       haproxy
    daemon
    stats socket /var/lib/haproxy/stats

defaults
    mode                    http
    log                     global
    option                  httplog
    option                  dontlognull
    option http-server-close
    option forwardfor       except 127.0.0.0/8
    option                  redispatch
    retries                 3
    timeout http-request    10s
    timeout queue           1m
    timeout connect         10s
    timeout client          1m
    timeout server          1m
    timeout http-keep-alive 10s
    timeout check           10s
    maxconn                 3000

:set paste 

listen stats        #添加此块
    mode http
    bind 0.0.0.0:1080
    stats enable
    stats hide-version
    stats uri     /haproxyadmin?stats
    stats realm   Haproxy\ Statistics
    stats auth    admin:admin
    stats admin if TRUE

frontend http-in        #修改很多内容
    bind *:80
    mode http
    log global
    option httpclose
    option logasap
    option dontlognull
    capture request  header Host len 20
    capture request  header Referer len 60
    acl url_static       path_beg       -i /upload/static/image/
    acl url_static       path_end       -i .html .jpeg .gif .png .jpg
    acl url_dynamic      path_end       -i .php .css .js .jsp
    use_backend static_servers         if url_static
    use_backend dynamic_servers        if url_dynamic
    default_backend static_servers

backend static_servers
    balance roundrobin
    server staticsrv1 172.16.31.13:80 check maxconn 3000
    server staticsrv2 172.16.31.14:80 check maxconn 3000

backend dynamic_servers
    balance source
    server dynamicsrv1 172.16.31.11:80 check maxconn 3000
    server dynamicsrv2 172.16.31.12:80 check maxconn 3000
//

service haproxy start

五. 测试访问

====================== windows =====================
访问论坛
http://172.16.31.10/upload/forum.php

访问haproxy的状态页
http://172.16.31.10:1080/haproxyadmin?stats
admin
admin

window浏览器F12检查缓存命中情况

varnish上的日志
[root@node3 ~]# varnishlog

原始服务器的日志
[root@node1 ~]# tail /var/log/httpd/access_log








本文转自 zhuhc1988 51CTO博客,原文链接:http://blog.51cto.com/changeflyhigh/1707489,如需转载请自行联系原作者
版权声明:本文内容由互联网用户自发贡献,该文观点仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 举报,一经查实,本站将立刻删除。

发布者:全栈程序员-用户IM,转载请注明出处:https://javaforall.cn/108048.html原文链接:https://javaforall.cn

【正版授权,激活自己账号】: Jetbrains全家桶Ide使用,1年售后保障,每天仅需1毛

【官方授权 正版激活】: 官方授权 正版激活 支持Jetbrains家族下所有IDE 使用个人JB账号...

(0)
blank

相关推荐

  • Q学习(Q learning) 强化学习的简单例子 Matlab实现 可视化「建议收藏」

    Q学习(Q learning) 强化学习的简单例子 Matlab实现 可视化「建议收藏」参考链接:https://blog.csdn.net/Maggie_zhangxin/article/details/73481417实现的内容很简单,存为.m文件可以直接在matlab上运行,就是利用Q学习(Qlearning)完成自主路径寻优简单示例,并进行可视化,Q学习部分参考了如上链接中的内容,供大家交流学习使用,请多提宝贵意见如图为最终路径,红色方框代表机器人,绿色区域代表障碍…

  • 网站接入微信扫码登录并获取用户基本信息(微信开放平台)

    网站接入微信扫码登录并获取用户基本信息(微信开放平台)现在的网站基本都接入微信登陆了,就好像下面这样的。只需要用微信扫一下二维码,这个网站就可以展示你的微信昵称和头像,免去注册账号和输入密码登录的步骤,还免去设置头像和昵称的步骤,所以是挺方便的。那么如何把自己的网站接入这个呢?首先咱们得先在微信开放平台注册账号并且创建一个网站应用,等待审核通过后就可以获得Appid和AppSecretbingqie并且还要设置回调域名,这个不多说。首…

  • ThreadPool.QueueUserWorkItem启动慢

    ThreadPool.QueueUserWorkItem启动慢一、问题描述ThreadPool.QueueUserWorkItem启动慢,在项目过程中发现当线程数量达到一定量的时候发线线程启动速度慢,影响了正常的实时性业务。二、解决方法加入ThreadPool.SetMinThreads(1000,1000)后,整个进程启动变块。如图1和图2,变快了。…

  • MySQL——MySQL 图形化管理工具的介绍

    MySQL——MySQL 图形化管理工具的介绍文章目录MySQL——MySQL图形化管理工具的介绍1、MySQLWorkbench2、Navicat3、SQLyog4、DBeaver5、DataGripMySQL——MySQL图形化管理工具的介绍MySQL图形化管理工具极大地方便了数据库的操作与管理,常用的图形化管理工具有:MysQLWorkbench、phpMyAdmin、NavicatPreminum、MySQLDumper、SQLyog、dbeaver、MysQLODBcConnector、DataGrip。1、MySQL

  • java培训学费_北京Java培训班学费很贵吗,包含了哪些收费项目

    java培训学费_北京Java培训班学费很贵吗,包含了哪些收费项目原标题:北京Java培训班学费很贵吗,包含了哪些收费项目北京的Java培训班有很多,价格却是相差不多的,但培训的课程就参差不齐了,有的培训班就是为了赚钱而存在的,想要系统的学习Java,确保学习效果,那么你一定要挑选正规的Java培训班,挑选适合自己的Java课程,培训费用可以在和机构老师详谈。我们先来看看Java培训机构的收费情况,学费都包含了哪些呢?专业的Java培训在硬件设施上,在师资力量上…

  • 两个数组拼接

    两个数组拼接方法一:vara1=[‘aa’,12,13];vara2=[21,22,23];varnewA=a1.concat(a2)方法二:vara1=[‘aa’,12,13];vara2=[21,22,23];varnewA=a1.join()+’,’+a2.join();方法三:vara1=[‘aa’,12,13];vara2…

发表回复

您的电子邮箱地址不会被公开。

关注全栈程序员社区公众号