user_agent屏蔽+炸弹，防护Nginx Web服务器

不知道为什么，我这个小破站这么多人扫，用了百度云加速+防火墙，整体还是有效果的。我这个站啥也没有，最多的是做做记录，写写日志，分享分享。碰巧最近在研究这方面的东西，先试试看。首先是屏蔽恶意user_agent，面板直接在防火墙里添加

//如果用的是宝塔面板直接添加即可
(Abonti|aggregator|AhrefsBot|asterias|BDCbot|BLEXBot|BuiltBotTough|Bullseye|BunnySlippers|ca-crawler|CCBot|Cegbfeieh|CheeseBot|CherryPicker|CopyRightCheck|cosmos|Crescent|discobot|DittoSpyder|DOC|DotBot|Download Ninja|EmailCollector|EmailSiphon|EmailWolf|EroCrawler|Exabot|ExtractorPro|Fasterfox|FeedBooster|Foobot|Genieo|grub-client|Harvest|hloader|httplib|humanlinks|ieautodiscovery|InfoNaviRobot|IstellaBot|JennyBot|k2spider|Kenjin Spider|Keyword Density0.9|larbin|LexiBot|libWeb|LinkextractorPro|linko|LinkScan8.1a Unix|LinkWalker|LNSpiderguy|lwp-trivial|magpie|Mata Hari|MaxPointCrawler|MegaIndex|Microsoft URL Control|MIIxpc|Mippin|Missigua Locator|Mister PiX|MJ12bot|moget|MSIECrawler|NetAnts|NICErsPRO|Niki-Bot|NPBot|Nutch|Offline Explorer|Openfind|panscient.com|ProPowerBot2.14|ProWebWalker|QueryN Metasearch|RepoMonkey|RMA|SemrushBot|SeznamBot|SISTRIX|sitecheck.Internetseer.com|SiteSnagger|SnapPreviewBot|SpankBot|spanner|spbot|Spinn3r|suzuran|Szukacz1.4|Teleport|Telesoft|The Intraformant|TheNomad|TightTwatBot|Titan|True_Robot|turingos|TurnitinBot|UbiCrawler|UnisterBot|URLy Warning|VCI|WBSearchBot|Web Downloader6.9|WebAuto|WebBandit|WebCopier|WebEnhancer|WebmasterWorldForumBot|WebReaper|WebSauger|Website Quester|Webster Pro|WebStripper|WebZip|Wotbox|wsr-agent|WWW-Collector-E|Xenu|yandex|Zao|Zeus|ZyBORG|coccoc|Incutio|lmspider|memoryBot|SemrushBot|serf|Unknown|uptime files|BOT/0.1|YisouSpider|Jullo|YandexBot|WinHttp|Linguee Bot|HTTrack|harvest|audit|dirbuster|pangolin|nmap|sqln|hydra|Parser|libwww|BBBike|sqlmap|w3af|owasp|Nikto|fimap|havij|zmeu|FeedDemon|Indy Library|Alexa Toolbar|AskTbFXTV|AhrefsBot|CrawlDaddy|CoolpadWebkit|Java|Feedly|UniversalFeedParser|ApacheBench|Swiftbot|ZmEu|oBot|jaunty|Python-urllib|lightDeckReports Bot|YYSpider|DigExt|HttpClient|MJ12bot|heritrix|EasouSpider|Ezooms|BabyKrokodil|netsparker|httperf|Auto Spider| SF/)

//这些UA也挺烦人
(Gecko/20100101 Firefox/4|Gecko/20100101 Firefox/7.0.1|Gecko/20100101 Firefox/3|Gecko/20061208 Firefox/2|Gecko/20100101 Firefox/79.0|Gecko/20100101 Firefox/81|Gecko/20100101 Firefox/5| SF/)

或者在添加到Nginx的配置里添加

#禁止访问的文件或目录
location ~ ^/(\.user.ini|\.htaccess|\.git|\.svn|\.project|LICENSE|README.md|index.html|wp-includes/wlwmanifest.xml)
{
return 403;
}


#禁止空agent的浏览器访问
if ($http_user_agent ~ ^$) {
return 403;
}

#禁止Scrapy等工具的抓取
if ($http_user_agent ~* (Scrapy|Curl|HttpClient)) {
return 403;
}

#禁止指定UA的访问
if ($http_user_agent ~ "ApacheBench|WebBench|HttpClient|Java|python|Go-http-client|FeedDemon|JikeSpider|Indy Library|Alexa Toolbar|AskTbFXTV|AhrefsBot|CrawlDaddy|CoolpadWebkit|Feedly|UniversalFeedParser|Microsoft URL Control|Swiftbot|ZmEu|oBot|jaunty|lightDeckReports Bot|YYSpider|DigExt|YisouSpider|MJ12bot|heritrix|EasouSpider|LinkpadBot|Ezooms" )
{
return 403;
}

#禁止爬虫抓取
if ($http_user_agent ~* "qihoobot|Googlebot-Mobile|Googlebot-Image|Mediapartners-Google|Adsbot-Google|Feedfetcher-Google|Yahoo! Slurp|Yahoo! Slurp China|YoudaoBot|Sosospider|Sogou spider|Sogou web spider|MSNBot|ia_archiver|Tomato Bot") 
{ 
    return 403; 
}

#禁止非GET|HEAD|POST方式的抓取
if ($request_method !~ ^(GET|HEAD|POST)$) {
return 403;
}

#禁止特殊的user_agent的访问
if ($http_user_agent ~ "Mozilla/4.0\ \(compatible;\ MSIE\ 6.0;\ Windows\ NT\ 5.1;\ SV1;\ .NET\ CLR\ 1.1.4322;\ .NET\ CLR\ 2.0.50727\)") {
return 404;
}

当然这么做还不是很给力，要知道caddy 有个http.nobots nobots保护你的网站免受网络爬虫和机器人，可以给采集的恶意程序返回特定的数据。比如实际1G压缩成1M大小的gzip文件(炸弹)，对于服务器的话就是发送了1M的文件，恶意程序请求后会自动解包为10G的数据，可以对恶意程序造成很大的压力。那么我们用PHP也可以整个。

//首先制造炸弹，cd到网站根目录，在Linux中，这确实很容易，您可以使用以下命令。根据需求生成不同大小的gzip压缩文件.我生成的10G.gzip
dd if=/dev/zero bs=1M count=1024 | gzip > 1G.gzip
dd if=/dev/zero bs=1M count=10240 | gzip > 10G.gzip
dd if=/dev/zero bs=1M count=1048576 | gzip > 1T.gzip


//在根目录新建一个bomb.php文件
<?php
header('Content-Encoding: gzip');
echo file_get_contents('10G.gzip');


//Nginx添加配置，可以根据实际需求把上述return 403换成“rewrite ^/.*  /bomb.php last;”，对方扫描的时候直接开始重定向到这个10G炸弹
location / {
  #UA BOMB
  if ($http_user_agent ~* "(恶意user-agent)") {
    rewrite ^/.*  /bomb.php last;
  }
}

此时恶意的扫描或者请求，就会重定向到这个包，然后就解这个gzip的包为10G的数据，多个请求就能直接干崩溃，如果崩溃不了就生成个1T的。

比如说，你随便访问一个不存在的asp页面，https://www.zkcoi.com/bucunzai.asp，状态码200，就不给你跳404，然后浏览器就直接就给干崩了……