大家好,又见面了,我是你们的朋友全栈君。
一步步实现对API的访问限制(节流)
如果客户端很频繁的请求服务器,会给给服务器造成很大的压力,需要对客户端对API的请求,做一些限制,如Python 爬虫对服务器API的请求,对API的请求限制也是反爬虫的一个手段之一,那如何实现对API的访问的限制呢?
实现API接口
一个基本的API接口实现,没有任何的限制,客户端可以随意访问,也没有访问限制
[HttpGet]
[Route("~/api/helloworld")]
public HttpResponseMessage HelloWorld()
{
return Request.CreateResponse(HttpStatusCode.OK, "Hello World");
}
复制代码
添加基本的限制
如果要做限制,首先想到的是在访问这个接口时,做一个计数器,记录访问的数量,达到一定的数量之后就不能访问,使用cache
来实现计数
[HttpGet]
[Route("~/api/helloworld")]
public HttpResponseMessage HelloWorld()
{
int? requestCount = (int?)System.Web.HttpRuntime.Cache["throttle"];
if (!requestCount.HasValue) requestCount = 0;
requestCount++;
HttpRuntime.Cache["throttle"] = requestCount;
if (requestCount > 10) return Request.CreateResponse((HttpStatusCode)429, "Too many requests");
return Request.CreateResponse(HttpStatusCode.OK, "Hello World");
}
复制代码
这样如果访问 /api/helloworld
这个接口超过10次,就返回 429
错误,但是这个实现是不能用于生产环境的,只能演示使用,虽然实现了访问限制,但是超过了次数之后,就无法访问这个接口了,这不是我们想要的,期望的是限制一段时间之后,用户可以重新访问这个API
添加过期时间
改造一下上面的代码,对访问的限制添加一个过期时间,如果超过了限制了,会在一段时间之后,就可以继续访问了
[HttpGet]
[Route("~/api/helloworld")]
public HttpResponseMessage HelloWorld()
{
ThrottleInfo throttleInfo = (ThrottleInfo)HttpRuntime.Cache["throttle"];
if (throttleInfo == null)
throttleInfo = new ThrottleInfo {
ExpiresAt = DateTime.Now.AddSeconds(10), RequestCount = 0 };
throttleInfo.RequestCount++;
HttpRuntime.Cache.Add("throttle", throttleInfo, null,
throttleInfo.ExpiresAt, Cache.NoSlidingExpiration,
CacheItemPriority.Normal, null);
if (throttleInfo.RequestCount > 10)
return Request.CreateResponse((HttpStatusCode)429, "Too many requests");
return Request.CreateResponse(HttpStatusCode.OK, "Hello World");
}
private class ThrottleInfo
{
public DateTime ExpiresAt { get; set; }
public int RequestCount { get; set; }
}
复制代码
这样访问超过了限制,等一段时间,就可以继续访问了
封装一下代码,将访问限制的代码提取出来
public HttpResponseMessage HelloWorld()
{
var throttler = new Throttler("helloworld");
if (throttler.RequestShouldBeThrottled())
return Request.CreateResponse(
(HttpStatusCode)429, "Too many requests");
return Request.CreateResponse(HttpStatusCode.OK, "Hello World");
}
复制代码
public class Throttler
{
private int _requestLimit;
private int _timeoutInSeconds;
private string _key;
public Throttler(string key, int requestLimit = 5, int timeoutInSeconds = 10)
{
_requestLimit = requestLimit;
_timeoutInSeconds = timeoutInSeconds;
_key = key;
}
public bool RequestShouldBeThrottled()
{
ThrottleInfo throttleInfo = (ThrottleInfo)HttpRuntime.Cache[_key];
if (throttleInfo == null) throttleInfo = new ThrottleInfo {
ExpiresAt = DateTime.Now.AddSeconds(_timeoutInSeconds),
RequestCount = 0
};
throttleInfo.RequestCount++;
HttpRuntime.Cache.Add(_key,
throttleInfo,
null,
throttleInfo.ExpiresAt,
Cache.NoSlidingExpiration,
CacheItemPriority.Normal,
null);
return (throttleInfo.RequestCount > _requestLimit);
}
private class ThrottleInfo
{
public DateTime ExpiresAt { get; set; }
public int RequestCount { get; set; }
}
}
复制代码
不一定需要依赖 HttpRuntime.Cache
,使用 ConcurrentDictionary
实现cache
public class Throttler
{
private int _requestLimit;
private int _timeoutInSeconds;
private string _key;
private static ConcurrentDictionary<string, ThrottleInfo> _cache =
new ConcurrentDictionary<string, ThrottleInfo>();
public Throttler(string key, int requestLimit = 5, int timeoutInSeconds = 10)
{
_requestLimit = requestLimit;
_timeoutInSeconds = timeoutInSeconds;
_key = key;
}
public bool RequestShouldBeThrottled()
{
ThrottleInfo throttleInfo = _cache.ContainsKey(_key) ? _cache[_key] : null;
if (throttleInfo == null || throttleInfo.ExpiresAt <= DateTime.Now)
{
throttleInfo = new ThrottleInfo {
ExpiresAt = DateTime.Now.AddSeconds(_timeoutInSeconds),
RequestCount = 0};
};
throttleInfo.RequestCount++;
_cache[_key] = throttleInfo;
return (throttleInfo.RequestCount > _requestLimit);
}
private class ThrottleInfo
{
public DateTime ExpiresAt { get; set; }
public int RequestCount { get; set; }
}
}
复制代码
使用user id作为key,也可以使用IP地址作为key
- UserName作为key
[HttpGet]
[Route("~/api/helloworld")]
public HttpResponseMessage HelloWorld()
{
var throttler = new Throttler(User.Identity.Name);
}
复制代码
- 使用IP地址作为key
[HttpGet]
[Route("~/api/helloworld")]
public HttpResponseMessage HelloWorld()
{
var ipAddress = System.Web.HttpContext.Current.Request.ServerVariables["REMOTE_ADDR"];
var throttler = new Throttler(ipAddress);
}
复制代码
让客户端知道访问限制,已经过期时间等
封装一下返回结果
public HttpResponseMessage HelloWorld()
{
var throttler = new Throttler(User.Identity.Name);
HttpResponseMessage response = createResponse("Hello World", throttler);
return response;
}
private HttpResponseMessage createResponse(object content, Throttler throttler)
{
HttpResponseMessage response;
if (throttler.RequestShouldBeThrottled())
response = Request.CreateResponse((HttpStatusCode)429, "Too many requests");
else
response = Request.CreateResponse(HttpStatusCode.OK, content);
response.Headers.Add("X-RateLimit-Limit", throttler.RequestLimit.ToString());
response.Headers.Add("X-RateLimit-Remaining", throttler.RequestsRemaining.ToString());
response.Headers.Add("X-RateLimit-Reset", toUnixTime(throttler.WindowResetDate).ToString());
return response;
}
private long toUnixTime(DateTime date)
{
var epoch = new DateTime(1970, 1, 1, 0, 0, 0, DateTimeKind.Utc);
return Convert.ToInt64((date.ToUniversalTime() - epoch).TotalSeconds);
}
复制代码
在返回的结果中添加了三个 header ,重构一下 Throttler
来支持这个三个属性
public class Throttler
{
public int RequestLimit { get; private set; }
public int RequestsRemaining { get; private set; }
public DateTime WindowResetDate { get; private set; }
private static ConcurrentDictionary<string, ThrottleInfo> _cache =
new ConcurrentDictionary<string, ThrottleInfo>();
private string _key;
private int _timeoutInSeconds;
public Throttler(string key, int requestLimit = 5, int timeoutInSeconds = 10)
{
RequestLimit = requestLimit;
_timeoutInSeconds = timeoutInSeconds;
_key = key;
}
public bool RequestShouldBeThrottled()
{
ThrottleInfo throttleInfo = _cache.ContainsKey(_key) ? _cache[_key] : null;
if (throttleInfo == null || throttleInfo.ExpiresAt <= DateTime.Now)
{
throttleInfo = new ThrottleInfo {
ExpiresAt = DateTime.Now.AddSeconds(_timeoutInSeconds),
RequestCount = 0};
};
WindowResetDate = throttleInfo.ExpiresAt;
throttleInfo.RequestCount++;
_cache[ThrottleGroup] = throttleInfo;
RequestsRemaining = Math.Max(RequestLimit - throttleInfo.RequestCount, 0);
return (throttleInfo.RequestCount > RequestLimit);
}
private class ThrottleInfo
{
public DateTime ExpiresAt { get; set; }
public int RequestCount { get; set; }
}
}
复制代码
创建一个属性
using System;
using System.Net;
using System.Net.Http;
using System.Runtime.CompilerServices;
using System.Web;
using System.Web.Http.Controllers;
using System.Web.Http.Filters;
namespace Throttling
{
public class ThrottleFilter : ActionFilterAttribute
{
private Throttler _throttler;
private string _throttleGroup;
public ThrottleFilter(
int RequestLimit = 5,
int TimeoutInSeconds = 10,
[CallerMemberName] string ThrottleGroup = null)
{
_throttleGroup = ThrottleGroup;
_throttler = new Throttler(ThrottleGroup, RequestLimit, TimeoutInSeconds);
}
public override void OnActionExecuting(HttpActionContext actionContext)
{
setIdentityAsThrottleGroup();
if (_throttler.RequestShouldBeThrottled)
{
actionContext.Response = actionContext.Request.CreateResponse(
(HttpStatusCode)429, "Too many requests");
addThrottleHeaders(actionContext.Response);
}
base.OnActionExecuting(actionContext);
}
public override void OnActionExecuted(HttpActionExecutedContext actionExecutedContext)
{
setIdentityAsThrottleGroup();
if (actionExecutedContext.Exception == null) _throttler.IncrementRequestCount();
addThrottleHeaders(actionExecutedContext.Response);
base.OnActionExecuted(actionExecutedContext);
}
private void setIdentityAsThrottleGroup()
{
if (_throttleGroup == "identity")
_throttler.ThrottleGroup = HttpContext.Current.User.Identity.Name;
if (_throttleGroup == "ipaddress")
_throttler.ThrottleGroup = HttpContext.Current.Request.UserHostAddress;
}
private void addThrottleHeaders(HttpResponseMessage response)
{
if (response == null) return;
foreach (var header in _throttler.GetRateLimitHeaders())
response.Headers.Add(header.Key, header.Value);
}
}
}
复制代码
对应的 Throttler
代码
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
namespace Throttling
{
public class Throttler
{
public int RequestLimit { get; private set; }
public int RequestsRemaining { get; private set; }
public DateTime WindowResetDate { get; private set; }
private static ConcurrentDictionary<string, ThrottleInfo> _cache =
new ConcurrentDictionary<string, ThrottleInfo>();
public string ThrottleGroup { get; set; }
private int _timeoutInSeconds;
public Throttler(string key, int requestLimit = 5, int timeoutInSeconds = 10)
{
RequestLimit = requestLimit;
_timeoutInSeconds = timeoutInSeconds;
ThrottleGroup = key;
}
private ThrottleInfo getThrottleInfoFromCache()
{
ThrottleInfo throttleInfo =
_cache.ContainsKey(ThrottleGroup) ? _cache[ThrottleGroup] : null;
if (throttleInfo == null || throttleInfo.ExpiresAt <= DateTime.Now)
{
throttleInfo = new ThrottleInfo
{
ExpiresAt = DateTime.Now.AddSeconds(_timeoutInSeconds),
RequestCount = 0
};
};
return throttleInfo;
}
public bool RequestShouldBeThrottled
{
get
{
ThrottleInfo throttleInfo = getThrottleInfoFromCache();
WindowResetDate = throttleInfo.ExpiresAt;
RequestsRemaining = Math.Max(RequestLimit - throttleInfo.RequestCount, 0);
return (throttleInfo.RequestCount > RequestLimit);
}
}
public void IncrementRequestCount()
{
ThrottleInfo throttleInfo = getThrottleInfoFromCache();
throttleInfo.RequestCount++;
_cache[ThrottleGroup] = throttleInfo;
}
private class ThrottleInfo
{
public DateTime ExpiresAt { get; set; }
public int RequestCount { get; set; }
}
public Dictionary<string,string> GetRateLimitHeaders()
{
ThrottleInfo throttleInfo = getThrottleInfoFromCache();
int requestsRemaining = Math.Max(RequestLimit - throttleInfo.RequestCount, 0);
var headers = new Dictionary<string,string>();
headers.Add("X-RateLimit-Limit", RequestLimit.ToString());
headers.Add("X-RateLimit-Remaining", RequestsRemaining.ToString());
headers.Add("X-RateLimit-Reset", toUnixTime(throttleInfo.ExpiresAt).ToString());
return headers;
}
private long toUnixTime(DateTime date)
{
var epoch = new DateTime(1970, 1, 1, 0, 0, 0, DateTimeKind.Utc);
return Convert.ToInt64((date.ToUniversalTime() - epoch).TotalSeconds);
}
}
}
复制代码
如何使用
- 基本使用
[ThrottleFilter()]
[HttpGet]
[Route("~/api/helloworld")]
public HttpResponseMessage HelloWorld()
{
return Request.CreateResponse(HttpStatusCode.OK, "Hello World");
}
复制代码
- 添加限制
[ThrottleFilter(RequestLimit: 50, TimeoutInSeconds: 5)]
[HttpGet]
[Route("~/api/allow-more")]
public HttpResponseMessage HelloWorld2()
{
return Request.CreateResponse(HttpStatusCode.OK, "Hello World2");
}
复制代码
- 使用IP地址
[ThrottleFilter(ThrottleGroup: "ipaddress")]
[HttpGet]
[Route("~/api/name")]
public HttpResponseMessage GetName(int id)
{
return Request.CreateResponse(HttpStatusCode.OK, "John Smith");
}
复制代码
- 使用user Id
[ThrottleFilter(ThrottleGroup: "identity")]
[HttpGet]
[Route("~/api/name")]
public HttpResponseMessage GetName(int id)
{
return Request.CreateResponse(HttpStatusCode.OK, "Jane Doe");
}
复制代码
也可以使用第三方库实现节流 WebApiThrottle Github
发布者:全栈程序员-用户IM,转载请注明出处:https://javaforall.cn/107391.html原文链接:https://javaforall.cn
【正版授权,激活自己账号】: Jetbrains全家桶Ide使用,1年售后保障,每天仅需1毛
【官方授权 正版激活】: 官方授权 正版激活 支持Jetbrains家族下所有IDE 使用个人JB账号...