读这篇文章之前,有没有想过,自己了解Http的缓存吗?知道Http 304状态码吗?
一、Http缓存基本知识
说到缓存,肯定会想到缓存的时效性,即什么时候读缓存,什么时候更新缓存。
Http缓存分为强制缓存和协商缓存,两种缓存策略,涉及到的策略是不同的,主要是header的不同,我们这里通过一张表格来看下
响应码 | Header字段 | |
强制缓存 | 200 |
Pragma Expire Cache-Control |
协商缓存 | 304 |
Etag/If-None-Match Last-Modified/If-Modified-Since |
首先,我们知道,缓存是第二次Http请求的,首次请求不涉及缓存问题。对于第二次请求,如果命中强制缓存时,直接返回200,如果没有,就在请求头中添加参数,看能否命中协商缓存。接下来我们看下各个header参数的含义
Header位置 | 常用值 | 说明 | 优先级 | |
Pragma | 响应头 | no-cache |
适用于HTTP /1.0 HTTP /1.1中已废弃 | 高 |
Expire | 响应头 | 过期时间 | 低 | |
Cache-Control | 响应头 | no-cache, no-store, max-age, public, private |
1)不适用于HTTP /1.0 2)适用于HTTP /1.1 2)响应头和请求头都支持这个属性 3)缓存失效前,获取不到新资源 | 中 |
Etag | 响应头 | 文件Hash值 | 高 | |
If-None-Match | 请求头 | 响应头中的Etag值 | ||
Last-Modified | 响应头 | 资源最近更新时间 | 中 | |
If-Modified-Since | 请求头 | 响应头中的Last-Modified的时间 |
通过上面这张表,我们总结下,Pragma是HTTP /1.0的遗留产物,我们可以不关心了;Cache-Control的优先级高于Expire的,之后我们看下Cache-Control几个值的意义
no-cache: 需要使用协商缓存来验证缓存数据,字面意思意思比较有迷惑性
no-store: 不会缓存,协商缓存和强制缓存都不会触发
max-age: 强制缓存会触发,值的含义是多少秒后失效,类似Expire
public: 客户端和服务端都可以缓存
private: 仅客户端可以缓存
可以看出,强制缓存是通过Cache-Control的max-age和Expire来实现,它的缺点是服务端资源已经更新,但是只要在有效期内,就不会获取新资源,明显是不够灵活的,因此,就需要协商缓存。协商缓存会涉及的字段是:Last-Modified/If-Modified-Since还有Etag/If-None-Match
Last-Modified客户端第一次请求后,服务端会在响应头中返回这个字段,告诉客户端资源最近的更新时间,在客户端第二次请求时候,通过请求头里面If-Modified-Since字段,携带上次资源的更新时间,服务端收到后,对比资源的最近修改时间,如果没有新的资源更新,也就是比If-Modified-Since的值小,直接返回304,客户端需要使用缓存,如果比If-Modified-Since的值大,说明资源更新了,需要下载最新资源,返回最新资源,状态码是200
Last-Modified/If-Modified-Since也是存在缺点的,主要有两点:
1)精确度是秒级,在一秒内修改了,就会存在误差
2)更新时间来作为标识资源唯一性,具有不可靠性,时间变化了,内容不一定更改
因此,需要一个优先级更高的协商机制
Etag是文件的hash值,具有唯一性,第一次服务端会返回给客户端,客户端第二次请求的时候,会通过If-None-Match来携带给服务端,服务端通过对比If-None-Match携带的hash值和文件的Etag是否一致,如果一直,返回304,客户端使用缓存,如果不一致,返回200状态码,返回最新资源。
通过一张图来做一个总结
二、OkHttp缓存流程分析
首先看下CacheInterceptor的核心代码
override fun intercept(chain: Interceptor.Chain): Response {
val call = chain.call()
//1
val cacheCandidate = cache?.get(chain.request())
val now = System.currentTimeMillis()
//2
val strategy = CacheStrategy.Factory(now, chain.request(), cacheCandidate).compute()
//3
val networkRequest = strategy.networkRequest
val cacheResponse = strategy.cacheResponse
cache?.trackResponse(strategy)
val listener = (call as? RealCall)?.eventListener ?: EventListener.NONE
//4
if (cacheCandidate != null && cacheResponse == null) {
// The cache candidate wasn't applicable. Close it.
cacheCandidate.body?.closeQuietly()
}
// If we're forbidden from using the network and the cache is insufficient, fail.
//5
if (networkRequest == null && cacheResponse == null) {
return Response.Builder()
.request(chain.request())
.protocol(Protocol.HTTP_1_1)
.code(HTTP_GATEWAY_TIMEOUT)
.message("Unsatisfiable Request (only-if-cached)")
.body(EMPTY_RESPONSE)
.sentRequestAtMillis(-1L)
.receivedResponseAtMillis(System.currentTimeMillis())
.build().also {
listener.satisfactionFailure(call, it)
}
}
//6
// If we don't need the network, we're done.
if (networkRequest == null) {
return cacheResponse!!.newBuilder()
.cacheResponse(stripBody(cacheResponse))
.build().also {
listener.cacheHit(call, it)
}
}
if (cacheResponse != null) {
listener.cacheConditionalHit(call, cacheResponse)
} else if (cache != null) {
listener.cacheMiss(call)
}
var networkResponse: Response? = null
try {
//7
networkResponse = chain.proceed(networkRequest)
} finally {
// If we're crashing on I/O or otherwise, don't leak the cache body.
if (networkResponse == null && cacheCandidate != null) {
cacheCandidate.body?.closeQuietly()
}
}
// If we have a cache response too, then we're doing a conditional get.
//8
if (cacheResponse != null) {
if (networkResponse?.code == HTTP_NOT_MODIFIED) {
val response = cacheResponse.newBuilder()
.headers(combine(cacheResponse.headers, networkResponse.headers))
.sentRequestAtMillis(networkResponse.sentRequestAtMillis)
.receivedResponseAtMillis(networkResponse.receivedResponseAtMillis)
.cacheResponse(stripBody(cacheResponse))
.networkResponse(stripBody(networkResponse))
.build()
networkResponse.body!!.close()
// Update the cache after combining headers but before stripping the
// Content-Encoding header (as performed by initContentStream()).
cache!!.trackConditionalCacheHit()
cache.update(cacheResponse, response)
return response.also {
listener.cacheHit(call, it)
}
} else {
cacheResponse.body?.closeQuietly()
}
}
//9
val response = networkResponse!!.newBuilder()
.cacheResponse(stripBody(cacheResponse))
.networkResponse(stripBody(networkResponse))
.build()
//10
if (cache != null) {
if (response.promisesBody() && CacheStrategy.isCacheable(response, networkRequest)) {
// Offer this request to the cache.
val cacheRequest = cache.put(response)
return cacheWritingResponse(cacheRequest, response).also {
if (cacheResponse != null) {
// This will log a conditional cache miss only.
listener.cacheMiss(call)
}
}
}
//11
if (HttpMethod.invalidatesCache(networkRequest.method)) {
try {
cache.remove(networkRequest)
} catch (_: IOException) {
// The cache cannot be written.
}
}
}
return response
}
根据注释标号,我们大致来梳理一下CacheInterceptor的流程
- 先判断是否有候选/可用的缓存,如果有,则取出作为候选
- 通过候选缓存,获取缓存策略,得到网络请求
- 通过缓存策略,得到缓存的响应数据
- 缓存的响应数据为null,则清理候选缓存
- 如果网络请求为null,并且缓存的响应数据也为null,则报网络错误504
- 如果只是网络请求为null,则返回缓存的响应数据
- 上面的缓存无效,则在次步进行网络请求
- 通过服务端校验后,缓存可用,返回304,不可用,返回数据,更新缓存
- 以上条件都不满足的情况下,即缓存的响应数据不为null,同时也没有返回304,则构建一个响应数据
- 将响应数据缓存
- 校验请求的方法,如果是"POST","PATCH","PUT","DELETE","MOVE"方法,则从缓存中移除这个请求
条件判断就是判断网络请求和缓存的响应数据,通过下面一个表做一下总结:
networkRequest | cacheResponse | CacheStrategy |
---|---|---|
null | null | 返回 503 错误,步骤5 |
null | not-null | 不进行网络请求,并且缓存可以用,直接返回缓存,步骤6 |
not-null | not-null | 判断header中是否含有ETag/Last-Modified标识,可能返回304,也可能缓存不可用,无论何种都需要访问网络,步骤8 |
non-null | null | 需要进行网络请求,而且缓存不可用,需要直接访问网络,步骤9 |
大致的整体流程我们就介绍完了,可以看出,和Http的缓存流程是十分相似的,OkHttp的缓存策略在CacheStrategy这个类中,我们接下来看下它的源码
三、缓存源码分析
先来看CacheStrategy的Factory中的init方法
init {
if (cacheResponse != null) {
this.sentRequestMillis = cacheResponse.sentRequestAtMillis
this.receivedResponseMillis = cacheResponse.receivedResponseAtMillis
val headers = cacheResponse.headers
for (i in 0 until headers.size) {
val fieldName = headers.name(i)
val value = headers.value(i)
when {
fieldName.equals("Date", ignoreCase = true) -> {
servedDate = value.toHttpDateOrNull()
servedDateString = value
}
fieldName.equals("Expires", ignoreCase = true) -> {
expires = value.toHttpDateOrNull()
}
fieldName.equals("Last-Modified", ignoreCase = true) -> {
lastModified = value.toHttpDateOrNull()
lastModifiedString = value
}
fieldName.equals("ETag", ignoreCase = true) -> {
etag = value
}
fieldName.equals("Age", ignoreCase = true) -> {
ageSeconds = value.toNonNegativeInt(-1)
}
}
}
}
}
我们看到了,在初始化时,会读取缓存响应数据里面的响应头信息,我们前面提到的"Expires","Last-Modified","ETag"等,接下来看下他的compute()方法
private fun computeCandidate(): CacheStrategy {
// No cached response.
//没有缓存数据,返回不带缓存响应的缓存策略
if (cacheResponse == null) {
return CacheStrategy(request, null)
}
// Drop the cached response if it's missing a required handshake.
if (request.isHttps && cacheResponse.handshake == null) {
return CacheStrategy(request, null)
}
// If this response shouldn't have been stored, it should never be used as a response source.
// This check should be redundant as long as the persistence store is well-behaved and the
// rules are constant.
//判断cacheControl参数,涉及“Expires”、“maxAgeSeconds”、“isPublic”、“isPrivate”
if (!isCacheable(cacheResponse, request)) {
return CacheStrategy(request, null)
}
//请求头中判断是否有noCache,或者"If-Modified-Since" 或者"If-None-Match",需要服务器决策,则返回只包含网络请求的策略
val requestCaching = request.cacheControl
if (requestCaching.noCache || hasConditions(request)) {
return CacheStrategy(request, null)
}
val responseCaching = cacheResponse.cacheControl
val ageMillis = cacheResponseAge()
var freshMillis = computeFreshnessLifetime()
if (requestCaching.maxAgeSeconds != -1) {
freshMillis = minOf(freshMillis, SECONDS.toMillis(requestCaching.maxAgeSeconds.toLong()))
}
var minFreshMillis: Long = 0
if (requestCaching.minFreshSeconds != -1) {
minFreshMillis = SECONDS.toMillis(requestCaching.minFreshSeconds.toLong())
}
var maxStaleMillis: Long = 0
if (!responseCaching.mustRevalidate && requestCaching.maxStaleSeconds != -1) {
maxStaleMillis = SECONDS.toMillis(requestCaching.maxStaleSeconds.toLong())
}
//如果缓存在过期时间内则可以直接使用,直接返回上次缓存
if (!responseCaching.noCache && ageMillis + minFreshMillis < freshMillis + maxStaleMillis) {
val builder = cacheResponse.newBuilder()
if (ageMillis + minFreshMillis >= freshMillis) {
builder.addHeader("Warning", "110 HttpURLConnection \"Response is stale\"")
}
val oneDayMillis = 24 * 60 * 60 * 1000L
if (ageMillis > oneDayMillis && isFreshnessLifetimeHeuristic()) {
builder.addHeader("Warning", "113 HttpURLConnection \"Heuristic expiration\"")
}
return CacheStrategy(null, builder.build())
}
// Find a condition to add to the request. If the condition is satisfied, the response body
// will not be transmitted.
val conditionName: String
val conditionValue: String?
//如果缓存过期,且有ETag、lastModified等信息,则添加If-None-Match、If-Modified-Since等条件请求
when {
etag != null -> {
conditionName = "If-None-Match"
conditionValue = etag
}
lastModified != null -> {
conditionName = "If-Modified-Since"
conditionValue = lastModifiedString
}
servedDate != null -> {
conditionName = "If-Modified-Since"
conditionValue = servedDateString
}
else -> return CacheStrategy(request, null) // No condition! Make a regular request.
}
val conditionalRequestHeaders = request.headers.newBuilder()
conditionalRequestHeaders.addLenient(conditionName, conditionValue!!)
val conditionalRequest = request.newBuilder()
.headers(conditionalRequestHeaders.build())
.build()
return CacheStrategy(conditionalRequest, cacheResponse)
}
方法比较长,关键部分直接做了注释,基本上都是Http缓存头信息的一些判断和处理逻辑,那么还有一个问题,上面我们说的都是Http缓存策略在OkHttp中的具体实现,那么缓存的文件在客户端存到哪里呢?由谁来实现呢?是由Cache类来实现的,我们在CacheInterceptor的构造方法中,就会把它传进来,可以往上面看下源码,这里面我们直接看下Cache的put方法
internal fun put(response: Response): CacheRequest? {
val requestMethod = response.request.method
if (HttpMethod.invalidatesCache(response.request.method)) {
try {
remove(response.request)
} catch (_: IOException) {
// The cache cannot be written.
}
return null
}
if (requestMethod != "GET") {
// Don't cache non-GET responses. We're technically allowed to cache HEAD requests and some
// POST requests, but the complexity of doing so is high and the benefit is low.
return null
}
if (response.hasVaryAll()) {
return null
}
val entry = Entry(response)
var editor: DiskLruCache.Editor? = null
try {
editor = cache.edit(key(response.request.url)) ?: return null
entry.writeTo(editor)
return RealCacheRequest(editor)
} catch (_: IOException) {
abortQuietly(editor)
return null
}
}
上面的put方法,我们要注意两点,第一点是,缓存只Get方法的请求,第二点是,具体缓存是通过DiskLruCache方法来实现的,使用时设置在磁盘上缓存空间的大小,通过LRU算法进行缓存淘汰
四、总结
- OkHttp的缓存机制是按照Http的缓存机制实现的
- 缓存的具体实现是通过DiskLruCache来完成的,但是默认是不开启缓存的,如果我们需要使用,需要用户手动设置OkHttpClient的Cache缓存对象
- OkHttp默认只支持缓存Get方法的请求