1 Traffic Server插件cache_range_request

最新推荐文章于 2024-07-16 05:22:49 发布
最新推荐文章于 2024-07-16 05:22:49 发布
阅读量706 收藏 1
点赞数
CC 4.0 BY-SA版权
分类专栏： ATS&&Nginx
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.youkuaiyun.com/sinat_24820331/article/details/75247979
本文介绍了一种针对TrafficServer不支持Range请求的问题解决方案。通过将Range请求转化为不同存储对象，以减少磁盘IO请求，提高处理效率。同时，讨论了当源站支持或不支持Range请求时的具体处理流程。

摘要生成于 C知道 ，由 DeepSeek-R1 满血版支持， 前往体验 >

图上的transform写错了 而是transaction

很显然 设置了cache-key后得查cache 所以得启remapInit操作

感觉这个插件案例是一个分水岭 用到了example里基本的东西 

但核心还是1 TSPluginInit负责启 read_req阶段的判断 捕获参数 而后 又启一个协程把2/3/4事件钩上去 

从而两个协程的回调不断轮序相应的事件   


抛开机制 说一下原理：

！！！首先traffic server不支持range和206！！！
因为大量的range请求会造成磁盘大量IO负载(IO等待) 这是因为存储内容分布在单独的一个stripes里
所以还是把不同的range内容做成不同的单独的存储对象 这些对象就均匀的分布在多个stripes里  可减少磁盘IO请求 
用户请求带range时 我们的处理是把range变成key 
再发送源请求(假设源站支持range请求) 返回206修改为200 


若源站不支持range请求则返回的是全部内容并返回200 
则将全部文件进行修剪 这就是另外一个插件了

HTTP范围请求
QingtianD的博客
02-29 977
​ 人们下载文件时断网重连之后，需要一种可恢复的机制。所谓恢复是指能从之前下载中断处恢复下载。 ​ 要实现该功能需要指定下载的实体范围。像这样，指定范围发送的请求叫做范围请求（Range Request）。基于范围请求也可实现多线程分片下载。 ​ 只有双端（客户端和服务器端）都支持HTTP/1.1才能使用范围请求。 1.范围请求的请求报文 ​ 在HTTP/1.1的请求报文中，有一个首部字段...
Nginx架构及配置详解
企业实战系列集 ●●● https://ximenjianxue.blog.youkuaiyun.com
09-10 5929
概述 Nginx采用C进行编写，是俄罗斯程序员开发的一款轻量级的可构建Web 服务器/反向代理服务器及电子邮件（IMAP/POP3）代理服务器的软件， Nginx服务器是一个高性能的HTTP和反向代理web服务器，同时也可提供了IMAP/POP3/SMTP服务。nginx常用作跑静态和做负载反向代理，动态php交给apache处理，因后者比较稳定，jsp交给tomcat、resin或jboss。n......
ats_pagespeed:由Google PageSpeed支持的Apache Traffic Server Web内容优化插件
05-15
由Google PageSpeed支持的Apache Traffic Server Web内容优化插件 要构建，一个简单的“ make”应该可以工作。 使用“ sudo make install”进行安装。 （可选）使用ethread.patch修补ATS有助于消除在同步ATS和PSOL线程池时有时引起的延迟。 之后，使用以下命令更新ATS的plugin.config： ats_pagespeed.so gzip.so /usr/local/etc/trafficserver/gzip.config gzip.so也是使用ats_pagespeed构建的，因为它是ATS存储库中官方版本的略微修改版本。 插件中有一些硬编
ts-flv:FLV流媒体，作为Apache TrafficServer的插件实现
07-03
名称 ts-flv - FLV 流媒体，作为 Apache TrafficServer 的插件实现。 地位 该模块正在积极开发中。 描述 该模块为 FLV 文件提供流媒体服务器支持。 用户可以使用以秒为单位的start参数向服务器发送 HTTP 请求，服务器将响应流，使其开始位置对应于请求的时间，例如： http: // v.foo.com/kenshin.flv?start=187.63 这允许随时执行随机搜索。 我们可以使用flash player、vlc、mplayer来播放流媒体。 我们可以在 remap.config 中这样写： map http: // v.foo.com/ http://i.foo.com/ @plugin=/xx/libtsflv.so 系统要求 linux/freebsd 64位 流量服务器 建造 步骤1 ：获取ts-flv git clone
入门TrafficServer插件开发的正确姿势
alphaCat的专栏
04-16 4014
入门TrafficServer插件开发的正确姿势–本文转自：纸鸢-可道 可道技术前言　普及文章，高手飘过。 发现不少同学刚搞ats插件开发时候经常找不着道。　举个栗子： 　通过阅读插件代码，发现对ats的处理流程、request/respone方向没弄清楚。 如果不先搞清楚简单的流程、request/respone方向问题，会陷入坑中。下面我们一起来看看在搞ats插件之前，我们需要搞清
RFC7233 HTTP范围请求(Range Requests)
博客
08-11 6826
RFC7233 HTTP范围请求(Range Requests) 目录 介绍(Introduction) 范围单位(Range Units) 2.1 字节(bytes)范围 2.2 其他范围单位 2.3 HTTP响应头部 Accept Ranges 范围请求(Range Reqeust) 3.1 HTTP请求头部 Range 3.2 HTTP请求头部 If-Range 对范围请求的响应
Fetching 334 files: 83%|████████▎ | 277/334 [00:00<00:00, 573.27it/s] Traceback (most recent call last): File "C:\Users\l60080208\AppData\Roaming\Python\Python312\site-packages\huggingface_hub\utils\_http.py", line 409, in hf_raise_for_status response.raise_for_status() File "D:\anaconda3\Lib\site-packages\requests\models.py", line 1024, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 416 Client Error: Requested Range Not Satisfiable for url: https://cdn-lfs-us-1.hf.co/repos/ed/91/ed91c2f4c4a7e464a63e321b1e4a078dfcc8b60bdede6e1c27f731ec9ac46cac/ffb5e932cab2241009ce80b808cb697f26f20409b63c7aa3ba5d1b4213f11fbb?response-content-disposition=inline%3B+filename*%3DUTF-8%27%27data-00029-of-00080.arrow%3B+filename%3D%22data-00029-of-00080.arrow%22%3B&Expires=1751970232&Policy=eyJTdGF0ZW1lbnQiOlt7IkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTc1MTk3MDIzMn19LCJSZXNvdXJjZSI6Imh0dHBzOi8vY2RuLWxmcy11cy0xLmhmLmNvL3JlcG9zL2VkLzkxL2VkOTFjMmY0YzRhN2U0NjRhNjNlMzIxYjFlNGEwNzhkZmNjOGI2MGJkZWRlNmUxYzI3ZjczMWVjOWFjNDZjYWMvZmZiNWU5MzJjYWIyMjQxMDA5Y2U4MGI4MDhjYjY5N2YyNmYyMDQwOWI2M2M3YWEzYmE1ZDFiNDIxM2YxMWZiYj9yZXNwb25zZS1jb250ZW50LWRpc3Bvc2l0aW9uPSoifV19&Signature=tcd~AVP775PxaiR7NwUCHB04ybbwObICy0WkufCUm6dbCweqkHe0A~gIziLO5pYRfk96vjDFxyhvXz~CO7oPSzgfmwryJyp~wbCK6xzi38VX4g51nl8yGUhufO00cNunFGlc8klKzmgnQao2Aohjc8vtKMgGdOpkoaWjSYxgAy10Hvygjj6Zv6EWB2NsVNbr~Or0CKr8hrSCFiD4Lpumi3HJkPUCqRm89wJyibZG~ygQVwH9HdV9P2aydWiBGKNGMxWNHGVyWOie096FHDX-HKWsttKmtPikjpCjo39mzu3H6MdRzN1lZTlVJO0i3GCJukvCt8kmBZuOzQcALEvC7g__&Key-Pair-Id=K24J24Z295AEI9 The above exception was the direct cause of the following exception: Traceback (most recent call last): File "D:\lzh\code\Traffic_prediction\PreTrainDataChoose\download_huggingface_datasets.py", line 40, in <module> snapshot_download( File "C:\Users\l60080208\AppData\Roaming\Python\Python312\site-packages\huggingface_hub\utils\_validators.py", line 114, in _inner_fn return fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "C:\Users\l60080208\AppData\Roaming\Python\Python312\site-packages\huggingface_hub\_snapshot_download.py", line 327, in snapshot_download thread_map( File "D:\anaconda3\Lib\site-packages\tqdm\contrib\concurrent.py", line 69, in thread_map return _executor_map(ThreadPoolExecutor, fn, *iterables, **tqdm_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\anaconda3\Lib\site-packages\tqdm\contrib\concurrent.py", line 51, in _executor_map return list(tqdm_class(ex.map(fn, *iterables, chunksize=chunksize), **kwargs)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\anaconda3\Lib\site-packages\tqdm\std.py", line 1181, in __iter__ for obj in iterable: File "D:\anaconda3\Lib\concurrent\futures\_base.py", line 619, in result_iterator yield _result_or_cancel(fs.pop()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\anaconda3\Lib\concurrent\futures\_base.py", line 317, in _result_or_cancel return fut.result(timeout) ^^^^^^^^^^^^^^^^^^^ File "D:\anaconda3\Lib\concurrent\futures\_base.py", line 456, in result return self.__get_result() ^^^^^^^^^^^^^^^^^^^ File "D:\anaconda3\Lib\concurrent\futures\_base.py", line 401, in __get_result raise self._exception File "D:\anaconda3\Lib\concurrent\futures\thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\l60080208\AppData\Roaming\Python\Python312\site-packages\huggingface_hub\_snapshot_download.py", line 301, in _inner_hf_hub_download return hf_hub_download( ^^^^^^^^^^^^^^^^ File "C:\Users\l60080208\AppData\Roaming\Python\Python312\site-packages\huggingface_hub\utils\_validators.py", line 114, in _inner_fn return fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "C:\Users\l60080208\AppData\Roaming\Python\Python312\site-packages\huggingface_hub\file_download.py", line 988, in hf_hub_download return _hf_hub_download_to_local_dir( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\l60080208\AppData\Roaming\Python\Python312\site-packages\huggingface_hub\file_download.py", line 1290, in _hf_hub_download_to_local_dir _download_to_tmp_and_move( File "C:\Users\l60080208\AppData\Roaming\Python\Python312\site-packages\huggingface_hub\file_download.py", line 1725, in _download_to_tmp_and_move http_get( File "C:\Users\l60080208\AppData\Roaming\Python\Python312\site-packages\huggingface_hub\file_download.py", line 420, in http_get r = _request_wrapper( ^^^^^^^^^^^^^^^^^ File "C:\Users\l60080208\AppData\Roaming\Python\Python312\site-packages\huggingface_hub\file_download.py", line 310, in _request_wrapper hf_raise_for_status(response) File "C:\Users\l60080208\AppData\Roaming\Python\Python312\site-packages\huggingface_hub\utils\_http.py", line 478, in hf_raise_for_status raise _format(HfHubHTTPError, message, response) from e huggingface_hub.errors.HfHubHTTPError: 416 Client Error: Requested Range Not Satisfiable for url: https://cdn-lfs-us-1.hf.co/repos/ed/91/ed91c2f4c4a7e464a63e321b1e4a078dfcc8b60bdede6e1c27f731ec9ac46cac/ffb5e932cab2241009ce80b808cb697f26f20409b63c7aa3ba5d1b4213f11fbb?response-content-disposition=inline%3B+filename*%3DUTF-8%27%27data-00029-of-00080.arrow%3B+filename%3D%22data-00029-of-00080.arrow%22%3B&Expires=1751970232&Policy=eyJTdGF0ZW1lbnQiOlt7IkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTc1MTk3MDIzMn19LCJSZXNvdXJjZSI6Imh0dHBzOi8vY2RuLWxmcy11cy0xLmhmLmNvL3JlcG9zL2VkLzkxL2VkOTFjMmY0YzRhN2U0NjRhNjNlMzIxYjFlNGEwNzhkZmNjOGI2MGJkZWRlNmUxYzI3ZjczMWVjOWFjNDZjYWMvZmZiNWU5MzJjYWIyMjQxMDA5Y2U4MGI4MDhjYjY5N2YyNmYyMDQwOWI2M2M3YWEzYmE1ZDFiNDIxM2YxMWZiYj9yZXNwb25zZS1jb250ZW50LWRpc3Bvc2l0aW9uPSoifV19&Signature=tcd~AVP775PxaiR7NwUCHB04ybbwObICy0WkufCUm6dbCweqkHe0A~gIziLO5pYRfk96vjDFxyhvXz~CO7oPSzgfmwryJyp~wbCK6xzi38VX4g51nl8yGUhufO00cNunFGlc8klKzmgnQao2Aohjc8vtKMgGdOpkoaWjSYxgAy10Hvygjj6Zv6EWB2NsVNbr~Or0CKr8hrSCFiD4Lpumi3HJkPUCqRm89wJyibZG~ygQVwH9HdV9P2aydWiBGKNGMxWNHGVyWOie096FHDX-HKWsttKmtPikjpCjo39mzu3H6MdRzN1lZTlVJO0i3GCJukvCt8kmBZuOzQcALEvC7g__&Key-Pair-Id=K24J24Z295AEI9. Requested range: bytes=12861552-. Content-Range: bytes */12861552. 416 Client Error: Requested Range Not Satisfiable for url: https://cdn-lfs-us-1.hf.co/repos/ed/91/ed91c2f4c4a7e464a63e321b1e4a078dfcc8b60bdede6e1c27f731ec9ac46cac/ffb5e932cab2241009ce80b808cb697f26f20409b63c7aa3ba5d1b4213f11fbb?response-content-disposition=inline%3B+filename*%3DUTF-8%27%27data-00029-of-00080.arrow%3B+filename%3D%22data-00029-of-00080.arrow%22%3B&Expires=1751970232&Policy=eyJTdGF0ZW1lbnQiOlt7IkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTc1MTk3MDIzMn19LCJSZXNvdXJjZSI6Imh0dHBzOi8vY2RuLWxmcy11cy0xLmhmLmNvL3JlcG9zL2VkLzkxL2VkOTFjMmY0YzRhN2U0NjRhNjNlMzIxYjFlNGEwNzhkZmNjOGI2MGJkZWRlNmUxYzI3ZjczMWVjOWFjNDZjYWMvZmZiNWU5MzJjYWIyMjQxMDA5Y2U4MGI4MDhjYjY5N2YyNmYyMDQwOWI2M2M3YWEzYmE1ZDFiNDIxM2YxMWZiYj9yZXNwb25zZS1jb250ZW50LWRpc3Bvc2l0aW9uPSoifV19&Signature=tcd~AVP775PxaiR7NwUCHB04ybbwObICy0WkufCUm6dbCweqkHe0A~gIziLO5pYRfk96vjDFxyhvXz~CO7oPSzgfmwryJyp~wbCK6xzi38VX4g51nl8yGUhufO00cNunFGlc8klKzmgnQao2Aohjc8vtKMgGdOpkoaWjSYxgAy10Hvygjj6Zv6EWB2NsVNbr~Or0CKr8hrSCFiD4Lpumi3HJkPUCqRm89wJyibZG~ygQVwH9HdV9P2aydWiBGKNGMxWNHGVyWOie096FHDX-HKWsttKmtPikjpCjo39mzu3H6MdRzN1lZTlVJO0i3GCJukvCt8kmBZuOzQcALEvC7g__&Key-Pair-Id=K24J24Z295AEI9. Requested range: bytes=12861552-. Content-Range: bytes */12861552. Fetching 334 files: 83%|████████▎ | 277/334 [00:00<00:00, 500.26it/s] Traceback (most recent call last): File "C:\Users\l60080208\AppData\Roaming\Python\Python312\site-packages\huggingface_hub\utils\_http.py", line 409, in hf_raise_for_status response.raise_for_status() File "D:\anaconda3\Lib\site-packages\requests\models.py", line 1024, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 416 Client Error: Requested Range Not Satisfiable for url: https://cdn-lfs-us-1.hf.co/repos/ed/91/ed91c2f4c4a7e464a63e321b1e4a078dfcc8b60bdede6e1c27f731ec9ac46cac/ffb5e932cab2241009ce80b808cb697f26f20409b63c7aa3ba5d1b4213f11fbb?response-content-disposition=inline%3B+filename*%3DUTF-8%27%27data-00029-of-00080.arrow%3B+filename%3D%22data-00029-of-00080.arrow%22%3B&Expires=1751970317&Policy=eyJTdGF0ZW1lbnQiOlt7IkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTc1MTk3MDMxN319LCJSZXNvdXJjZSI6Imh0dHBzOi8vY2RuLWxmcy11cy0xLmhmLmNvL3JlcG9zL2VkLzkxL2VkOTFjMmY0YzRhN2U0NjRhNjNlMzIxYjFlNGEwNzhkZmNjOGI2MGJkZWRlNmUxYzI3ZjczMWVjOWFjNDZjYWMvZmZiNWU5MzJjYWIyMjQxMDA5Y2U4MGI4MDhjYjY5N2YyNmYyMDQwOWI2M2M3YWEzYmE1ZDFiNDIxM2YxMWZiYj9yZXNwb25zZS1jb250ZW50LWRpc3Bvc2l0aW9uPSoifV19&Signature=sJzVhc2ThL8upQq2M3v-AWGp-9qfV~a9tXtlGZdhmP~2A5U-AftEOGDxVzWjfxIpvITnmv7brTV-CVqqrAnu23R63Pt~s-LOpkWt4Gr0iMIQxCm53JypggiDKf-5hiy2eK3unLTRj7y8-yXOaGJ-SP3vAYkTeCth6KBotInsFGowzMQ1YoXApvCrElpY0X935xSHMnmtainvmCwvA~qXVGpNJuqGR329oqUQIkvwFIM9c55S7iwlal4nd3L2yJwAQb6hGqibb4vISjVdJtiGJl~iRID98o~pqtg~DLXUBYfA9y9XOchtvyf0GMn6UghCbnoEEpqRBNn52SB26Q-jSQ__&Key-Pair-Id=K24J24Z295AEI9 The above exception was the direct cause of the following exception: Traceback (most recent call last): File "D:\lzh\code\Traffic_prediction\PreTrainDataChoose\download_huggingface_datasets.py", line 40, in <module> snapshot_download( File "C:\Users\l60080208\AppData\Roaming\Python\Python312\site-packages\huggingface_hub\utils\_validators.py", line 114, in _inner_fn return fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "C:\Users\l60080208\AppData\Roaming\Python\Python312\site-packages\huggingface_hub\_snapshot_download.py", line 327, in snapshot_download thread_map( File "D:\anaconda3\Lib\site-packages\tqdm\contrib\concurrent.py", line 69, in thread_map return _executor_map(ThreadPoolExecutor, fn, *iterables, **tqdm_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\anaconda3\Lib\site-packages\tqdm\contrib\concurrent.py", line 51, in _executor_map return list(tqdm_class(ex.map(fn, *iterables, chunksize=chunksize), **kwargs)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\anaconda3\Lib\site-packages\tqdm\std.py", line 1181, in __iter__ for obj in iterable: File "D:\anaconda3\Lib\concurrent\futures\_base.py", line 619, in result_iterator yield _result_or_cancel(fs.pop()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\anaconda3\Lib\concurrent\futures\_base.py", line 317, in _result_or_cancel return fut.result(timeout) ^^^^^^^^^^^^^^^^^^^ File "D:\anaconda3\Lib\concurrent\futures\_base.py", line 456, in result return self.__get_result() ^^^^^^^^^^^^^^^^^^^ File "D:\anaconda3\Lib\concurrent\futures\_base.py", line 401, in __get_result raise self._exception File "D:\anaconda3\Lib\concurrent\futures\thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\l60080208\AppData\Roaming\Python\Python312\site-packages\huggingface_hub\_snapshot_download.py", line 301, in _inner_hf_hub_download return hf_hub_download( ^^^^^^^^^^^^^^^^ File "C:\Users\l60080208\AppData\Roaming\Python\Python312\site-packages\huggingface_hub\utils\_validators.py", line 114, in _inner_fn return fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "C:\Users\l60080208\AppData\Roaming\Python\Python312\site-packages\huggingface_hub\file_download.py", line 988, in hf_hub_download return _hf_hub_download_to_local_dir( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\l60080208\AppData\Roaming\Python\Python312\site-packages\huggingface_hub\file_download.py", line 1290, in _hf_hub_download_to_local_dir _download_to_tmp_and_move( File "C:\Users\l60080208\AppData\Roaming\Python\Python312\site-packages\huggingface_hub\file_download.py", line 1725, in _download_to_tmp_and_move http_get( File "C:\Users\l60080208\AppData\Roaming\Python\Python312\site-packages\huggingface_hub\file_download.py", line 420, in http_get r = _request_wrapper( ^^^^^^^^^^^^^^^^^ File "C:\Users\l60080208\AppData\Roaming\Python\Python312\site-packages\huggingface_hub\file_download.py", line 310, in _request_wrapper hf_raise_for_status(response) File "C:\Users\l60080208\AppData\Roaming\Python\Python312\site-packages\huggingface_hub\utils\_http.py", line 478, in hf_raise_for_status raise _format(HfHubHTTPError, message, response) from e huggingface_hub.errors.HfHubHTTPError: 416 Client Error: Requested Range Not Satisfiable for url: https://cdn-lfs-us-1.hf.co/repos/ed/91/ed91c2f4c4a7e464a63e321b1e4a078dfcc8b60bdede6e1c27f731ec9ac46cac/ffb5e932cab2241009ce80b808cb697f26f20409b63c7aa3ba5d1b4213f11fbb?response-content-disposition=inline%3B+filename*%3DUTF-8%27%27data-00029-of-00080.arrow%3B+filename%3D%22data-00029-of-00080.arrow%22%3B&Expires=1751970317&Policy=eyJTdGF0ZW1lbnQiOlt7IkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTc1MTk3MDMxN319LCJSZXNvdXJjZSI6Imh0dHBzOi8vY2RuLWxmcy11cy0xLmhmLmNvL3JlcG9zL2VkLzkxL2VkOTFjMmY0YzRhN2U0NjRhNjNlMzIxYjFlNGEwNzhkZmNjOGI2MGJkZWRlNmUxYzI3ZjczMWVjOWFjNDZjYWMvZmZiNWU5MzJjYWIyMjQxMDA5Y2U4MGI4MDhjYjY5N2YyNmYyMDQwOWI2M2M3YWEzYmE1ZDFiNDIxM2YxMWZiYj9yZXNwb25zZS1jb250ZW50LWRpc3Bvc2l0aW9uPSoifV19&Signature=sJzVhc2ThL8upQq2M3v-AWGp-9qfV~a9tXtlGZdhmP~2A5U-AftEOGDxVzWjfxIpvITnmv7brTV-CVqqrAnu23R63Pt~s-LOpkWt4Gr0iMIQxCm53JypggiDKf-5hiy2eK3unLTRj7y8-yXOaGJ-SP3vAYkTeCth6KBotInsFGowzMQ1YoXApvCrElpY0X935xSHMnmtainvmCwvA~qXVGpNJuqGR329oqUQIkvwFIM9c55S7iwlal4nd3L2yJwAQb6hGqibb4vISjVdJtiGJl~iRID98o~pqtg~DLXUBYfA9y9XOchtvyf0GMn6UghCbnoEEpqRBNn52SB26Q-jSQ__&Key-Pair-Id=K24J24Z295AEI9. Requested range: bytes=12861552-. Content-Range: bytes */12861552. 416 Client Error: Requested Range Not Satisfiable for url: https://cdn-lfs-us-1.hf.co/repos/ed/91/ed91c2f4c4a7e464a63e321b1e4a078dfcc8b60bdede6e1c27f731ec9ac46cac/ffb5e932cab2241009ce80b808cb697f26f20409b63c7aa3ba5d1b4213f11fbb?response-content-disposition=inline%3B+filename*%3DUTF-8%27%27data-00029-of-00080.arrow%3B+filename%3D%22data-00029-of-00080.arrow%22%3B&Expires=1751970317&Policy=eyJTdGF0ZW1lbnQiOlt7IkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTc1MTk3MDMxN319LCJSZXNvdXJjZSI6Imh0dHBzOi8vY2RuLWxmcy11cy0xLmhmLmNvL3JlcG9zL2VkLzkxL2VkOTFjMmY0YzRhN2U0NjRhNjNlMzIxYjFlNGEwNzhkZmNjOGI2MGJkZWRlNmUxYzI3ZjczMWVjOWFjNDZjYWMvZmZiNWU5MzJjYWIyMjQxMDA5Y2U4MGI4MDhjYjY5N2YyNmYyMDQwOWI2M2M3YWEzYmE1ZDFiNDIxM2YxMWZiYj9yZXNwb25zZS1jb250ZW50LWRpc3Bvc2l0aW9uPSoifV19&Signature=sJzVhc2ThL8upQq2M3v-AWGp-9qfV~a9tXtlGZdhmP~2A5U-AftEOGDxVzWjfxIpvITnmv7brTV-CVqqrAnu23R63Pt~s-LOpkWt4Gr0iMIQxCm53JypggiDKf-5hiy2eK3unLTRj7y8-yXOaGJ-SP3vAYkTeCth6KBotInsFGowzMQ1YoXApvCrElpY0X935xSHMnmtainvmCwvA~qXVGpNJuqGR329oqUQIkvwFIM9c55S7iwlal4nd3L2yJwAQb6hGqibb4vISjVdJtiGJl~iRID98o~pqtg~DLXUBYfA9y9XOchtvyf0GMn6UghCbnoEEpqRBNn52SB26Q-jSQ__&Key-Pair-Id=K24J24Z295AEI9. Requested range: bytes=12861552-. Content-Range: bytes */12861552. Fetching 334 files: 83%|████████▎ | 277/334 [00:00<00:00, 529.18it/s] Traceback (most recent call last): File "C:\Users\l60080208\AppData\Roaming\Python\Python312\site-packages\huggingface_hub\utils\_http.py", line 409, in hf_raise_for_status response.raise_for_status() File "D:\anaconda3\Lib\site-packages\requests\models.py", line 1024, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 416 Client Error: Requested Range Not Satisfiable for url: https://cdn-lfs-us-1.hf.co/repos/ed/91/ed91c2f4c4a7e464a63e321b1e4a078dfcc8b60bdede6e1c27f731ec9ac46cac/ffb5e932cab2241009ce80b808cb697f26f20409b63c7aa3ba5d1b4213f11fbb?response-content-disposition=inline%3B+filename*%3DUTF-8%27%27data-00029-of-00080.arrow%3B+filename%3D%22data-00029-of-00080.arrow%22%3B&Expires=1751970264&Policy=eyJTdGF0ZW1lbnQiOlt7IkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTc1MTk3MDI2NH19LCJSZXNvdXJjZSI6Imh0dHBzOi8vY2RuLWxmcy11cy0xLmhmLmNvL3JlcG9zL2VkLzkxL2VkOTFjMmY0YzRhN2U0NjRhNjNlMzIxYjFlNGEwNzhkZmNjOGI2MGJkZWRlNmUxYzI3ZjczMWVjOWFjNDZjYWMvZmZiNWU5MzJjYWIyMjQxMDA5Y2U4MGI4MDhjYjY5N2YyNmYyMDQwOWI2M2M3YWEzYmE1ZDFiNDIxM2YxMWZiYj9yZXNwb25zZS1jb250ZW50LWRpc3Bvc2l0aW9uPSoifV19&Signature=j3OTpfO4fi500NaPj3rqp-Q7kJ6y6qGT8CrRKnCimUPWXn~zkG4HbtMEsaqbjQT9Ema16pXtKKK3VKpy2VUDZ5aAkMDVlNMQsq9yThxpzXYYuwASUfPek8rONQVqf3KsF2Q22bBPdKJXugvZBC2xzPKmvSdKkP7Ry0QY02F5GP-UD0xAG-wo8vb8I~iyrcX2IFgPGa~wWAKfVnRqHOpFCKtUi6yHAup4R7TvE1xvX5DPtzesDd7ogswJQ66v0Jq8UgwUmTNbZrWBlQG~xtPvEqJB4fXkgcZ7uEEqJHd~X4uT~JtUXW4MZUMs0TEh71xUbC3cc1E4F6Hfo8kyZpbo7A__&Key-Pair-Id=K24J24Z295AEI9 The above exception was the direct cause of the following exception: Traceback (most recent call last): File "D:\lzh\code\Traffic_prediction\PreTrainDataChoose\download_huggingface_datasets.py", line 40, in <module> snapshot_download( File "C:\Users\l60080208\AppData\Roaming\Python\Python312\site-packages\huggingface_hub\utils\_validators.py", line 114, in _inner_fn return fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "C:\Users\l60080208\AppData\Roaming\Python\Python312\site-packages\huggingface_hub\_snapshot_download.py", line 327, in snapshot_download thread_map( File "D:\anaconda3\Lib\site-packages\tqdm\contrib\concurrent.py", line 69, in thread_map return _executor_map(ThreadPoolExecutor, fn, *iterables, **tqdm_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\anaconda3\Lib\site-packages\tqdm\contrib\concurrent.py", line 51, in _executor_map return list(tqdm_class(ex.map(fn, *iterables, chunksize=chunksize), **kwargs)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\anaconda3\Lib\site-packages\tqdm\std.py", line 1181, in __iter__ for obj in iterable: File "D:\anaconda3\Lib\concurrent\futures\_base.py", line 619, in result_iterator yield _result_or_cancel(fs.pop()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\anaconda3\Lib\concurrent\futures\_base.py", line 317, in _result_or_cancel return fut.result(timeout) ^^^^^^^^^^^^^^^^^^^ File "D:\anaconda3\Lib\concurrent\futures\_base.py", line 456, in result return self.__get_result() ^^^^^^^^^^^^^^^^^^^ File "D:\anaconda3\Lib\concurrent\futures\_base.py", line 401, in __get_result raise self._exception File "D:\anaconda3\Lib\concurrent\futures\thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\l60080208\AppData\Roaming\Python\Python312\site-packages\huggingface_hub\_snapshot_download.py", line 301, in _inner_hf_hub_download return hf_hub_download( ^^^^^^^^^^^^^^^^ File "C:\Users\l60080208\AppData\Roaming\Python\Python312\site-packages\huggingface_hub\utils\_validators.py", line 114, in _inner_fn return fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "C:\Users\l60080208\AppData\Roaming\Python\Python312\site-packages\huggingface_hub\file_download.py", line 988, in hf_hub_download return _hf_hub_download_to_local_dir( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\l60080208\AppData\Roaming\Python\Python312\site-packages\huggingface_hub\file_download.py", line 1290, in _hf_hub_download_to_local_dir _download_to_tmp_and_move( File "C:\Users\l60080208\AppData\Roaming\Python\Python312\site-packages\huggingface_hub\file_download.py", line 1725, in _download_to_tmp_and_move http_get( File "C:\Users\l60080208\AppData\Roaming\Python\Python312\site-packages\huggingface_hub\file_download.py", line 420, in http_get r = _request_wrapper( ^^^^^^^^^^^^^^^^^ File "C:\Users\l60080208\AppData\Roaming\Python\Python312\site-packages\huggingface_hub\file_download.py", line 310, in _request_wrapper hf_raise_for_status(response) File "C:\Users\l60080208\AppData\Roaming\Python\Python312\site-packages\huggingface_hub\utils\_http.py", line 478, in hf_raise_for_status raise _format(HfHubHTTPError, message, response) from e huggingface_hub.errors.HfHubHTTPError: 416 Client Error: Requested Range Not Satisfiable for url: https://cdn-lfs-us-1.hf.co/repos/ed/91/ed91c2f4c4a7e464a63e321b1e4a078dfcc8b60bdede6e1c27f731ec9ac46cac/ffb5e932cab2241009ce80b808cb697f26f20409b63c7aa3ba5d1b4213f11fbb?response-content-disposition=inline%3B+filename*%3DUTF-8%27%27data-00029-of-00080.arrow%3B+filename%3D%22data-00029-of-00080.arrow%22%3B&Expires=1751970264&Policy=eyJTdGF0ZW1lbnQiOlt7IkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTc1MTk3MDI2NH19LCJSZXNvdXJjZSI6Imh0dHBzOi8vY2RuLWxmcy11cy0xLmhmLmNvL3JlcG9zL2VkLzkxL2VkOTFjMmY0YzRhN2U0NjRhNjNlMzIxYjFlNGEwNzhkZmNjOGI2MGJkZWRlNmUxYzI3ZjczMWVjOWFjNDZjYWMvZmZiNWU5MzJjYWIyMjQxMDA5Y2U4MGI4MDhjYjY5N2YyNmYyMDQwOWI2M2M3YWEzYmE1ZDFiNDIxM2YxMWZiYj9yZXNwb25zZS1jb250ZW50LWRpc3Bvc2l0aW9uPSoifV19&Signature=j3OTpfO4fi500NaPj3rqp-Q7kJ6y6qGT8CrRKnCimUPWXn~zkG4HbtMEsaqbjQT9Ema16pXtKKK3VKpy2VUDZ5aAkMDVlNMQsq9yThxpzXYYuwASUfPek8rONQVqf3KsF2Q22bBPdKJXugvZBC2xzPKmvSdKkP7Ry0QY02F5GP-UD0xAG-wo8vb8I~iyrcX2IFgPGa~wWAKfVnRqHOpFCKtUi6yHAup4R7TvE1xvX5DPtzesDd7ogswJQ66v0Jq8UgwUmTNbZrWBlQG~xtPvEqJB4fXkgcZ7uEEqJHd~X4uT~JtUXW4MZUMs0TEh71xUbC3cc1E4F6Hfo8kyZpbo7A__&Key-Pair-Id=K24J24Z295AEI9. Requested range: bytes=12861552-. Content-Range: bytes */12861552. 416 Client Error: Requested Range Not Satisfiable for url:
07-09
1. 客户端请求的字节范围 `Range: bytes=500-1000` 2. 服务器文件小于 500 字节或范围无效 3. 常见于： - 文件更新但缓存未同步 - 网络中断导致部分下载 - 服务器配置禁止范围请求 > 通过强制完整下载 + 缓存...
simpy构建Cache仿真系统
最新发布
07-26
env.run(until=1_000_000) # 运行100万时钟周期 # 输出结果 print(f"命中率: {monitor.hit_rate:.2%}") print(f"平均访问时间: {sum(monitor.access_times)/len(monitor.access_times):.1f} 周期") ``` #### ...
my cloud test bed (by quqi99)
技术并艺术着
03-10 2005
若容器里如果上不了网，如无法访问api.snapcraft.io，是因为lxd容易默认使用了eth1上的dns=10.10.10.1，下面的配置可让eth0, eth1, eth2都默认使用dns=192.168.99.1来避免特色网络对api.snapcraft.io的污染。下列netplan配置创建了br-eth0，也让br-eth0支持wol通过魔术包唤醒，也创建了一个没有dhcp的br-maas用于maas实验。
redis源码分析之server.h
小羽毛的博客
05-21 1061
#ifndef __REDIS_H #define __REDIS_H #include "fmacros.h" #include "config.h" #include "solarisfixes.h" #include "rio.h" #include <stdio.h> #include <stdlib.h> #include <string.h> #...
我在京东写的traffic server笔记
08-27
traffic server一般作为cdn proxy使用，文档中包含了大量的traffic server内容，包括源码分析，以及一些插件开发；
Traffic Server
03-03
The open source can improve the access speed when do the HTTP access
traffic server cacheurl
01-09
一些网络上的ats配置资料
HTTP范围请求（Range Requests）在Python视频流中的应用
weixin_41158143的博客
07-16 332
在Web开发中，处理大文件如视频文件时，HTTP范围请求（Range Requests）成为了一个至关重要的功能。范围请求允许客户端请求文件的特定部分，而不是整个文件，这对于视频流服务尤为重要，因为它可以显著减少数据传输量，提高用户体验。在Python中，利用HTTP服务器框架如Flask或更底层的库如http.serve...
ATS标准插件( 4 )　Cache Key Manipulation Plugin
superbfly的专栏
07-05 801
原文链接：   https://www.9mblog.com/132.html这个插件十分有用，可以对http请求中的一些值进行过滤，从而控制对于http请求的缓存，如：l 对请求参数进行排序，以防止由于请求参数重新排序引起的缓存未命中。l 通过名称或正则表达式的设置使得某些请求不被缓存（黑名单）。l 忽略全部请求，即让所有资源都不缓存。l 通过名称或正则表达式的设置使得某些请求被缓存（白名单）。...
Apache Traffic Server处理请求的过程
weixin_34138139的博客
11-25 280
1）用户请求一个web对象，TS收到请求2）TS通过对象的地址，在对象数据库（缓存）中去定位该对象a.如果对象在缓存中，TS会检查对象是否新鲜（fresh）如果新鲜，TS从缓存里返回该对象给用户，此时称为缓存命中（cachehit）如果不新鲜（stale），TS会连接源服务器去验证对象是否仍然...
Apache Traffic Server 简介
larryliuqing的专栏-xxx
03-15 2696
http://blog.sina.com.cn/s/blog_502c8cc40100mw7n.html                         作者：王柯龙 一 介绍 Apache Traffic Server（ATS或TS）是一个高性能的、模块化的 HTTP 代理和缓存服务器。Traffic Server 最初是 Inktomi 公司的商业产品，该公司在 20
Trafficserver实现代理和cdn缓存
liaomin416100569的专栏
02-28 6164
一。Trafficserver简介  Apache Traffic Server™ 提供高性能可伸缩的使用http/https通信的正向和反向代理解决方案，同时支持多种代理方案同时配置  关于代理中，正向代理，反向代理，透明代理的概念参考（http://blog.youkuaiyun.com/liaomin416100569/article/details/77774073）1.简易安装 （使用yum） 安装...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值