RuntimeError: Already borrowed

最新推荐文章于 2023-04-02 17:07:34 发布
原创 最新推荐文章于 2023-04-02 17:07:34 发布 · 2.4k 阅读 · 1 ·
CC 4.0 BY-SA版权
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
文章标签：

#自然语言处理 #深度学习

本文记录了一个关于SBERT模型接口部署后出现的RuntimeError:Alreadyborrowed错误。该错误发生于多线程环境下使用同一tokenizer时，解决方案包括采用TFRecord进行数据转换并通过文件流读取，最终采取了对批量数据进行切分的方法。

问题：

之前写的SBERT模型接口部署上线后最近报了RuntimeError: Already borrowed的错误，在这里记录下。

现象：

具体的报错如下：

  File "/home/XXXX/XXX/src/sentence_proc.py", line 77, in compute_sentence_vectors
    sentences = self.tokenizer(sentences, padding='max_length', truncation=True, max_length=self.max_seq_length,return_tensors="tf")
  File "/home/XXX/.local/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 2249, in __call__
    return self.batch_encode_plus(
  File "/home/XXXX/.local/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 2434, in batch_encode_plus
    return self._batch_encode_plus(
  File "/home/XXXX/.local/lib/python3.8/site-packages/transformers/tokenization_utils_fast.py", line 370, in _batch_encode_plus
    self.set_truncation_and_padding(
  File "/home/XXXX/.local/lib/python3.8/site-packages/transformers/tokenization_utils

最低0.47元/天 解锁文章
Spring Boot入门教程(四):配置文件
人不风流枉少年
03-20 4943
配置方式 每个starter都有自己默认的配置，如果需要改变默认值，可以在其他地方配置来覆盖掉默认的值，覆盖默认的配置有多种方式，每种方式的优先级也不同，如果在多个地方配置则优先使用优先级高的值，其中命令行参数优先级最高, 其中大部分参数一般都配置在属性文件application.properties中，属性文件即可以覆盖starter中默认的值，也可以自定义值 在命令行行输入的参数 SPR...
Springboot - 项目的全部可配置属性及其说明
简简单单Onlinezuozuo
04-11 9487
Springboot - 项目的全部可配置属性及其说明 1.可配置的项目如下 // 包含了可配置的字段， 默认值，以及说明 // 有机会后面翻译一下 debug=false # Enable debug logs. trace=false # Enable trace logs. # LOGGING logging.config= # Location of the logging c...
python的多进程笔记
angryps的博客
04-02 414
在处理数据集时，常常会遇到用for循环处理数据集的情况。但是如果简单地用for循环就会出现cpu资源利用不充分的情况。
预训练模型语义相似性计算(一)--SBERT
choose_c的博客
02-22 3140
将预训练BERT迁移到文本语义相似性思路： 1.交互编码(cross-encoder)。文本拼接后进行微调。 缺点非常耗时； 2.向量空间模型 进行无监督训练，使用BERT模型的输出的cls向量，最后一层或几层的hidden state(avg)，一般后者效果好。 效果不如glove 3.SBERT 使用孪生网络，加快推理。 SBERT： 时间消耗：每个句子生成句向量+cosine的计算； 预训练模型的选择：BERT...
python报错：RuntimeError
qq_43183663的博客
12-18 8268
python报错：RuntimeError:fails to pass a sanity check due to a bug in the windows runtime这种类型的错误 这种错误原因： 1.当前的python与numpy版本之间有什么问题，比如我自己用的python3.9与numpy1.19.4会导致这种报错。 2.numpy1.19.4与当前很多python版本都有问题。 解决办法： 在File->Settings->Project:pycharmProjects->P
Thanks to your work so far, the library now has a functioning system. However, librarians have reported that some records in the CSV files are corrupted with invalid values. This has caused misbehaviour in the system, sometimes even crashing during busy exam periods, creating unnecessary stress for students. In real-world software engineering, robustness and correctness are just as important as functionality. Programs must gracefully handle unexpected situations and invalid inputs without failing. Two key approaches developers use are: Assertions: Internal checks that validate assumptions made by the programmer (e.g., "This variable should never be negative"). Assertions are mainly used during development and debugging. Defensive programming: Validates inputs and usage patterns to prevent invalid or unexpected behaviour at runtime. This includes checking for correct types, value ranges, and providing helpful error messages when something goes wrong. Description In this task, you will extend your library system by adding custom error handling to make it more robust. Python already raises built-in errors, but here you must define and use custom exceptions (in custom_errors.py) to handle library-specific error conditions. Building upon Task 3, update your user.py and book.py modules to handle invalid input, and modify task4.py to handle exceptions gracefully so that the system does not crash. You must ensure that this system is: Correct: Through use of assert to validate assumptions in your logic. Safe: Through defensive programming techniques that prevent invalid inputs (e.g. wrong types, negative amounts). In this task we will be using custom errors as python will already raise some of these errors when we execute invalid code. Our new errors are located in cusom_errors.py. You must use these custom errors, please see rubric for related penalties in this task. After auditing the CSV data, the librarian has introduced new validation rules to be applied in addition to the requirements described in Tasks 1–3. These rules strengthen data quality and prevent corrupted records from entering the system. Always include a helpful and descriptive error message when raising errors. Doing so contributes to the readability and overall quality of your assignment. Users Every user must have a password. Passwords cannot be empty. Every word in a user's name must contain only alphabetic characters. Numbers, symbols, and special characters are not allowed. Department is mandatory for students and staff, but must remain empty for other users. The only valid departments are: IT, Business, Arts, Science, Engineering, Education, Medicine and Library. Books The library only recognizes two valid book types: physical and online. The following fields are required for all books: Title, Author, Year Each book may have at most 5 keywords. Each word in keywords must consist of letters and numbers only. The only allowed special character is the hyphen (-). Loans Every loan record must be associated with an existing user and an existing book. The due date is a required field and cannot be left empty. Defensive Programming The following exceptions are provided to you in custom_errors.py: CustomTypeError: A value is the wrong type (e.g. a string instead of a float or int). CustomValueError: A value is of the correct type but invalid (e.g. negative copies). CustomDateError: Raised when a date is in the wrong format or when the date logic is invalid (e.g., borrow date occurs in the future). MissingRequiredFieldError: Raised when a required field is missing. UserNotFoundError or BookNotFoundError: Raised when a loan references a user or book that does not exist. CustomOperationError : Raised when an invalid borrow or return action is attempted. CustomLimitError: Raised when a user exceeds their quota or borrowing limit. CustomKeyError: Occurs when you try to access a key in the dictionary that doesn't exist.
10-18
self.password = password # Already checked non-empty above self.borrowed_books = [] # Assert internal consistency assert isinstance(self.user_id, str) and len(self.user_id) > 0, "User ID failed ...
from xmlrpc.client import Boolean class Borrower: """Represents a person that can borrow books""" newIdCode = 1 # Class variable def __init__(self, firstname, lastname): self.firstname = firstname self.lastname = lastname self.id = Borrower.newIdCode Borrower.newIdCode += 1 # Updates for next new borrower ... self.booksBorrowed = [] def borrowBook(self, book): self.booksBorrowed.append(book) if Book.__init__.OnLoan: print("该书已经被借走了") def showAllBooks(self): for i in range(len(self.booksBorrowed)): print(self.firstname,self.lastname,"已经借走了",booksBorrowed[i]) def showBorrowerDetails(self): print(self.firstname, self.lastname, self.id) class Book: """A class to represent a book in a library""" def __init__(self, title, author, code): self.title = title self.author = author self.code = code OnLoan = False def showTitle(self): print(self.title) class Library: """A class to represent a lending library""" def __init__(self): self.allBorrowers = [] self.allBooks =[] def addBook(self, book): self.allBooks.append(book) def addBorrower(self, borrower): self.allBorrowers.append(borrower) def lendBook(self,borrower,book): if borrower in self.allBorrowers: print(borrower.firstname, borrower.lastname,end=" ") self.allBorrowers.remove(borrower) if book in self.allBooks: print("借走了",book.title) self.allBooks.remove(book) Book.__init__.OnLoan = True def main(): ## Create some books ... book1 = Book('Kafkas motorbike','Bridget Jones', 1290) book2 = Book('Cooking with Custard','Jello Biafra', 1002) book3 = Book('History of Bagpipes','John Cairncross', 987) # Put the books in the library library = Library() # Creates the library library.addBook(book1) library.addBook(book2) library.addBook(book3) # Create some borrowers ... bor1 = Borrower('Kevin','Wilson') bor2 = Borrower('Rita','Shapiro') bor3 = Borrower('Max','Normal') library.addBorrower(bor1) library.addBorrower(bor2) library.addBorrower(bor3) # make some loans ... library.lendBook(bor1, book1) bor1.showBorrowerDetails() bor1.showAllBooks() library.lendBook(bor2,book3) bor2.showBorrowerDetails() bor2.showAllBooks() # Try to lend book3 out again .... library.lendBook(bor3,book3) if __name__ == "__main__": main()
最新发布
10-28
raise RuntimeError(f"书籍已被{book.borrower}借阅") book.borrow(borrower) # 委托给Book类的方法 ``` --- #### 问题2：状态管理缺失 ```python # 问题代码 (引用[1]) library[isbn]['is_borrowed'] = True ...
Spring Boot学习总结（8）——SpringBoot Common application properties（application.properties）详解
科技D人生
05-05 4154
各种属性可以在您的application.properties/application.yml文件或命令行开关指定。下面提供了常见的Spring启动属性和引用它们的基础类的列表。 # =================================================================== # COMMON SPRING BOOT PROPERTIES # # T
python出现RuntimeError错误
热门推荐
weixin_42099082的博客
04-17 4万+
RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase. This probably means that you are not using for...
Python 在创建多进程时抛出RuntimeError错误
海纳百川
05-04 2万+
一、错误信息 RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase. This probably means that you are not using fork to start your chil...
pytorch以及transformers各种问题总结中
xhsun的博客
10-06 6378
RuntimeError: cuda runtime error (38) : no CUDA-capable device is detected at /pytorch/aten/src/THC/THCGeneral.cpp:50 将os.environ[‘CUDA_VISIBLE_DEVICES’]赋值为’0’ os.environ['CUDA_VISIBLE_DEVICES']='0' RuntimeError: Expected object of device type cuda but go
runtime error python_python出现RuntimeError错误
weixin_39602280的博客
11-28 1996
RuntimeError:An attempt has been made to start a new process before thecurrent process has finished its bootstrapping phase.This probably means that you are not using fork to start yourchild processes...
Python_RuntimeError问题的解决
qq_40313024的博客
04-19 1万+
implement_array_function method already has a docstring 本人开发使用的是apache2.4，python3.6和django，在配置Apache的时候，内外网同时访问就会报错， 只能有一方访问， Traceback (most recent call last):\r, referer: http://***/***.html File ...
Tokenizer总结
choose_c的博客
08-02 1万+
Introduciton transformer类型的预训练模型层出不穷，其中的tokenizer方法作为一个非常重要的模块也出现了一些方法。本文对tokenizer方法做一些总结。参考来自hunggingface。 tokenizer在中文中叫做分词器，就是将句子分成一个个小的词块(token),生成一个词表，并通过模型学习到更好的表示。其中词表的大小和token的长短是很关键的因素，两者需要进行权衡，token太长，则它的表示也能更容易学习到，相应的词表也会变小；to...
RuntimeError: Address already in use
07-25
这个错误通常是由于端口冲突导致的。当某个程序或服务正在使用特定的端口时，另一个程序或服务尝试使用相同的端口就会导致此错误。 要解决这个问题，你可以尝试以下几种方法： 1. 更改端口号：尝试将你的程序或服务配置文件中使用的端口号更改为一个未被占用的端口号。 2. 结束正在使用该端口的进程：你可以使用命令行工具（如Windows下的`Task Manager`或Linux下的`lsof`命令）查找正在使用该端口的进程，并结束它。 3. 重新启动设备：有时候，端口冲突可能是由于之前未正确关闭的进程导致的。重新启动设备可以清除这些进程，从而解决冲突。 希望这些方法能帮助你解决问题。如果问题仍然存在，请提供更多详细信息以便我能够给出更具体的建议。
评论 2
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
红包 添加红包
表情包 插入表情
表情包 代码片
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值