本文翻译自:Why is [] faster than list()?
I recently compared the processing speeds of [] and list() and was surprised to discover that [] runs more than three times faster than list() . 我最近比较了[]和list()的处理速度,并惊讶地发现[]运行速度比list() 快三倍。 I ran the same test with {} and dict() and the results were practically identical: [] and {} both took around 0.128sec / million cycles, while list() and dict() took roughly 0.428sec / million cycles each. 我使用{}和dict()进行了相同的测试,结果实际上是相同的: []和{}都花费了约0.128sec /百万个周期,而list()和dict()花费了约0.428sec /百万个周期。
Why is this? 为什么是这样? Do [] and {} (and probably () and '' , too) immediately pass back a copies of some empty stock literal while their explicitly-named counterparts ( list() , dict() , tuple() , str() ) fully go about creating an object, whether or not they actually have elements? Do []和{} (可能还有()和'' )也会立即传回一些空股票文字的副本,同时使用它们的显式名称副本( list() , dict() , tuple() , str() )完全去创建一个对象,无论它们实际上是否包含元素?
I have no idea how these two methods differ but I'd love to find out. 我不知道这两种方法有何不同,但我很想找出答案。 I couldn't find an answer in the docs or on SO, and searching for empty brackets turned out to be more problematic than I'd expected. 我在文档中或SO上都找不到答案,而寻找空括号却比我预期的要麻烦得多。
I got my timing results by calling timeit.timeit("[]") and timeit.timeit("list()") , and timeit.timeit("{}") and timeit.timeit("dict()") , to compare lists and dictionaries, respectively. 我通过调用timeit.timeit("[]")和timeit.timeit("list()")以及timeit.timeit("{}")和timeit.timeit("dict()")获得计时结果。分别比较列表和字典。 I'm running Python 2.7.9. 我正在运行Python 2.7.9。
I recently discovered " Why is if True slower than if 1? " that compares the performance of if True to if 1 and seems to touch on a similar literal-versus-global scenario; 我最近发现“ 为什么True慢于if 1? ”将if True与if 1的性能进行了比较,似乎触及了类似的文字对全局的情况。 perhaps it's worth considering as well. 也许也值得考虑。
#1楼
参考:https://stackoom.com/question/22mYq/为什么-比list-快
#2楼
Because list is a function to convert say a string to a list object, while [] is used to create a list off the bat. 由于list是一个功能转化说一个字符串列表对象,而[]用于创建一个列表蝙蝠。 Try this (might make more sense to you): 尝试一下(可能对您更有意义):
x = "wham bam"
a = list(x)
>>> a
["w", "h", "a", "m", ...]
While 而
y = ["wham bam"]
>>> y
["wham bam"]
Gives you a actual list containing whatever you put in it. 为您提供包含您所输入内容的实际列表。
#3楼
Because [] and {} are literal syntax . 因为[]和{}是文字语法 。 Python can create bytecode just to create the list or dictionary objects: Python可以创建字节码仅用于创建列表或字典对象:
>>> import dis
>>> dis.dis(compile('[]', '', 'eval'))
1 0 BUILD_LIST 0
3 RETURN_VALUE
>>> dis.dis(compile('{}', '', 'eval'))
1 0 BUILD_MAP 0
3 RETURN_VALUE
list() and dict() are separate objects. list()和dict()是单独的对象。 Their names need to be resolved, the stack has to be involved to push the arguments, the frame has to be stored to retrieve later, and a call has to be made. 它们的名称需要解析,必须包含堆栈以推入参数,必须存储框架以供以后检索,并且必须进行调用。 That all takes more time. 这都需要更多时间。
For the empty case, that means you have at the very least a LOAD_NAME (which has to search through the global namespace as well as the __builtin__ module ) followed by a CALL_FUNCTION , which has to preserve the current frame: 对于空的情况,这意味着您至少有一个LOAD_NAME (必须搜索全局名称空间以及__builtin__模块 ),然后是CALL_FUNCTION ,后者必须保留当前帧:
>>> dis.dis(compile('list()', '', 'eval'))
1 0 LOAD_NAME 0 (list)
3 CALL_FUNCTION 0
6 RETURN_VALUE
>>> dis.dis(compile('dict()', '', 'eval'))
1 0 LOAD_NAME 0 (dict)
3 CALL_FUNCTION 0
6 RETURN_VALUE
You can time the name lookup separately with timeit : 您可以使用timeit分别对名称查找进行计时:
>>> import timeit
>>> timeit.timeit('list', number=10**7)
0.30749011039733887
>>> timeit.timeit('dict', number=10**7)
0.4215109348297119
The time discrepancy there is probably a dictionary hash collision. 时间差异可能是字典哈希冲突。 Subtract those times from the times for calling those objects, and compare the result against the times for using literals: 从调用这些对象的时间中减去这些时间,然后将结果与使用文字的时间进行比较:
>>> timeit.timeit('[]', number=10**7)
0.30478692054748535
>>> timeit.timeit('{}', number=10**7)
0.31482696533203125
>>> timeit.timeit('list()', number=10**7)
0.9991960525512695
>>> timeit.timeit('dict()', number=10**7)
1.0200958251953125
So having to call the object takes an additional 1.00 - 0.31 - 0.30 == 0.39 seconds per 10 million calls. 因此,每1千万次调用必须调用对象额外花费1.00 - 0.31 - 0.30 == 0.39秒。
You can avoid the global lookup cost by aliasing the global names as locals (using a timeit setup, everything you bind to a name is a local): 您可以通过将全局名称别名为本地名称来避免全局查找成本(使用timeit设置,您绑定到名称的所有内容都是本地名称):
>>> timeit.timeit('_list', '_list = list', number=10**7)
0.1866450309753418
>>> timeit.timeit('_dict', '_dict = dict', number=10**7)
0.19016098976135254
>>> timeit.timeit('_list()', '_list = list', number=10**7)
0.841480016708374
>>> timeit.timeit('_dict()', '_dict = dict', number=10**7)
0.7233691215515137
but you never can overcome that CALL_FUNCTION cost. 但您永远无法克服CALL_FUNCTION费用。
#4楼
list() requires a global lookup and a function call but [] compiles to a single instruction. list()需要全局查找和函数调用,但是[]编译为一条指令。 See: 看到:
Python 2.7.3
>>> import dis
>>> print dis.dis(lambda: list())
1 0 LOAD_GLOBAL 0 (list)
3 CALL_FUNCTION 0
6 RETURN_VALUE
None
>>> print dis.dis(lambda: [])
1 0 BUILD_LIST 0
3 RETURN_VALUE
None
#5楼
The answers here are great, to the point and fully cover this question. 至此,答案非常好,并完全涵盖了这个问题。 I'll drop a further step down from byte-code for those interested. 对于那些感兴趣的人,我将进一步从字节码中删除。 I'm using the most recent repo of CPython; 我正在使用CPython的最新仓库; older versions behave similar in this regard but slight changes might be in place. 在这方面,旧版本的行为类似,但可能会稍作更改。
Here's a break down of the execution for each of these, BUILD_LIST for [] and CALL_FUNCTION for list() . 这是每个命令执行的BUILD_LIST , [] CALL_FUNCTION [] CALL_FUNCTION和list() CALL_FUNCTION 。
The BUILD_LIST instruction: BUILD_LIST指令:
You should just view the horror: 您应该只查看恐怖:
PyObject *list = PyList_New(oparg);
if (list == NULL)
goto error;
while (--oparg >= 0) {
PyObject *item = POP();
PyList_SET_ITEM(list, oparg, item);
}
PUSH(list);
DISPATCH();
Terribly convoluted, I know. 我知道那令人费解。 This is how simple it is: 这是多么简单:
- Create a new list with
PyList_New(this mainly allocates the memory for a new list object),opargsignalling the number of arguments on the stack. 使用PyList_New创建一个新列表(这主要为新的列表对象分配内存),oparg指示堆栈上的参数数量。 Straight to the point. 开门见山。 - Check that nothing went wrong with
if (list==NULL). 检查if (list==NULL)。 - Add any arguments (in our case this isn't executed) located on the stack with
PyList_SET_ITEM(a macro). 使用PyList_SET_ITEM(一个宏)添加位于堆栈上的所有参数(在我们的示例中不执行)。
No wonder it is fast! 难怪它很快! It's custom-made for creating new lists, nothing else :-) 它是为创建新列表而定制的,仅此而已:-)
The CALL_FUNCTION instruction: CALL_FUNCTION指令:
Here's the first thing you see when you peek at the code handling CALL_FUNCTION : 窥视代码处理CALL_FUNCTION时,这是您看到的第一件事:
PyObject **sp, *res;
sp = stack_pointer;
res = call_function(&sp, oparg, NULL);
stack_pointer = sp;
PUSH(res);
if (res == NULL) {
goto error;
}
DISPATCH();
Looks pretty harmless, right? 看起来很无害吧? Well, no, unfortunately not, call_function is not a straightforward guy that will call the function immediately, it can't. 好吧,不,不幸的是, call_function不是一个直截了当的家伙,它不会立即调用该函数。 Instead, it grabs the object from the stack, grabs all arguments of the stack and then switches based on the type of the object; 相反,它从堆栈中获取对象,获取堆栈中的所有参数,然后根据对象的类型进行切换。 is it a: 它是:
-
PyCFunction_Type?PyCFunction_Type? Nope, it islist,listisn't of typePyCFunction不,它是list,list不是PyCFunction类型的 -
PyMethodType?PyMethodType? Nope, see previous. 不,请参见前面。 -
PyFunctionType?PyFunctionType? Nopee, see previous. 不,请参见前面。
We're calling the list type, the argument passed in to call_function is PyList_Type . 我们正在调用list类型,传递给call_function的参数是PyList_Type 。 CPython now has to call a generic function to handle any callable objects named _PyObject_FastCallKeywords , yay more function calls. CPython现在必须调用一个泛型函数来处理名为_PyObject_FastCallKeywords所有可调用对象,还需要更多函数调用。
This function again makes some checks for certain function types (which I cannot understand why) and then, after creating a dict for kwargs if required , goes on to call _PyObject_FastCallDict . 该函数再次检查某些函数类型(我不明白为什么),然后在为kwargs创建字典之后, 如果需要 ,继续调用_PyObject_FastCallDict 。
_PyObject_FastCallDict finally gets us somewhere! _PyObject_FastCallDict终于使我们到了某个地方! After performing even more checks it grabs the tp_call slot from the type of the type we've passed in, that is, it grabs type.tp_call . 在执行更多检查之后,它从我们传入的type 的type中抢占了tp_call插槽 ,即,它type.tp_call 。 It then proceeds to create a tuple out of of the arguments passed in with _PyStack_AsTuple and, finally, a call can finally be made ! 然后, _PyStack_AsTuple根据_PyStack_AsTuple传入的参数来创建元组,最后可以最终进行调用 !
tp_call , which matches type.__call__ takes over and finally creates the list object. tp_call ,它与type.__call__匹配type.__call__接管并最终创建列表对象。 It calls the lists __new__ which corresponds to PyType_GenericNew and allocates memory for it with PyType_GenericAlloc : This is actually the part where it catches up with PyList_New , finally . 它调用对应于PyType_GenericNew的列表__new__ ,并使用PyType_GenericAlloc分配内存: 实际上,这实际上是它最终赶上PyList_New的部分 。 All the previous are necessary to handle objects in a generic fashion. 所有以前的内容对于以通用方式处理对象都是必需的。
In the end, type_call calls list.__init__ and initializes the list with any available arguments, then we go on a returning back the way we came. 最后, type_call调用list.__init__并使用任何可用参数初始化该列表,然后继续返回原来的方式。 :-) :-)
Finally, remmeber the LOAD_NAME , that's another guy that contributes here. 最后,记住LOAD_NAME ,这是另一个在这里做出贡献的家伙。
It's easy to see that, when dealing with our input, Python generally has to jump through hoops in order to actually find out the appropriate C function to do the job. 很容易看到,在处理我们的输入时,Python通常必须跳过圈,才能真正找到合适的C函数来完成这项工作。 It doesn't have the curtesy of immediately calling it because it's dynamic, someone might mask list ( and boy do many people do ) and another path must be taken. 它不具有立即调用它的功能,因为它是动态的,有人可能会屏蔽list ( 男孩会做很多人做的事情 ),因此必须采取另一条路径。
This is where list() loses much: The exploring Python needs to do to find out what the heck it should do. 这是list()损失惨重的地方:不断探索的Python需要做的事情才能弄清楚它应该做什么。
Literal syntax, on the other hand, means exactly one thing; 另一方面,字面语法恰好意味着一回事。 it cannot be changed and always behaves in a pre-determined way. 它无法更改,并且始终以预定的方式运行。
Footnote: All function names are subject to change from one release to the other. 脚注:所有功能名称均可能从一个版本更改为另一个版本。 The point still stands and most likely will stand in any future versions, it's the dynamic look-up that slows things down. 关键点仍然存在,并且很可能在将来的任何版本中都存在,这是动态查找使事情变慢的原因。
#6楼
Why is
[]faster thanlist()? 为什么[]比list()快?
The biggest reason is that Python treats list() just like a user-defined function, which means you can intercept it by aliasing something else to list and do something different (like use your own subclassed list or perhaps a deque). 最大的原因是Python像对待用户定义的函数一样对待list() ,这意味着您可以通过别名其他对象来list并执行其他操作(例如使用您自己的子类列表或双端队列)来拦截它。
It immediately creates a new instance of a builtin list with [] . 它立即使用[]创建内置列表的新实例。
My explanation seeks to give you the intuition for this. 我的解释旨在为您提供直觉。
Explanation 说明
[] is commonly known as literal syntax. []通常称为文字语法。
In the grammar, this is referred to as a "list display". 在语法中,这称为“列表显示”。 From the docs : 从文档 :
A list display is a possibly empty series of expressions enclosed in square brackets: 列表显示是括在方括号中的一系列可能为空的表达式:
list_display ::= "[" [starred_list | comprehension] "]"A list display yields a new list object, the contents being specified by either a list of expressions or a comprehension. 列表显示将产生一个新的列表对象,其内容由表达式列表或理解列表指定。 When a comma-separated list of expressions is supplied, its elements are evaluated from left to right and placed into the list object in that order. 提供逗号分隔的表达式列表时,将按从左到右的顺序评估其元素,并将其按顺序放入列表对象中。 When a comprehension is supplied, the list is constructed from the elements resulting from the comprehension. 提供理解后,将根据理解产生的元素来构建列表。
In short, this means that a builtin object of type list is created. 简而言之,这意味着将创建类型为list的内置对象。
There is no circumventing this - which means Python can do it as quickly as it may. 不能回避这一点-这意味着Python可以尽快完成它。
On the other hand, list() can be intercepted from creating a builtin list using the builtin list constructor. 在另一方面, list()可以从创建一个内置的拦截list使用内置列表构造。
For example, say we want our lists to be created noisily: 例如,假设我们希望创建噪音较大的列表:
class List(list):
def __init__(self, iterable=None):
if iterable is None:
super().__init__()
else:
super().__init__(iterable)
print('List initialized.')
We could then intercept the name list on the module level global scope, and then when we create a list , we actually create our subtyped list: 然后,我们可以在模块级别的全局范围内拦截名称list ,然后在创建list ,实际上创建了子类型列表:
>>> list = List
>>> a_list = list()
List initialized.
>>> type(a_list)
<class '__main__.List'>
Similarly we could remove it from the global namespace 同样,我们可以将其从全局名称空间中删除
del list
and put it in the builtin namespace: 并将其放在内置名称空间中:
import builtins
builtins.list = List
And now: 现在:
>>> list_0 = list()
List initialized.
>>> type(list_0)
<class '__main__.List'>
And note that the list display creates a list unconditionally: 并注意列表显示无条件创建列表:
>>> list_1 = []
>>> type(list_1)
<class 'list'>
We probably only do this temporarily, so lets undo our changes - first remove the new List object from the builtins: 我们可能只是暂时执行此操作,所以请撤消更改-首先从内置文件中删除新的List对象:
>>> del builtins.list
>>> builtins.list
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: module 'builtins' has no attribute 'list'
>>> list()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'list' is not defined
Oh, no, we lost track of the original. 哦,不,我们失去了原来的踪迹。
Not to worry, we can still get list - it's the type of a list literal: 不用担心,我们仍然可以获得list -这是列表文字的类型:
>>> builtins.list = type([])
>>> list()
[]
So... 所以...
Why is
[]faster thanlist()? 为什么[]比list()快?
As we've seen - we can overwrite list - but we can't intercept the creation of the literal type. 如我们所见-我们可以覆盖list -但我们无法拦截文字类型的创建。 When we use list we have to do the lookups to see if anything is there. 当使用list我们必须进行查找以查看是否存在任何内容。
Then we have to call whatever callable we have looked up. 然后,我们必须调用已查找的任何可调用对象。 From the grammar: 从语法上:
A call calls a callable object (eg, a function) with a possibly empty series of arguments: 调用使用一系列可能为空的参数来调用可调用对象(例如,函数):
call ::= primary "(" [argument_list [","] | comprehension] ")"
We can see that it does the same thing for any name, not just list: 我们可以看到它对任何名称都具有相同的作用,而不仅仅是列表:
>>> import dis
>>> dis.dis('list()')
1 0 LOAD_NAME 0 (list)
2 CALL_FUNCTION 0
4 RETURN_VALUE
>>> dis.dis('doesnotexist()')
1 0 LOAD_NAME 0 (doesnotexist)
2 CALL_FUNCTION 0
4 RETURN_VALUE
For [] there is no function call at the Python bytecode level: 对于[] ,在Python字节码级别没有函数调用:
>>> dis.dis('[]')
1 0 BUILD_LIST 0
2 RETURN_VALUE
It simply goes straight to building the list without any lookups or calls at the bytecode level. 它只是直接建立列表而无需在字节码级别进行任何查找或调用。
Conclusion 结论
We have demonstrated that list can be intercepted with user code using the scoping rules, and that list() looks for a callable and then calls it. 我们已经证明,可以使用范围规则使用用户代码拦截list ,并且list()查找可调用对象,然后调用它。
Whereas [] is a list display, or a literal, and thus avoids the name lookup and function call. 而[]是列表显示或文字,因此避免了名称查找和函数调用。
探讨了Python中[]与list()在创建列表时的速度差异,解析了字面量与函数调用在底层实现上的区别,揭示了为何[]更快。
快?&spm=1001.2101.3001.5002&articleId=105510176&d=1&t=3&u=73381b19e46f47578cee75d041eabe87)
1953

被折叠的 条评论
为什么被折叠?



