python怎么对单词分词,Python：查找所有可能的带有字符序列的单词组合（分词）...

I'm doing some word segmentation experiments like the followings.

lst is a sequence of characters, and output is all the possible words.

lst = ['a', 'b', 'c', 'd']

def foo(lst):

...

return output

output = [['a', 'b', 'c', 'd'],

['ab', 'c', 'd'],

['a', 'bc', 'd'],

['a', 'b', 'cd'],

['ab', 'cd'],

['abc', 'd'],

['a', 'bcd'],

['abcd']]

I've checked combinations and permutations in itertools library,

and also tried combinatorics.

However, it seems that I'm looking at the wrong side because this is not pure permutation and combinations...

It seems that I can achieve this by using lots of loops, but the efficiency might be low.

EDIT

The word order is important so combinations like ['ba', 'dc'] or ['cd', 'ab'] are not valid.

The order should always be from left to right.

EDIT

@Stuart's solution doesn't work in Python 2.7.6

EDIT

@Stuart's solution does work in Python 2.7.6, see the comments below.

解决方案

itertools.product should indeed be able to help you.

The idea is this:-

Consider A1, A2, ..., AN separated by slabs. There will be N-1 slabs.

If there is a slab there is a segmentation. If there is no slab, there is a join.

Thus, for a given sequence of length N, you should have 2^(N-1) such combinations.

Just like the below

import itertools

lst = ['a', 'b', 'c', 'd']

combinatorics = itertools.product([True, False], repeat=len(lst) - 1)

solution = []

for combination in combinatorics:

i = 0

one_such_combination = [lst[i]]

for slab in combination:

i += 1

if not slab: # there is a join

one_such_combination[-1] += lst[i]

else:

one_such_combination += [lst[i]]

solution.append(one_such_combination)

print solution