如何获取文本中出现的所有单词

原创于 2022-06-29 08:27:48 发布 · 191 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#sql #字符串拆分 #去重

JAVA计算专栏收录该内容

363 篇文章

订阅专栏

用户希望将CSV文件中的所有单词首字母转为小写，并创建一个包含所有不重复单词的列表。已经实现了将单词转小写，但无法生成唯一的单词列表。提供的代码尝试通过检查结果列表来去重，但并未达到预期效果。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

【问题】

I have this csv file

Cat, and, dog, bites

Yahoo, news, claims, a, cat, mated, with, a, dog, and, produced, viable, offspring

Cat, killer, likely, is, a, big, dog

Professional, free, advice, on, dog, training, puppy, training

Cat, and, kitten, training, and, behavior

Dog, &, Cat, provides, dog, training, in Eugene, Oregon

Dog, and, cat, is, a, slang, term, used, by, police, officers, for, a, male-female, relationship

Shop, for, your, show, dog, grooming, and, pet, supplies

I want to make all the words start with a small letter and create a list which will include all the unique items from the above csv file. Have you any idea? Thanks in advance! So far, I have managed to convert all the words with a small letter:

unique_row_items = set([field.strip().lower()for field in row])

But i can't manage the other one.

def unique():

rows = list(csv.reader(open('example_1.csv', 'r'), delimiter=','))

result = []

for r in rows:

key = r

if key not in result:

result.append(r)

return result

Which does not give the results I want

【回答】

这个问题用SPL实现很方便，脚本如下：

	A
1	= file("D://Words.txt").read()
2	=lower(A1)
3	=A2.words().id()

A1：从文本读取字符串

A2：将字符串转换成小写字母

A3：通过 words 函数将字符串拆分成单个单词组成的序列，再通过 id 函数去除重复成员