1-gram 中文分词

最新推荐文章于 2025-06-18 15:02:08 发布

原创

最新推荐文章于 2025-06-18 15:02:08 发布 · 2.9k 阅读

1 ·

CC 4.0 BY-SA版权

文章标签：

#encoding #output #input #windows #dictionary #each

这篇博客介绍了一个使用Perl编写的1-gram中文分词程序，旨在娱乐并作为作业备份。程序无版权，可自由获取。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

一个1-gram实现，网上有个类似的python的，由于要交作业，写了个perl的，娱乐而已。

备份一下。无版权，需要自取。

#!/usr/bin/perl -w

# Attention please!
# This program should only be executed in UNIX-like platform.
# If run in windows, some unexpected problem will appear.
# In most cases, windows text editor will add in some special control byte in your text file.
# They are sheltered by windows and you almost cannot find them.
# What is worse, this is a chinese segmentation program, an unwanted byte will cause disaster!
#===============================================================================
# Introduction:
# The algorithm used is 1-gram.
# Simply, isn't it? But I don't think so...
# 
# Basicly, three extra files is needed.
# 1. The one used for create a dictinary. Default: 199801q.txt
# 2. The input file. Default: input.txt
# 3. The output file. Default: output.txt
# * Program will create a temp file called "config" to speed up itself.
#==============================