Blog Post Extraction Using Titles ------------- by songlinhai

最新推荐文章于 2024-12-22 23:08:34 发布

最新推荐文章于 2024-12-22 23:08:34 发布 · 162 阅读

本文提出一种利用标题查找的博客文章提取算法，分为两个阶段：首先使用机器学习方法找到标题节点作为文章起始；其次确定文章结束位置，实验结果显示该算法效果理想。

Blog Post Extraction Using Title Finding
Linhai Song, Xueqi Cheng, Yan Guo, Bo Wu, Yu Wang

accepted by ccir'2009: Proceedings of Chinese conference on Information Retrieval
Abstract: With the development of Web2.0, web mining applications pay more attention to blog pages. In order to prevent noises in blog pages from affecting the precision of web mining algorithms, it is very necessary to acquire posts from blog pages correctly. In this paper, we propose a blog post extraction algorithm which uses title finding. There are two stages in this algorithm. In the first stage, text nodes which indicate the title of the post are found and used as the beginning of the post. We take a machine learning approach to realize this stage, and employ SVM as classification model. In the second stage, we find the end of the post. Two methods are introduced in this stage, one uses VIPS segmentation results, and the other is based on hand-coded rules. Experiments are conducted to see how we find titles and how we extract posts. Experimental results show that our algorithm has ideal effects.