The Semantic Search Engine

Azure - 尝试创建并使用一下Azure AI Search 我们可以使用Azure AI Search作为向量的存储库,同时Azure AI Search也提供了keyword search, vector search和Hybrid search等功能。在Azure AI Search中有Index,Indexer, Skillset, Scale, Semantic Ranker等概念,具体的内容可以参考微软的mslearn文章。这边将尝试去创建一个Azure AI Search并以Rag的方式使用一下。 阅读详情

How the Search Engines of the future are going to operate is anybody’s guess, and guessing about the future is always hard. If you read up on the subject in the more theoretical parts of the net, if you take an interest in algorithms and the sorts, you will quickly se some signs in the (not so) distant future. One of the buzzwords in the business is “Semantic” as in “The Semantic Web” or “Latent Semantic Indexing”. These principles will have quit an impact on how we search the net for information and on how we, as webmasters, design web sites and optimize our pages and web to attract vital traffic from the Search Engines.

字串9

 

If you are to succeed in coming to terms with the future, you must first understand the past and the present. Let us, therefore, start with looking into how Search Engines work today and what principles govern the algorithms in use today. 字串8

If we are to design a search system, we can apply to basically different methods of approach. We can either choose to use text indexing or we can choose META indexing. (We could of course combine the two, but we will get to that later)

字串7

 

Text Indexing

字串9

 

In text indexing, the Search Engine will harvest all text of the page, process it to extract a list of relevant content words from each page. This, of course, can be done in many ways, but a likely approach could be the following:

字串4

 

1. Discard articles, prepositions, and conjunctions
2. Discard common verbs (know, see, do, be)
3. Discard pronouns
4. Discard common adjectives (big, late, high)
5. Discard frilly words (therefore, thus, however, albeit, etc.)
6. Discard any words that appear in every document
7. Discard any words that appear in only one document

字串5

 

META Indexing 字串6

META indexing works with META data placed in the different documents and web pages by the author or webmaster. It is the author or webmaster who decides what keywords are relevant for the webpage and inserts these in META tags, which in turn are indexed by the Search Engine. The advantage in this system is that searches can be made to worrk fast and efficient, especially if the keywords used in META are applied intelligently and with a degree of standardisation. Unfortunately the reality of the web in not like this and shady types misused META placing popular but irrelevant keywords just to get traffic. There are very few Search Engines, if any at all, that rely soly on META today.

字串3

 

The Real World

字串5

 

The reality is that Search Engines uses a combination of the two systems. Text indexing is used to extract the content words, and META++ (++ meaning and a lot of other tags and codes apart from META tags) is used to weight the content words individually. When the content words of a webpage are found, the other codes are analyzed. The search engine will evaluate things like content word density, frequency and proximity. These parameters must be within certain threshold levels to do well in the Search Engine.

字串7

 

Content words, keywords and search words are the same thing, what name you use depends on viewpoint. The SE will call them content words, the searcher search words and the web master keywords.

字串4

 

Search engines will also often use different site specific parameter. A good example of this, is Google’s PageRank where links are used to rank the pages and sites for relevant content words.

字串1

 

The Problem. 字串9

The problem with the Search Engines of to day is lag of intelligence. The Search Engine can only find pages that have the chosen key/search/content word in the text. It you for instance are in need of information on “French impressionism” the Search Engines will only find pages which have the words French and impressionism on them. Pages regarding Claude Monet, Renoir exhibitions, the museum at Giverny, or Salon des Refusés will not appear in the Search Engine Result pages even though they are or could be very relevant. If you yourself know very little about French impressionism, you will, perhaps, never consider searching for these words and their fore never find this relevant information.

字串1

 

The ideal Search Engine
The ideal Search Engine does not exists and probably never will, but describing it will be helpful if we are to design a Search Engine better then the ones we have today. Again this could be done in many ways, but this is how it is done in the article Latent Semantic Indexing (se link below)
字串1

?Scope: The ideal engine would be able to search every document on the Internet
?Speed: Results would be available immediately
?Currency: All the information would be kept completely up-to-date
?Recall: We could always find every document relevant to our query. No false positives
?Precision: There would be no irrelevant documents in our result set. No false negatives
?Ranking: The most relevant results would come first, and the ones furthest afield would come last

字串1

 

The Search Engine of the future

字串7

 

The Search Engine of the future will to some extent be semantic. One might say, to the greatest extent possible, be semantic. I believe that the Search Engine of the future will use all the elements in use today plus Latent Semantic Indexing and probably some elements we haven’t seen yet.
Latent Semantic Indexing is a well-defined mathematical method, which uses pure mathematics to create the semantic cohesion between documents and collections of documents. The mathematics is not that complicated and should you be interested, just follow the link below for further explanation.

字串9

 

The way I se things, the Semantic Search Engines will harvest content words in much the same way as is done today. They will also weight in much the same way as today, but when today’s Search Engine stops and present the results, the Semantic Search Engine will go on some steps further.
The Semantic Search Engine will analyse the collection of content words, the relative weight, the cohesion between them and the way they are (semantically) connected. The Search Engine will then find other pages or collections of pages with the same semantic profile or with a semantic profile that falls within an acceptable threshold of values. This might sound as an impossible task, but it is already being done with very good results on some of American universities. It is here worth remembering that Google started as a university project at Stanford University and that Google still keeps close ties with Stanford. Tries have shown that it is indeed possible to produce search results that are relevant and that do not contain the actual search word on all the pages

字串7

 

The future seen form the point of view of the searcher 字串3

When the Search Engines are equipped with semantic intelligence, it will invite the searcher to search for information in a semantic manner, instead of using just single or double keywords. When today, you will search for “French Impressionism”, tomorrow you might be better off searching for “the French impressionism, in particular Claude Monet and Renoir, not exhibitions”. How the specific search is formulated is impossible to say anything about. What operators and wildcards will be made available remains to be seen, but the days of the single keyword search are coming to an end for professional searchers. 字串1

The future form the point of view of the Web Master

字串9

 

Things will change and if you want to have a presence in the semantic search engines of tomorrow, you should start your planning and preparations today. Where the Search Engines of today are relatively easy to second-guess and where achieving a good placement, if you are not in a heavily competitive marked segment, is also relatively easy. The semantic Search Engine will present a more complex problem. The use of multi keyword searches alone gives many more possibilities when you operate in a semantic world. Guessing the actual search sentence might not be possible or not even desirable. You would be much better off trying to optimize your site to land in the middle of the semantic cloud, enabling your pages to appear in the top of many different searches. Her is my bid on what’s important to emphasise

字串9

 

Content

字串7

 

Content will be even more important tomorrow then it is today, and the way in witch we write our content will be essential. Today it is important to know your relevant and realistic keywords and then optimize your pages to hit high for these words. Tomorrow the content will need to be much more varied. When your are aiming for a semantic middle of something, keywords are not enough. You will also need synonyms, acronyms, alternatives, opposites and variations, in fact all the nyms, tives, sites and tions you can think of. 字串6

The scope of the content in the entire site will increase in importance. Where the Search Engines of today look at each page separately (site parameters such as PageRank added of course), the semantic Search Engine will also consider the semantic context between the pages of the site as a whole. The content space of the site will therefore increase in importance.
In other words, if you what to do well in the Search Engines of the future, you need to rewrite and add to your content and maybe broaden the scope of the site,
字串1

Internal links 字串4

As Search Engines move by links and as links bind together the site, it is natural, that the internal link structure of the site, will tell a lot about the semantic cohesion of that site. It is my belief that e.g. Google will use links as a parameter in the semantic evaluation of a site even though links are not part of the math behind Latent Semantic Indexing. As a Web master, you need to look at your link stricture as a semantic road map that, properly made, will enhance the semantic content of your site. Analyse your links, and design them in such a way, that they support the content space you are aiming for. 字串2

External links

字串6

 

Where Google today uses external links to calculate the PageRank of your pages, it is my belief htah external links, inbound as well as outbound, will have enhanced importance in the future. Who you link to, and who links to you, says a lot about what semantic cohesion your site most clearly belongs to. In doing so, the external links also will be an important parameter in the semantic ranking of your site. Outbound links maybe more so then inbound since it is the web Master who controls who the site links to.

字串3

 

Miscellaneous. 字串8

There will be a shift in the way we as web masters look at the sites we manage. Where the basis of today, is the all-important keywords, tomorrow we will need to emphasize context and cohesion. We might still start off with the keywords analysis, but we need to look at the individual keyword in a semantic context as well. Today we might end up with a list of 25 words or phrases that we optimize our pages for. In the future I think we will start with 25 pages, each with a keyword or phrase as the headline and then the semantic “ingredients” underneath. These 25 pages will end up in a collection of content where words like “collection” and “context” are just as important as the word content 字串6

It’s a wrap

字串7

 

This is my bid for the future in SEO, how it actually goes, time will tell. This article might change a lot in the near future, or it might disappear all together because I’m embarrassingly wrong 字串6

For further info follow these links 字串8

http://javelina.cet.middlebury.edu/lsa/out/lsa_definition.htm
http://www.w3.org/2001/sw/
http://infomesh.net/2001/swintro/

字串7
 
Elasticsearch中的语义搜索(Semantic Search)介绍 Elasticsearch语义搜索技术解析 Elasticsearch语义搜索基于自然语言理解技术,通过将文本转换为向量表示(如BERT生成的768维向量)实现语义相关性匹配,突破了传统关键词搜索的局限性。其核心包含向量嵌入(Dense Vector字段)、相似度计算(余弦/欧氏距离)和混合搜索(结合关键词匹配)三大技术组件。该技术能有效解决同义词、多义词问题,在智能客服、内容推荐等场景表现优异,但面临计算开销大、领域适配等挑战。实现时需创建专用向量索引,通过Inference API或外部模型生成向量,并 阅读详情

相关推荐

在 Enhanced Material Search 结果工具栏里加一个 Function Trigger,BAdI_WSD_SEARCH_RESULTS 的设计与落地

文章摘要: 物料主数据项目中,Enhanced Material Search功能常需扩展工具栏按钮以提升操作效率。SAP通过BADI_WSD_SEARCH_RESULTS标准扩展点,允许在搜索结果界面添加自定义按钮(如显示库存、图片等),避免用户频繁切换事务码。该BAdI不改变核心搜索逻辑,而是基于当前选中物料提供快捷操作入口。实施时需注意:按钮功能应与搜索结果强相关(如"显示选中物料库存"而非通用功能),避免工具栏杂乱;应参考但不直接使用SAP示例代码BADI_WSD_SEARCH_RESULTS_EX

2007 年 ~ 2025 年,深耕 SAP 技术 18 年 1239

Semantic Search:一款简单高效的语义搜索库

Semantic Search 是一个专为小型至中型项目设计的语义搜索库,它提供嵌入式向量搜索和语义嵌入功能。这个库特别适合那些需要强大语义搜索能力,但不想被传统搜索系统复杂性所困扰的项目。它的核心优势在于简单性,以及对GGUF BERT模型的支持,让你能够利用复杂的嵌入技术,而不需要深入了解传统搜索系统的复杂性。

u013818406的博客 1361

SemanticSearchEngine:该存储库包含有关软件产品线工程(SPLE的领域)信息的语义搜索引擎开发文件。

语义搜索引擎 该存储库包含有关软件产品线工程(SPLE的领域)信息的语义搜索引擎开发文件。

什么是 Semantic Search(语义搜索)

的搜索技术,它不仅仅匹配关键词,还会分析用户的意图和查询的实际含义,以提供更相关的搜索结果。

bigcarp的专栏 621

Semantic Matching in Search

Semantic Matching in Search。原微软,现在华为诺亚方舟实验室的李航的小册子哦。

博图HMI多语言文本列表一键生成攻略:从Excel模板到TIA Portal的完整流程

本文详细介绍了如何利用Excel模板和TIA Portal API实现博图HMI多语言文本列表的一键生成。通过标准化Excel模板设计、解析TIA Portal的XML结构,并结合C#代码实现自动化导入,大幅提升多语言项目的开发效率。特别适用于需要支持多语言版本的自动化设备HMI界面开发。

weixin_26934905的博客 218

Elasticsearch:语义搜索 - Semantic Search in python

当 OpenAI 于 2022 年 11 月发布 ChatGPT 时,引发了人们对人工智能和机器学习的新一波兴趣。 尽管必要的技术创新已经出现了近十年,而且基本原理的历史甚至更早,但这种巨大的转变引发了各种发展的“寒武纪大爆炸”,特别是在大型语言模型和生成 transformers 领域。 一些怀疑论者认为,这些模型是 “随机鹦鹉”,只能生成他们所接受训练的内容的排列。 有些人认为这些模型是 “黑匣子”,超出了人类理解范围,甚至可能是“黑魔法”,其工作原理完全深奥。

Elastic 中国社区官方博客 3589

麒麟操作系统安装达梦数据库ARM版

Linux安装达梦数据库ARM版

运维@小兵的博客 8769

Elasticsearch:使用 Transformers 和 Elasticsearch 进行语义搜索

语义/向量搜索是一种强大的技术,可以大大提高搜索结果的准确性和相关性。与传统的基于关键字的搜索方法不同,语义搜索使用单词的含义和上下文来理解查询背后的意图并提供更准确的结果。Elasticsearch 是实现语义搜索最流行的工具之一,它是一种高度可扩展且功能强大的搜索引擎,可用于索引和搜索大量数据。在本文中,我们将探讨语义搜索的基础知识以及如何使用 Elasticsearch 实现它。到本文结束时,你将深入了解语义搜索的工作原理以及在你自己的项目中实现它的实用技能。

Elastic 中国社区官方博客 4011

Paddle账户与支付宝功能快速开通指南:全球化支付解决方案

本文为开发者提供了Paddle账户与支付宝功能集成的详细开通与配置指南。通过Paddle这一全球化支付与销售平台,开发者可一站式管理全球支付与税务合规,而集成支付宝则能无缝覆盖中国用户,实现“一个平台,覆盖全球主流支付方式”,有效提升支付转化率与收款效率。

qq_27496129的博客 1119

西门子WinCC VBS脚本实战:5分钟搞定变量记录导出到Excel(附完整代码)

本文提供了一套高效的西门子WinCC VBS脚本方案,帮助工程师在5分钟内将变量记录数据导出至Excel。方案详细解析了时区转换、数据库连接、SQL查询优化等核心难点,并附带了完整、可直接使用的VBS代码,解决了工业自动化项目中数据导出的实际痛点。

moss5的博客 609

An Ontology Search Engine Based on Semantic Analysis

AbstractSearching useful information and locating appropriateOntology from WWW or Semantic Web is an important taskin Ontology research domain. The most difference betweenOntology information and comm

mudboy的专栏 2427

BUUCTF--[OGeek2019]babyrop

才学习了基本的ROP流程,到处找题练,不过也没做出来几道题,以这道32位的题作为例题吧。 这道题是BUUCTF上pwn练习题里的[OGeek2019]babyrop。 代码审计 老规矩先checksec一下: 没有canary保护,nx保护开启排除shellcode可能性,FULL RELEO为地址随机化。 观察主函数,先设定了一个闹铃,到时间就会强制退出程序,解除闹铃的方法在上一篇博客中提到过了, 这道题中闹铃函数依然对我们解题起不到什么干扰作用。 主函数的思路大概是:生成一个随机数,把这个随机数作为

N1rvana的博客 6005

PaddleNLP 语义匹配模型库 semantic_search 模块详解:ERNIE 双塔与交叉编码器实战

semantic_search 是 PaddleNLP transformers 目录下的语义匹配(语义检索)模型模块,围绕 ERNIE 预训练模型封装了 ErnieEncoder、ErnieDualEncoder 与 ErnieCrossEncoder 三类组件,分别对应向量编码、双塔召回与交叉排序三种经典语义匹配范式。读完本文,你将掌握这三类编码器的构造参数、推理与训练调用方式、RocketQ

gitblog_00531的博客 427

《遥感大模型生产级实战:从数据到决策的端到端AI工程》系列第三部分:城市洪涝灾害应急响应与评估平台实战-文章 14:项目二启动与多源异构数据对齐

我们将系统性地梳理城市洪涝场景下的所有关键数据源,从天上的卫星(光学与SAR)、空中的无人机,到地面的物联网传感器和社交媒体信息流,深入剖析每种数据在洪涝响应中的独特价值、获取方式与内在局限性。本章的核心将是一个详尽的、生产级的数据对齐技术全景,我们将分步解决三大核心对齐难题:空间对齐(将所有数据投影到统一的地理坐标系)、时间对齐(处理不同采样频率的时间序列)以及最困难的语义对齐(将文本描述与物理观测相关联)。

panshengwu的博客 2003

linux系统下XDMA驱动的安装与测试问题总结

先看两个常见的问题。一个是驱动安装问题,一个是驱动测试错误;error 512 能识别PCIE的设备,但是The kernel modle installed correctly, but no devices were recognized.

whm128的博客 3803

shadPS4 完整实操指南:PC 上跑通血源诅咒,帧率与排错一次讲清

shadPS4 是用 C++ 编写的 PS4 模拟器,当前可在 Windows、Linux、macOS 上运行血源诅咒、如龙 0 等作品。面向会基本终端操作的中级用户,带你从零走通编译、固件导出、启动第一款游戏,再到帧率调优。 ## 构建 shadPS4 前先核对这 4 项:硬件、平台与编译器门槛 判断标准如下表,任何一项不满足,先别动手编译: | 检查项 | 门槛 | 说明 | |----

gitblog_01021的博客 830

波士顿房价预测分析----以线性回归LinearRegression为例

波士顿房价数据集包括506个样本,每个样本包括13个特征变量和该地区的平均房价,房价显然和多个特征变量相关,先选择一元线性回归与多个特征建立线性方程,观察模型预测的好坏,再选择多元线性回归进行房价预测。13个特征变量缩写含义CRIM城镇人均犯罪率ZN住宅用地所占比例INDUS城镇中非住宅用地所占比例CHAS虚拟变量,用于回归分析NOX环保指数RM每栋住宅的房间数AGE1940 年以前建成的自住单位的比例DIS距离 5 个波士顿的就业中心的加权距离RAD。

我走的每一步都算数 842

ELMo in Action: Building a Semantic Search Engine from Scratch

本文详细介绍了如何利用ELMo(Embeddings from Language Models)构建高效的语义搜索引擎。通过分析ELMo的核心优势,如上下文感知的词向量生成能力,以及多义词处理等特性,文章提供了从系统架构设计到性能优化的完整技术方案,帮助开发者实现精准的语义搜索功能。

690

汉书HE8,好用的汇编代码编写工具

汉书HE8,好用的汇编代码编写工具早期编写汇编代码所用功能强大

Data-Structures-and-Algorithm-Analysis-in-Java.pdf

作者: Weiss, Mark A. 出版社: Addison Wesley Data Structures and Algorithm Analysis in Java is an “advanced algorithms” book that fits between traditional CS2 and Algorithms Analysis courses. In the old ACM Curriculum Guidelines, this course was known as CS7. This text is for readers who want to learn good programming and algorithm analysis skills simultaneously so that they can develop such programs with the maximum amo...

【阅读笔记】Beyond Lexical:A semantic Retrieval Framework for Textual Search Engine

框架 如图所示,右半部分为传统term-based query,左半部分为本文主要专注的deep matching部分。 两边得到返回文档后有一个综合ranking模块。 模型训练与embedding 下面就主要讲BERT模型的用处,以及基本结构,训练方式等。 主要思路是:使用BERT预训练模型,根据人工标注数据和负采样数据来fine-tune模型。 使用模型对doc和query进行embeding:deep semantic representation model。 本文提出了两个fine-tu

weixin_45729745的博客 583

WINCC7.3破解文件

WINCC7.3硬件够破解文件,有需要的下载!WINCC7.3硬件够破解文件,有需要的下载!

Hister语义搜索调参指南:semantic_weight如何平衡关键词与语义

Hister 是一款自托管的个人搜索引擎,除了经典的关键词匹配,它还内置了**语义搜索(Semantic Search)**能力。本文是一份面向新手的 Hister 语义搜索调参指南,带你弄懂核心参数 `semantic_weight` 的合并公式,并给出 3 种典型场景下的推荐取值,帮你快速找到"关键词精准"与"语义模糊"之间的最佳平衡点。 [![Hister Web 界面搜索结果与预览面板]

gitblog_01151的博客 438

怀旧版电脑GBC模拟器

再现原版GBC掌机的界面,电脑运行连游戏界面都一样大小,很怀旧,很真实,(英文版)强烈推荐,很小的文件,很难才找到的。

semantic search examples

Example 1: Find the Top Key Phrases in a Specific Document The following example retrieves the top 10 key phrases from the document specified by the @DocumentId variable in the Document column of the...

数据库天地 124

如何微调PlatoLM-7B:LoRA/PEFT低秩适配实战完整教程

想让 70 亿参数的大模型学会你的专属任务?**PlatoLM-7B** 是一个基于 Llama-2-7b 架构的开源对话模型,用它做微调的最佳路径就是 **LoRA / PEFT 低秩适配**:冻结原始权重,只训练少量适配参数,单张 16GB 显卡就能跑通。这篇完整教程带你从模型文件解读、环境配置、LoRA 参数设置,到训练监控与模型合并,一步步动手微调 PlatoLM-7B。 ## 一、为什

gitblog_00536的博客 540

用 Higress AI Search 插件为 DeepSeek 实现联网搜索:完整配置指南与源码原理

本篇技术指南以 Higress 开源仓库中的 [ai-search 插件教程](https://link.gitcode.com/i/3f43c1b634a7e64956c6c10082d85fe7) 为核心,讲解如何基于 Higress 的 `ai-search` Wasm 插件,为 DeepSeek 等大模型补齐联网搜索能力:通过 Google / Bing / 夸克 / Arxiv 等搜索引

gitblog_00612的博客 333

ChatterBot 存储层文本搜索机制详解:从 search_text 字段到 Bigram 二元组索引

本文以 ChatterBot 官方文档 [docs/storage/text-search.rst](https://link.gitcode.com/i/261dbc3f075bc3ea9bde3456274b8561) 为主体,结合源码深入讲解 ChatterBot 存储适配器(Storage Adapter)的文本搜索能力:训练时如何生成 `search_text` 索引、检索时如何通过 B

gitblog_00006的博客 260

不打开网页也能搜表情包:meme-search Chromium弹窗扩展安装与实战指南

meme-search 是一款开源、可自托管的表情包搜索引擎(Meme Search Engine),用 Python、Ruby 和 Docker 构建,能用 AI 自动识别你本地表情包的内容并支持关键词与语义搜索。它内置的 Chromium 浏览器弹窗扩展,让你**不用打开任何网页**,直接在浏览器工具栏里搜索自己的表情包,选中后一键复制或下载 📎。 [![meme-search 表情包搜索

gitblog_00159的博客 158
上一篇: The Road to the Semantic Web
下一篇: MIT-BIH ECG 信号的数据读取方法和Matlab程序
funson
博客等级 码龄22年 12粉丝 25原创
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付:10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值