ElasticSearch43：初识搜索引擎_相关度评分TF&IDF算法揭秘

最新推荐文章于 2025-06-10 15:54:49 发布
原创 最新推荐文章于 2025-06-10 15:54:49 发布 · 1k 阅读 · 1 ·
CC 4.0 BY-SA版权
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
文章标签：

#elasticsearch

本文详细解析了Elasticsearch中TF-IDF算法的工作原理，包括词频（TF）、逆文档频率（IDF）及字段长度规范化等概念，并通过具体示例展示了_score分数的计算过程。

摘要生成于 C知道 ，由 DeepSeek-R1 满血版支持， 前往体验 >

1.算法介绍
relevance score算法，简单来说，就是计算出，一个索引中的文本，与搜索的文本，他们之间的关联匹配程序
ElasticSearch使用的是term frequency/inverse document frequency算法，简称TF/IDF算法
Term frequency，搜索文本中的各个词条在field文本中出现了多少次，出现次数越多，就越相关
Inverse document frequency：搜索文本中的各个词条在整个索引的所有文档中出现多少次，出现的次数越多，就越不相关
Field-length norm：field长度，field越长，相关度越弱

TF例子：搜索请求hello world
doc1：hello you，and world is very good
doc2：hello you，how are you
显然，搜索的请求在doc1中出现了两次，在doc2中出现了一次，doc1更相关

IDF例子：搜索请求hello world
doc1：hello，today is very good -- hello
doc2:hi world,how are you  -- world
比如说，在index中有一万条document，hello这个单词在所有的document中，一共出现了1000次，word这个单词在所有的document中一共出现了100次，
那么显然doc2更相关，因为world在所有document中出现的次数更少，越不相关

Field-length norm：field长度，field越长，相关度越弱
搜索请求：hello world
doc1：{"title":"hello article","contennt":"balabala  一万个单词"}
doc2: {"title":"my article","content":"balabala 一万个单词 ,hi world"}

hello world在整个index中出现的次数是一样多的
doc1的field：（hello出现在title中，world出现在content中），field更短，所以相关度更强



2._score是如何被计算出来的？

执行一次请求
GET /test_index/test_type/_search
{
  "query":{
    "match":{
      "test_field": "test world"
    }
  }
}


执行结果：

{
  "took": 36,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 5,
    "max_score": 1.4867525,
    "hits": [
      {
        "_index": "test_index",
        "_type": "test_type",
        "_id": "6",
        "_score": 1.4867525,
        "_source": {
          "test_field": "test world test"
        }
      },
      {
        "_index": "test_index",
        "_type": "test_type",
        "_id": "3",
        "_score": 1.0754046,
        "_source": {
          "test_field": "test world"
        }
      },
      {
        "_index": "test_index",
        "_type": "test_type",
        "_id": "5",
        "_score": 0.9471576,
        "_source": {
          "test_field": "test 001"
        }
      },
      {
        "_index": "test_index",
        "_type": "test_type",
        "_id": "1",
        "_score": 0.7126236,
        "_source": {
          "test_field": "test hello001"
        }
      },
      {
        "_index": "test_index",
        "_type": "test_type",
        "_id": "2",
        "_score": 0.45203948,
        "_source": {
          "test_field": "world xxxxxxxxxxxxxxxxxxxxxxxxxxxx"
        }
      }
    ]
  }
}



我们可以查看一下如何计算出的_score
GET /test_index/test_type/_search?explain
{
  "query":{
    "match":{
      "test_field": "test world"
    }
  }
}


执行结果，可以看到每个document的_score怎么计算出来的

{
  "took": 294,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 5,
    "max_score": 1.4867525,
    "hits": [
      {
        "_shard": "[test_index][2]",
        "_node": "5JcZFTo8TMGAcBR5psWKmg",
        "_index": "test_index",
        "_type": "test_type",
        "_id": "6",
        "_score": 1.4867525,
        "_source": {
          "test_field": "test world test"
        },
        "_explanation": {
          "value": 1.4867526,
          "description": "sum of:",
          "details": [
            {
              "value": 1.4867526,
              "description": "sum of:",
              "details": [
                {
                  "value": 1.1230313,
                  "description": "weight(test_field:test in 0) [PerFieldSimilarity], result of:",
                  "details": [
                    {
                      "value": 1.1230313,
                      "description": "score(doc=0,freq=2.0 = termFreq=2.0\n), product of:",
                      "details": [
                        {
                          "value": 0.98082924,
                          "description": "idf, computed as log(1 + (docCount - docFreq + 0.5) / (docFreq + 0.5)) from:",
                          "details": [
                            {
                              "value": 1,
                              "description": "docFreq",
                              "details": []
                            },
                            {
                              "value": 3,
                              "description": "docCount",
                              "details": []
                            }
                          ]
                        },
                        {
                          "value": 1.1449814,
                          "description": "tfNorm, computed as (freq * (k1 + 1)) / (freq + k1 * (1 - b + b * fieldLength / avgFieldLength)) from:",
                          "details": [
                            {
                              "value": 2,
                              "description": "termFreq=2.0",
                              "details": []
                            },
                            {
                              "value": 1.2,
                              "description": "parameter k1",
                              "details": []
                            },
                            {
                              "value": 0.75,
                              "description": "parameter b",
                              "details": []
                            },
                            {
                              "value": 2.3333333,
                              "description": "avgFieldLength",
                              "details": []
                            },
                            {
                              "value": 4,
                              "description": "fieldLength",
                              "details": []
                            }
                          ]
                        }
                      ]
                    }
                  ]
                },
                {
                  "value": 0.36372137,
                  "description": "weight(test_field:world in 0) [PerFieldSimilarity], result of:",
                  "details": [
                    {
                      "value": 0.36372137,
                      "description": "score(doc=0,freq=1.0 = termFreq=1.0\n), product of:",
                      "details": [
                        {
                          "value": 0.47000363,
                          "description": "idf, computed as log(1 + (docCount - docFreq + 0.5) / (docFreq + 0.5)) from:",
                          "details": [
                            {
                              "value": 2,
                              "description": "docFreq",
                              "details": []
                            },
                            {
                              "value": 3,
                              "description": "docCount",
                              "details": []
                            }
                          ]
                        },
                        {
                          "value": 0.7738693,
                          "description": "tfNorm, computed as (freq * (k1 + 1)) / (freq + k1 * (1 - b + b * fieldLength / avgFieldLength)) from:",
                          "details": [
                            {
                              "value": 1,
                              "description": "termFreq=1.0",
                              "details": []
                            },
                            {
                              "value": 1.2,
                              "description": "parameter k1",
                              "details": []
                            },
                            {
                              "value": 0.75,
                              "description": "parameter b",
                              "details": []
                            },
                            {
                              "value": 2.3333333,
                              "description": "avgFieldLength",
                              "details": []
                            },
                            {
                              "value": 4,
                              "description": "fieldLength",
                              "details": []
                            }
                          ]
                        }
                      ]
                    }
                  ]
                }
              ]
            },
            {
              "value": 0,
              "description": "match on required clause, product of:",
              "details": [
                {
                  "value": 0,
                  "description": "# clause",
                  "details": []
                },
                {
                  "value": 1,
                  "description": "*:*, product of:",
                  "details": [
                    {
                      "value": 1,
                      "description": "boost",
                      "details": []
                    },
                    {
                      "value": 1,
                      "description": "queryNorm",
                      "details": []
                    }
                  ]
                }
              ]
            }
          ]
        }
      },
      {
        "_shard": "[test_index][4]",
        "_node": "5JcZFTo8TMGAcBR5psWKmg",
        "_index": "test_index",
        "_type": "test_type",
        "_id": "3",
        "_score": 1.0754046,
        "_source": {
          "test_field": "test world"
        },
        "_explanation": {
          "value": 1.0754046,
          "description": "sum of:",
          "details": [
            {
              "value": 1.0754046,
              "description": "sum of:",
              "details": [
                {
                  "value": 0.5377023,
                  "description": "weight(test_field:test in 0) [PerFieldSimilarity], result of:",
                  "details": [
                    {
                      "value": 0.5377023,
                      "description": "score(doc=0,freq=1.0 = termFreq=1.0\n), product of:",
                      "details": [
                        {
                          "value": 0.6931472,
                          "description": "idf, computed as log(1 + (docCount - docFreq + 0.5) / (docFreq + 0.5)) from:",
                          "details": [
                            {
                              "value": 1,
                              "description": "docFreq",
                              "details": []
                            },
                            {
                              "value": 2,
                              "description": "docCount",
                              "details": []
                            }
                          ]
                        },
                        {
                          "value": 0.7757405,
                          "description": "tfNorm, computed as (freq * (k1 + 1)) / (freq + k1 * (1 - b + b * fieldLength / avgFieldLength)) from:",
                          "details": [
                            {
                              "value": 1,
                              "description": "termFreq=1.0",
                              "details": []
                            },
                            {
                              "value": 1.2,
                              "description": "parameter k1",
                              "details": []
                            },
                            {
                              "value": 0.75,
                              "description": "parameter b",
                              "details": []
                            },
                            {
                              "value": 1.5,
                              "description": "avgFieldLength",
                              "details": []
                            },
                            {
                              "value": 2.56,
                              "description": "fieldLength",
                              "details": []
                            }
                          ]
                        }
                      ]
                    }
                  ]
                },
                {
                  "value": 0.5377023,
                  "description": "weight(test_field:world in 0) [PerFieldSimilarity], result of:",
                  "details": [
                    {
                      "value": 0.5377023,
                      "description": "score(doc=0,freq=1.0 = termFreq=1.0\n), product of:",
                      "details": [
                        {
                          "value": 0.6931472,
                          "description": "idf, computed as log(1 + (docCount - docFreq + 0.5) / (docFreq + 0.5)) from:",
                          "details": [
                            {
                              "value": 1,
                              "description": "docFreq",
                              "details": []
                            },
                            {
                              "value": 2,
                              "description": "docCount",
                              "details": []
                            }
                          ]
                        },
                        {
                          "value": 0.7757405,
                          "description": "tfNorm, computed as (freq * (k1 + 1)) / (freq + k1 * (1 - b + b * fieldLength / avgFieldLength)) from:",
                          "details": [
                            {
                              "value": 1,
                              "description": "termFreq=1.0",
                              "details": []
                            },
                            {
                              "value": 1.2,
                              "description": "parameter k1",
                              "details": []
                            },
                            {
                              "value": 0.75,
                              "description": "parameter b",
                              "details": []
                            },
                            {
                              "value": 1.5,
                              "description": "avgFieldLength",
                              "details": []
                            },
                            {
                              "value": 2.56,
                              "description": "fieldLength",
                              "details": []
                            }
                          ]
                        }
                      ]
                    }
                  ]
                }
              ]
            },
            {
              "value": 0,
              "description": "match on required clause, product of:",
              "details": [
                {
                  "value": 0,
                  "description": "# clause",
                  "details": []
                },
                {
                  "value": 1,
                  "description": "*:*, product of:",
                  "details": [
                    {
                      "value": 1,
                      "description": "boost",
                      "details": []
                    },
                    {
                      "value": 1,
                      "description": "queryNorm",
                      "details": []
                    }
                  ]
                }
              ]
            }
          ]
        }
      },
      {
        "_shard": "[test_index][1]",
        "_node": "5JcZFTo8TMGAcBR5psWKmg",
        "_index": "test_index",
        "_type": "test_type",
        "_id": "5",
        "_score": 0.9471576,
        "_source": {
          "test_field": "test 001"
        },
        "_explanation": {
          "value": 0.9471576,
          "description": "sum of:",
          "details": [
            {
              "value": 0.9471576,
              "description": "sum of:",
              "details": [
                {
                  "value": 0.9471576,
                  "description": "weight(test_field:test in 0) [PerFieldSimilarity], result of:",
                  "details": [
                    {
                      "value": 0.9471576,
                      "description": "score(doc=0,freq=1.0 = termFreq=1.0\n), product of:",
                      "details": [
                        {
                          "value": 1.3862944,
                          "description": "idf, computed as log(1 + (docCount - docFreq + 0.5) / (docFreq + 0.5)) from:",
                          "details": [
                            {
                              "value": 1,
                              "description": "docFreq",
                              "details": []
                            },
                            {
                              "value": 5,
                              "description": "docCount",
                              "details": []
                            }
                          ]
                        },
                        {
                          "value": 0.6832298,
                          "description": "tfNorm, computed as (freq * (k1 + 1)) / (freq + k1 * (1 - b + b * fieldLength / avgFieldLength)) from:",
                          "details": [
                            {
                              "value": 1,
                              "description": "termFreq=1.0",
                              "details": []
                            },
                            {
                              "value": 1.2,
                              "description": "parameter k1",
                              "details": []
                            },
                            {
                              "value": 0.75,
                              "description": "parameter b",
                              "details": []
                            },
                            {
                              "value": 1.2,
                              "description": "avgFieldLength",
                              "details": []
                            },
                            {
                              "value": 2.56,
                              "description": "fieldLength",
                              "details": []
                            }
                          ]
                        }
                      ]
                    }
                  ]
                }
              ]
            },
            {
              "value": 0,
              "description": "match on required clause, product of:",
              "details": [
                {
                  "value": 0,
                  "description": "# clause",
                  "details": []
                },
                {
                  "value": 1,
                  "description": "*:*, product of:",
                  "details": [
                    {
                      "value": 1,
                      "description": "boost",
                      "details": []
                    },
                    {
                      "value": 1,
                      "description": "queryNorm",
                      "details": []
                    }
                  ]
                }
              ]
            }
          ]
        }
      },
      {
        "_shard": "[test_index][3]",
        "_node": "5JcZFTo8TMGAcBR5psWKmg",
        "_index": "test_index",
        "_type": "test_type",
        "_id": "1",
        "_score": 0.7126236,
        "_source": {
          "test_field": "test hello001"
        },
        "_explanation": {
          "value": 0.71262366,
          "description": "sum of:",
          "details": [
            {
              "value": 0.71262366,
              "description": "sum of:",
              "details": [
                {
                  "value": 0.71262366,
                  "description": "weight(test_field:test in 0) [PerFieldSimilarity], result of:",
                  "details": [
                    {
                      "value": 0.71262366,
                      "description": "score(doc=0,freq=1.0 = termFreq=1.0\n), product of:",
                      "details": [
                        {
                          "value": 0.98082924,
                          "description": "idf, computed as log(1 + (docCount - docFreq + 0.5) / (docFreq + 0.5)) from:",
                          "details": [
                            {
                              "value": 1,
                              "description": "docFreq",
                              "details": []
                            },
                            {
                              "value": 3,
                              "description": "docCount",
                              "details": []
                            }
                          ]
                        },
                        {
                          "value": 0.7265522,
                          "description": "tfNorm, computed as (freq * (k1 + 1)) / (freq + k1 * (1 - b + b * fieldLength / avgFieldLength)) from:",
                          "details": [
                            {
                              "value": 1,
                              "description": "termFreq=1.0",
                              "details": []
                            },
                            {
                              "value": 1.2,
                              "description": "parameter k1",
                              "details": []
                            },
                            {
                              "value": 0.75,
                              "description": "parameter b",
                              "details": []
                            },
                            {
                              "value": 1.3333334,
                              "description": "avgFieldLength",
                              "details": []
                            },
                            {
                              "value": 2.56,
                              "description": "fieldLength",
                              "details": []
                            }
                          ]
                        }
                      ]
                    }
                  ]
                }
              ]
            },
            {
              "value": 0,
              "description": "match on required clause, product of:",
              "details": [
                {
                  "value": 0,
                  "description": "# clause",
                  "details": []
                },
                {
                  "value": 1,
                  "description": "_type:test_type, product of:",
                  "details": [
                    {
                      "value": 1,
                      "description": "boost",
                      "details": []
                    },
                    {
                      "value": 1,
                      "description": "queryNorm",
                      "details": []
                    }
                  ]
                }
              ]
            }
          ]
        }
      },
      {
        "_shard": "[test_index][2]",
        "_node": "5JcZFTo8TMGAcBR5psWKmg",
        "_index": "test_index",
        "_type": "test_type",
        "_id": "2",
        "_score": 0.45203948,
        "_source": {
          "test_field": "world xxxxxxxxxxxxxxxxxxxxxxxxxxxx"
        },
        "_explanation": {
          "value": 0.45203945,
          "description": "sum of:",
          "details": [
            {
              "value": 0.45203945,
              "description": "sum of:",
              "details": [
                {
                  "value": 0.45203945,
                  "description": "weight(test_field:world in 0) [PerFieldSimilarity], result of:",
                  "details": [
                    {
                      "value": 0.45203945,
                      "description": "score(doc=0,freq=1.0 = termFreq=1.0\n), product of:",
                      "details": [
                        {
                          "value": 0.47000363,
                          "description": "idf, computed as log(1 + (docCount - docFreq + 0.5) / (docFreq + 0.5)) from:",
                          "details": [
                            {
                              "value": 2,
                              "description": "docFreq",
                              "details": []
                            },
                            {
                              "value": 3,
                              "description": "docCount",
                              "details": []
                            }
                          ]
                        },
                        {
                          "value": 0.96177864,
                          "description": "tfNorm, computed as (freq * (k1 + 1)) / (freq + k1 * (1 - b + b * fieldLength / avgFieldLength)) from:",
                          "details": [
                            {
                              "value": 1,
                              "description": "termFreq=1.0",
                              "details": []
                            },
                            {
                              "value": 1.2,
                              "description": "parameter k1",
                              "details": []
                            },
                            {
                              "value": 0.75,
                              "description": "parameter b",
                              "details": []
                            },
                            {
                              "value": 2.3333333,
                              "description": "avgFieldLength",
                              "details": []
                            },
                            {
                              "value": 2.56,
                              "description": "fieldLength",
                              "details": []
                            }
                          ]
                        }
                      ]
                    }
                  ]
                }
              ]
            },
            {
              "value": 0,
              "description": "match on required clause, product of:",
              "details": [
                {
                  "value": 0,
                  "description": "# clause",
                  "details": []
                },
                {
                  "value": 1,
                  "description": "*:*, product of:",
                  "details": [
                    {
                      "value": 1,
                      "description": "boost",
                      "details": []
                    },
                    {
                      "value": 1,
                      "description": "queryNorm",
                      "details": []
                    }
                  ]
                }
              ]
            }
          ]
        }
      }
    ]
  }
}


说明：拿第一个document分析

"_explanation": {
          "value": 1.0754046,
          "description": "sum of:",
          "details": [
            {
              "value": 1.0754046,
              "description": "sum of:",
              "details": [
                {
                  "value": 0.5377023,
                  "description": "weight(test_field:test in 0) [PerFieldSimilarity], result of:",
                  "details": [
                    {
                      "value": 0.5377023,
					  注释：termFreq：term出现的频率为1
                      "description": "score(doc=0,freq=1.0 = termFreq=1.0\n), product of:",
                      "details": [
                        {
                          "value": 0.6931472,
						  注释：idf 计算公式
                          "description": "idf, computed as log(1 + (docCount - docFreq + 0.5) / (docFreq + 0.5)) from:",
                          "details": [
                            {
                              "value": 1,
                              "description": "docFreq",
                              "details": []
                            },
                            {
                              "value": 2,
                              "description": "docCount",
                              "details": []
                            }
                          ]
                        },
                        {
                          "value": 0.7757405,
                          "description": "tfNorm, computed as (freq * (k1 + 1)) / (freq + k1 * (1 - b + b * fieldLength / avgFieldLength)) from:",
                          "details": [
                            {
                              "value": 1,
                              "description": "termFreq=1.0",
                              "details": []
                            },
                            {
                              "value": 1.2,
                              "description": "parameter k1",
                              "details": []
                            },
                            {
                              "value": 0.75,
                              "description": "parameter b",
                              "details": []
                            },
                            {
                              "value": 1.5,
                              "description": "avgFieldLength",
                              "details": []
                            },
                            {
                              "value": 2.56,
                              "description": "fieldLength",
                              "details": []
                            }
                          ]
                        }
                      ]
                    }
                  ]
                },
                {
                  "value": 0.5377023,
                  "description": "weight(test_field:world in 0) [PerFieldSimilarity], result of:",
                  "details": [
                    {
                      "value": 0.5377023,
                      "description": "score(doc=0,freq=1.0 = termFreq=1.0\n), product of:",
                      "details": [
                        {
                          "value": 0.6931472,
                          "description": "idf, computed as log(1 + (docCount - docFreq + 0.5) / (docFreq + 0.5)) from:",
                          "details": [
                            {
                              "value": 1,
                              "description": "docFreq",
                              "details": []
                            },
                            {
                              "value": 2,
                              "description": "docCount",
                              "details": []
                            }
                          ]
                        },
                        {
                          "value": 0.7757405,
						  注释：tfNorm
                          "description": "tfNorm, computed as (freq * (k1 + 1)) / (freq + k1 * (1 - b + b * fieldLength / avgFieldLength)) from:",
                          "details": [
                            {
                              "value": 1,
                              "description": "termFreq=1.0",
                              "details": []
                            },
                            {
                              "value": 1.2,
                              "description": "parameter k1",
                              "details": []
                            },
                            {
                              "value": 0.75,
                              "description": "parameter b",
                              "details": []
                            },
                            {
                              "value": 1.5,
                              "description": "avgFieldLength",
                              "details": []
                            },
                            {
                              "value": 2.56,
                              "description": "fieldLength",
                              "details": []
                            }
                          ]
                        }
                      ]
                    }
                  ]
                }
              ]
            },
            {
              "value": 0,
              "description": "match on required clause, product of:",
              "details": [
                {
                  "value": 0,
                  "description": "# clause",
                  "details": []
                },
                {
                  "value": 1,
                  "description": "*:*, product of:",
                  "details": [
                    {
                      "value": 1,
                      "description": "boost",
                      "details": []
                    },
                    {
                      "value": 1,
                      "description": "queryNorm",
                      "details": []
                    }
                  ]
                }
              ]
            }
          ]
        }



也可以查询一条数据，执行explain
GET /test_index/test_type/1/_explain
{
  "query":{
    "match":{
      "test_field": "test world"
    }
  }
}









TF-IDF及其算法
weixin_34290000的博客
07-17 3734
概念 TF-IDF（term frequency–inverse document frequency）是一种用于资讯检索与资讯探勘的常用加权技术。TF-IDF是一种统计方法，用以评估一字词对于一个文件集或一个语料库中的其中一份文件的重要程度。字词的重要性随着它在文件中出现的次数成正比增加，但同时会随着它在语料库中出现的频率成反比下降。TF-IDF加权的各种形式常被搜寻引擎应用，作为文...
Elasticsearch全文搜索与TF/IDF
weixin_33777877的博客
12-24 303
2019独角兽企业重金招聘Python工程师标准>>> ...
TF-IDF详解
最新发布
m0_53387935的博客
06-10 115
TF-IDF（Term Frequency – Inverse Document Frequency）是一种用于信息检索与文本挖掘的常用技术。
ElasticSearch系列——相关度评分TF&IDF算法
weixin_42894555的博客
07-12 409
文章目录算法介绍_score是如何被计算出来的 ElasticSearch系列——主目录 算法介绍 relevance score算法，简单来说，就是计算出，一个索引中的文本，与搜索文本，他们之间的关联匹配程度 Elasticsearch使用的是 term frequency/inverse document frequency算法，简称为TF/IDF算法 Term frequency： 搜索文本中的各个词条在field文本中出现了多少次，出现次数越多，就越相关 搜索请求：hello world .
ElasticSearch--TF/IDF算法
小毛贼_哪里逃
03-09 276
​​​​​ 查看搜索时所使用的算法信息 GET /accounts/person/_search?explain 查看一个文档匹配时所用的算法信息 GET /accounts/person/2/_explain { "query":{ "match":{ "desc":"good" } } } ...
Elasticsearch 理解相关性评分（TF-IDF、BM25等）
Dxy1239310216的博客
07-07 1588
相关性评分是衡量搜索结果与用户查询匹配程度的指标。Elasticsearch通过复杂的算法计算每个文档与查询的相关性得分，得分越高，表示文档与查询的匹配程度越高。在电商网站、知识库等应用场景中，相关性评分直接决定了用户能否快速找到他们需要的信息。TF-IDF（Term Frequency-Inverse Document Frequency）是一种经典的信息检索算法，用于评估一个词语对于一个文件集或一个语料库中的其中一份文件的重要程度。：词频，即词语在文档中出现的次数。
JavaScript全文搜索实现：TF-IDF与Okapi BM25解析
本文主要探讨了JavaScript环境下实现全文搜索并进行相关度评分的解决方案，从初识全文搜索的简单实现，如使用SQL的LIKE语句，到遇到性能和精准度问题后转向MySQL的FULLTEXT索引，再到面对大规模数据时采用Elastic...
Elasticsearch(二)：核心
IT巅峰技术专栏
10-25 830
1 开篇 上一篇文章对elasticsearch进行了整体的介绍，初步了解了下elasticsearch的发展、应用场景、整体功能架构和一些基本的概念，并且实际上手搭建了一个集群做了下文档的操作；elasticsearch是如何进行数据分片的呢，集群的容错性又是如何保证的呢。为什么elasticsearch是近实时搜索而不是实时搜索的呢？带着这些问题，我们将从分片容错、选主、索引生命周期、倒排索引等核心概念来深入了解elasticsearch。 2 分片与容错 2.1 分片 ...
ES核心知识篇part2
06-20
55.初识搜索引擎_相关度评分TF&IDF算法独家解密.avi将介绍搜索引擎如何评估文档的相关性，特别是TF-IDF算法，这是衡量一个词对于文档重要性的常见方法。36.分页搜索以及deep paging性能问题深度图解揭秘.avi探讨了...
分布式搜索引擎 elasticsearch
KYGALYX的博客
02-07 3409
分布式搜索引擎 elasticsearch 第一部分 1.初识elasticsearch 1.1.了解ES 1.1.1.elasticsearch的作用 elasticsearch是一款非常强大的开源搜索引擎，具备非常多强大功能，可以帮助我们从海量数据中快速找到需要的内容 例如： 在GitHub搜索代码 在电商网站搜索商品 在百度搜索答案 在打车软件搜索附近的车 1.1.2.ELK技术栈 elasticsearch结合kibana、Logstash、Beats，也就是elasti
Elasticsearch学习(2) 基本概念
qGANG的博客
03-17 2264
1、Es基本定义 1.1 文档(Document) Es 是面向文档的，文档是所有可搜索数据的最小单位。它对应关系型数据库中的表里的一条数据。 1.2 索引(Index) 索引简单来说就是相似结构文档的集合，索引有一个名称，一个索引可以包含很多文档，一个索引就代表了一类类似的或者相同的文档，比如说建立一个商品索引，里面可能就存放了所有的商品数据，也就是所有的商品文档。每一个索引都是自己的 Mapping 定义文件，用来去描述去包含文档字段的类型，分片（Shard）体现的是物理空间...
elasticsearch relevance score algorithm (一) ：TF / IDF
virgil
10-10 243
题外话 记忆缩写的时候，最好记住它们的全拼，这样就能快速的反应出这个缩写是干什么的。 比如说 API，大家都知道代表接口的意思，但是具体来说接口是什么，就开始含糊其辞了。假如说我们记住 API 的全拼是 application program interface, 就可以直接回答出 API 是应用程序的入口。 1、 TF / IDF Term Frequency / Inverse Document Frequency 词频 / 逆文档频率 使用场景： 在 elasticsearch version 5 及
elasticsearch 笔记十五：相关度评分TF&IDF算法简介
qzqanzc的博客
10-25 359
算法介绍 relevance score ,就是计算出一个索引中的文本，与搜索文本，他们之间的关系，它们之间的关联匹配度是由Elasticsearch使用的 term frequency/inverse document frequency 算法，简称TF/IDF算法 Term frequency :搜索文本中的各个词条在field 文本中出现多少次，出现的次数越多，就越相关 Inverse...
ElasticSearch原理
qq_17312239的博客
06-29 292
Elasticsearch－基础介绍及索引原理分析最近在参与一个基于Elasticsearch作为底层数据框架提供大数据量(亿级)的实时统计查询的方案设计工作，花了些时间学习Elasticsearch的基础理论知识，整理了一下，希望能对Elasticsearch感兴趣/想了解的同学有所帮助。 同时也希望有发现内容不正确或者有疑问的地方，望指明，一起探讨，学习，进步。介绍Elasticsearch ...
TF-IDF 算法 以及 Elasticsearch中的打分
weixin_44715126的博客
01-02 1362
TF-IDF算法 elasticsearch打分机制 根据阅读量提高曝光度的方法
Elasticsearch相关评分度TF/IDF算法揭秘
【原】编程界的小学生
05-18 1937
编程界的小学生一、算法介绍1、TF1.1、概念1.2、举例2、IDF2.1、概念2.2、举例3、补充3.1、说明3.2、举例二、Demo演示1、数据准备2、进行搜索3、结果分析 一、算法介绍 Elasticsearch采取的是TF/IDF算法来评估score的，而score决定了排序。每次搜索score分数越大的越靠前。 1、TF 1.1、概念 Term Frequency简称TF，就是搜索文本中的各个词条在要搜索的field文本中出现的次数，次数越多就越相关。 1.2、举例 比如： doc1：hell
白话Elasticsearch24- 深度探秘搜索技术之TF&IDF算法/向量空间模型算法/lucene的相关度分数算法
热门推荐
小工匠
08-04 1万+
文章目录概述boolean modelTF/IDFTF: term frequencyIDF：inversed document frequencylength normvector space model向量空间模型lucene的相关度分数算法boolean modellucene practical scoring functionquery normalization factorquery...
【Elasticsearch】Elasticsearch 相关度评分 TF&IDF
九师兄
10-04 561
1.概述 relevance score算法，简单来说，就是计算出，一个索引中的文本，与搜索文本，他们之间的关联匹配程度 Elasticsearch使用的是 term frequency/inverse document frequency算法，简称为TF/IDF算法 Term frequency(TF)：搜索文本中的各个词条在field文本中出现了多少次，出现次数越多，就越相关 Inverse document frequency(IDF)：搜索文本中的各个词条在整个索引的所有文档中出现了多少次，出现的.
24_ElasticSearch TF&IDF算法以及向量空间模型
架构师的成长之路的博客
03-23 1821
24_ElasticSearch TF&IDF算法以及向量空间模型更多干货分布式实战（干货）spring cloud 实战（干货）mybatis 实战（干货）spring boot 实战（干货）React 入门实战（干货）构建中小型互联网企业架构（干货）python 学习持续更新ElasticSearch 笔记一、概述boolean model 类似and这种逻辑操作符，先过滤出包含指定t...
Python库guillotina_elasticsearch-3.2.0：大数据搜索引擎优化
这意味着开发者可以利用Elasticsearch强大的搜索算法和分布式特性，以及Guillotina的Web服务架构，构建出具有高效搜索引擎功能的应用。 4. 大数据和搜索引擎的应用场景： 在大数据背景下，搜索引擎的作用尤为重要。...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值