ES 向量搜索 function score 报错

最新推荐文章于 2024-10-23 15:27:49 发布

悠风号

最新推荐文章于 2024-10-23 15:27:49 发布

阅读量1.8k

点赞数

分类专栏：自然语言处理数据库文章标签： elasticsearch 算法大数据

本文链接：https://blog.youkuaiyun.com/qq_23031939/article/details/126105143

版权

自然语言处理同时被 2 个专栏收录

4 篇文章

订阅专栏

数据库

4 篇文章

订阅专栏

reason "function score query returned an invalid score: NaN for doc: 17085

原因是向量搜索定义评分的计算方法consineSimilarity的计算过程中需要对两个向量求模

故，如果全文索引中存在全零向量数据时，可以将consineSimilarity计算换成其它向量相似度计算方法，例如 dotProduct

GET reference/_search
{ "explain": true, 
  "query": {
    "function_score": {
      "query": {
         "multi_match": {
            "query": "清华大学艺术系的小明同学",
            "fields": ["name", "school", "department", "home_address"]
  
          }
      },
      "functions": [
        {
          "script_score": {
            "script": {
              "source": "(cosineSimilarity(params.query_vector, 'text_vector') + 1.0)*0.6", 
              "params": {
                "query_vector": [0.17213014641107321, 0.03529149917147957, 0.021164631815954453, 0.04276578593911642, -0.011539837034197614, -0.040505111467529525, -0.013804620485218865, 0.07954305265937577, -0.03259408482301212, 0.042030898483276985, -0.08187475423584016, 0.02331000497613773, -0.012518793076655677, 0.028751927085989483, -0.0450599553695574, 0.02574408603742391, -0.012067035100387393, 0.08755896933278114, 0.0043671871492196096]
              }
            }
          }
        },
        {
          "field_value_factor": {
            "field": "page_r", 
            "modifier": "log1p", 
            "factor": 100, 
            "missing": 1
          }
        }
      ],
      "score_mode": "max", 
      "boost_mode": "multiply"
    }
  },
  "_source": ["student_id", "school", "name"], 
  "size": 20
}

consineSimilarity替换为

"source": "(dopProduct(params.query_vector, 'text_vector') + 1.0)*0.6",