Optimizing the Interface Between Knowledge Graphs and LLMs for Complex Reasoning

论文主要内容总结

研究背景与目的

大型语言模型(LLMs)在自然语言处理任务中表现出色,但存在事实性错误和知识更新困难等问题。检索增强生成(RAG)通过外部知识检索缓解这些问题,而结合知识图谱(KGs)的GraphRAG进一步支持多步推理和结构化知识访问。然而,此类系统的超参数优化(如分块大小、检索策略、提示模板等)尚未被系统研究。本文利用Cognee框架,在多跳问答基准上优化KG与LLM接口的超参数,探索性能提升的可能性及评估指标的局限性。

方法与实验设计
  1. 框架与参数:使用Cognee模块化框架,优化分块大小、检索类型(文本块或图三元组)、Top-k值、QA与图构建提示模板、任务处理方法等6个核心参数。
  2. 基准与指标:在HotPotQA、TwoWikiMultiHop、MuSiQue三个多跳QA基准上测试,采用Exact Match(EM)、F1分数及DeepEval的LLM-based正确性指标评估。
  3. 优化算法:使用Tree-structured Parzen Estimator(TPE)算法搜索参数空间,每个实验包含
Here are some possible ways to optimize the previous code: 1. Vectorize the calculations: Instead of using nested loops to compute the responsibility matrix, we can use vectorized operations to speed up the computation. For example, we can use broadcasting to compute the Euclidean distance between each pair of points in a matrix form. Similarly, we can use matrix multiplication to compute the weighted sums of the point clouds. ```python def em_for_alignment(xs: np.ndarray, ys: np.ndarray, num_iter: int = 10) -> Tuple[np.ndarray, np.ndarray]: """ The em algorithm for aligning two point clouds based on affine transformation :param xs: a set of points with size (N, D), N is the number of samples, D is the dimension of points :param ys: a set of points with size (M, D), M is the number of samples, D is the dimension of points :param num_iter: the number of EM iterations :return: ys_new: the aligned points: ys_new = ys @ affine + translation responsibility: the responsibility matrix P=[p(y_m | x_n)] with size (N, M), whose elements indicating the correspondence between the points """ # initialize the affine matrix and translation vector affine = np.eye(xs.shape[1]) translation = np.zeros(xs.shape[1]) # initialize the responsibility matrix responsibility = np.zeros((xs.shape[0], ys.shape[0])) for i in range(num_iter): # E-step: compute the responsibility matrix diff = xs[:, np.newaxis, :] - ys[np.newaxis, :, :] sq_dist = np.sum(diff ** 2, axis=-1) responsibility = np.exp(-0.5 * sq_dist) / (2 * np.pi) ** (xs.shape[1] / 2) responsibility /= np.sum(responsibility, axis=1, keepdims=True) # M-step: update the affine matrix and translation vector xs_weighted = responsibility.T @ xs ys_weighted = responsibility.T @ ys affine, _, _, _ = np.linalg.lstsq(xs_weighted, ys_weighted, rcond=None) translation = np.mean(ys, axis=0) - np.mean(xs @ affine, axis=0) # compute the aligned points ys_new = ys @ affine + translation return ys_new, responsibility ``` 2. Use the Kabsch algorithm: Instead of using the weighted least squares solution to update the affine matrix, we can use the Kabsch algorithm, which is a more efficient and numerically stable method for finding the optimal rigid transformation between two point clouds. The Kabsch algorithm consists of three steps: centering the point clouds, computing the covariance matrix, and finding the optimal rotation matrix. ```python def em_for_alignment(xs: np.ndarray, ys: np.ndarray, num_iter: int = 10) -> Tuple[np.ndarray, np.ndarray]: """ The em algorithm for aligning two point clouds based on affine transformation :param xs: a set of points with size (N, D), N is the number of samples, D is the dimension of points :param ys: a set of points with size (M, D), M is the number of samples, D is the dimension of points :param num_iter: the number of EM iterations :return: ys_new: the aligned points: ys_new = ys @ affine + translation responsibility: the responsibility matrix P=[p(y_m | x_n)] with size (N, M), whose elements indicating the correspondence between the points """ # center the point clouds xs_centered = xs - np.mean(xs, axis=0) ys_centered = ys - np.mean(ys, axis=0) # initialize the affine matrix and translation vector affine = np.eye(xs.shape[1]) translation = np.zeros(xs.shape[1]) # initialize the responsibility matrix responsibility = np.zeros((xs.shape[0], ys.shape[0])) for i in range(num_iter): # E-step: compute the responsibility matrix diff = xs_centered[:, np.newaxis, :] - ys_centered[np.newaxis, :, :] sq_dist = np.sum(diff ** 2, axis=-1) responsibility = np.exp(-0.5 * sq_dist) / (2 * np.pi) ** (xs.shape[1] / 2) responsibility /= np.sum(responsibility, axis=1, keepdims=True) # M-step: update the affine matrix and translation vector cov = xs_centered.T @ responsibility @ ys_centered u, _, vh = np.linalg.svd(cov) r = vh.T @ u.T t = np.mean(ys, axis=0) - np.mean(xs @ r, axis=0) affine = np.hstack((r, t[:, np.newaxis])) # compute the aligned points ys_new = ys @ affine[:, :-1] + affine[:, -1] return ys_new, responsibility ``` The Kabsch algorithm is more efficient than the weighted least squares solution, especially when the point clouds are high-dimensional or noisy. However, it only works for rigid transformations, i.e., rotations and translations. If the transformation between the point clouds is not rigid, we need to use a more general method, such as the Procrustes analysis or the Iterative Closest Point (ICP) algorithm.
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

UnknownBody

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值