6CCS3AIN Pacman MDP Solver

AI 应用开发 × 算法面试题:业务场景实战 12 题【TopK、LRU、RRF 融合、相似度、限流算法,含手撕代码与底层答案】 本文聚焦AI应用岗算法面试新趋势,剖析12道融合业务场景的高频题:从海量日志中挖掘慢查询(分治+小顶堆)、亿级去重(布隆过滤器)、向量与关键词融合排序(RRF)、语义缓存设计(带租户隔离与TTL的LRU)到限流策略(令牌桶+滑动窗口),强调“场景复述→算法选型→手撕代码→生产落地”四步答题框架。突出考察点已从纯算法转向工程权衡、业务理解与系统设计能力,如误判容忍度、实时性与内存预算的平衡,助力开发者应对2026年真实技术挑战。 阅读详情

Coursework

1 Introduction

This coursework exercise asks you to write code to create an MDP-solver to work in the Pacman

environment that we used for the practical exercises.

Read all these instructions before starting.

This exercise will be assessed.

2 Getting started

You should download the file pacman-cw.zip from KEATS. This contains a familiar set of files that

implement Pacman, and version 6 of api.py which defines the observability of the environment that

you will have to deal with, and the same non-deterministic motion model that the practicals used.

Version 6 of api.py, further extends what Pacman can know about the world. In addition to

knowing the location of all the objects in the world (walls, food, capsules, ghosts), Pacman can now

see what state the ghosts are in, and so can decide whether they have to be avoided or not.

3 What you need to do

3.1 Write code

This coursework requires you to write code to control Pacman and win games using an MDP-solver.

For each move, you will need to have the model of Pacman’s world, which consists of all the elements

of a Markov Decision Process, namely:

• A finite set of states S;

• A finite set of actions A;

• A state-transition function P(s

0

|s, a);

• A reward function R;

• A discount factor γ ∈ [0, 1];

Following this you can then compute the action to take, either via Value Iteration, Policy Iteration or

Modified Policy Iteration. It is expected that you will correctly implement such a solver and optimize

the choice of the parameters. There is a (rather familiar) skeleton piece of code to take as your

starting point in the file mdpAgents.py. This code defines the class MDPAgent.

There are two main aims for your code:

1 Mallmann-Trenn / McBurney / 6ccs3ain-cw(a) Win hard in smallGrid

(b) Win hard in mediumClassic

To win games, Pacman has to be able to eat all the food. In this coursework, for these objectives,

“winning” just means getting the environment to report a win. Score is irrelevant.

3.1.1 Getting Excellence points

There is a difference between winning a lot and winning well. This is why completing aim (a) and

(b) from previous section allows you to collect up to 80 points in the Coursework. The remaining

20 points are obtained by having a high Excellence Score Difference in the mediumClassic layout,

a metric that directly comes from having a high average winning score. This can be done through

different strategies, for example through chasing eatable ghosts.

A couple of things to be noted. Let W be the set of games won, i.e., |W| ∈ [0, 25]. For any won

game i ∈ W define sw(i) to be the score obtained in game/run i.

• ∆Se in the marksheet is the Excellence Score Difference. You can use the following formula

to calculate it when you test your code and compare the result against the values in Table 3

∆Se =

X

i∈W

(sw(i) − 1500) (1)

Losses count as 0 score and are not considered. If ∆Se < 0, we set it to 0 (you cannot have

a negative excellence score difference).

• Because smallGrid does not have room for score improvement, we will only look at the

mediumClassic layout

• You can still get excellence points if your code performs poorly in the number of wins; marking

points are assigned independently in the two sections

• Note however that marking points are assigned such that it is not convenient for you to directly

aim for a higher average winning score without securing previous sections’s aims (a) and (b)

first

• We will use the same runs in mediumClassic to derive the marks for Table 2 and Table 3.

3.2 Things to bear in mind

Some things that you may find helpful:

(a) We will evaluate whether your code can win games in smallGrid by running:

python pacman.py -q -n 25 -p MDPAgent -l smallGrid

-l is shorthand for -layout. -p is shorthand for -pacman. -q runs the game without the

interface (making it faster).

(b) We will evaluate whether your code can win games in mediumClassic by running:

python pacman.py -q -n 25 -p MDPAgent -l mediumClassic

The -n 25 runs 25 games in a row.

2 Mallmann-Trenn / McBurney / 6ccs3ain-cw(c) The time limit for evlauation is 25 minute for mediumClassic and 5 minutes for small grid.

It will run on a high performance computer with 26 cores and 192 Gb of RAM. The time

constraints are chosen after repeated practical experience and reflect a fair time bound.

(d) When using the -n option to run multiple games, the same agent (the same instance of

MDPAgent.py) is run in all the games.

That means you might need to change the values of some of the state variables that control

Pacman’s behaviour in between games. You can do that using the final() function.

(e) There is no requirement to use any of the methods described in the practicals, though you

can use these if you wish.

(f) If you wish to use the map code I provided in MapAgent, you may do this, but you need to

include comments that explain what you used and where it came from (just as you would for

any code that you make use of but don’t write yourself).

(g) You can only use libraries that are part of a the standard Python 2.7 distribution. This ensures

that (a) everyone has access to the same libraries (since only the standard distribution is

available on the lab machines) and (b) we don’t have trouble running your code due to some

library incompatibilities.

(h) You should comment your code and have a consistent style all over the file.

3.3 Limitations

There are some limitations on what you can submit.

(a) Your code must be in Python 2.7. Code written in a language other than Python will not be

marked.

Code written in Python 3.X is unlikely to run with the clean copy of pacman-cw that we will

test it against. If is doesn’t run, you will lose marks.

Code using libraries that are not in the standard Python 2.7 distribution will not run (in

particular, NumPy is not allowed). If you choose to use such libraries and your code does not

run as a result, you will lose marks.

(b) Your code must only interact with the Pacman environment by making calls through functions

 in Version 6 of api.py. Code that finds other ways to access information about the

environment will lose marks.

The idea here is to have everyone solve the same task, and have that task explore issues with

non-deterministic actions.

(c) You are not allowed to modify any of the files in pacman-cw.zip except mdpAgents.py.

Similar to the previous point, the idea is that everyone solves the same problem — you can’t

change the problem by modifying the base code that runs the Pacman environment. Therefore,

you are not allowed to modify the api.py file.

(d) You are not allowed to copy, without credit, code that you might get from other students or

find lying around on the Internet. We will be checking.

This is the usual plagiarism statement. When you submit work to be marked, you should only

seek to get credit for work you have done yourself. When the work you are submitting is code,

3 Mallmann-Trenn / McBurney / 6ccs3ain-cwyou can use code that other people wrote, but you have to say clearly that the other person

wrote it — you do that by putting in a comment that says who wrote it. That way we can

adjust your mark to take account of the work that you didn’t do.

(e) Your code must be based on solving the Pacman environment as an MDP. If you don’t submit

a program that contains a recognisable MDP solver, you will lose marks.

(f) The only MDP solvers we will allow are the ones presented in the lecture, i.e., Value iteration,

Policy iteration and Modified policy iteration. In particular, Q-Learning is unacceptable.

(g) Your code must only use the results of the MDP solver to decide what to do. If you submit

code which makes decisions about what to do that uses other information in addition to what

the MDP-solver generates (like ad-hoc ghost avoiding code, for example), you will lose marks.

This is to ensure that your MDP-solver is the thing that can win enough games to pass the

functionality test.

4 What you have to hand in

Your submission should consist of a single ZIP file. (KEATS will be configured to only accept a

single file.) This ZIP file must include a single Python .py file (your code).

The ZIP file must be named:

cw <lastname> <firstname>.zip

so my ZIP file would be named cw mallmann-trenn frederik.zip.

Remember that we are going to evaluate your code by running your code by using variations on

python pacman.py -p MDPAgent

(see Section 5 for the exact commands we will use) and we will do this in a vanilla copy of the

pacman-cw folder, so the base class for your MDP-solving agent must be called MDPAgent.

To streamline the marking of the coursework, you must put all your code in one file, and this file

must be called mdpAgents.py,

Do not just include the whole pacman-cw folder. You should only include the one file that includes

the code you have written.

Submissions that do not follow these instructions will lose marks. That includes submissions which

are RAR files. RAR is not ZIP.

5 How your work will be marked

See cw-marksheet.pdf for more information about the marking.

There will be six components of the mark for your work:

(a) Functionality

We will test your code by running your .py file against a clean copy of pacman-cw.

As discussed above, the number of games you win determines the number of marks you get.

Since we will check it this way, you may want to reset any internal state in your agent using

4 Mallmann-Trenn / McBurney / 6ccs3ain-cwfinal() (see Section 3.2). For the excellence marks, we will look at the winning scores for

the mediumClassic layout.

Since we have a lot of coursework to mark, we will limit how long your code has to demonstrate

that it can win. We will terminate the run of the 25smallGrid games after 5 minutes, and

will terminate the run of the 25 mediumClassic games after 25 minutes. If your code has

failed to win enough games within these times, we will mark it as if it lost. Note that we will

use the -q command, which runs Pacman without the interface, to speed things up.

(b) Code not written in Python will not be marked.

(c) Code that does not run in our test setting will receive 0 marks. Regardless of the reason.

(d) We will release the random seed that we use for marking. Say the seed is 42, then you need

to do the following to verify our marking is correct:

1) fix the random seed to 42 (int, not string type) at line 541 of pacman.py. (not ’42’)

2) download a fresh copy of the new api (to avoid using files you modified yourself)

3) run python pacman.py -q -f -n 25 -p MDPAgent -l mediumClassic

4) you should get the same result as us. If not repeat step 3) again. Should the outcome be

different, then you didn’t fix the random seed correctly. Go back to 1)

A copy of the marksheet, which shows the distribution of marks across the different elements of

C++ 极致工程美学:拆解 Meta Faiss 量化器的 AVX2 向量化与有偏收缩算法 在 Meta 向量检索库 Faiss 的头文件 内部,标量量化器为反量化系数预留了两个并列枚举:无偏(UNBIASED)与有偏(BIASED)。按照经典数理统计教材的直觉惯性,无偏估计一向被奉为无损推断的金科玉律;但在 100 万条 768 维真实特征向量的检索压测中,把缩放模式切到有偏估计后,重构均方误差(MSE)稳定下降 19.60%,P99 长尾失真压低 35.05%,最大极端离群误差更是被削减了 81.46%。向量检索面对的是单次离散几何判决,没有任何大数定律能抹平单次重构方差。本文聚焦 Fais 阅读详情

相关推荐

使用.NET 免费开源包ViewFaceCore做一个人脸识别并注册的功能,没有比我更全的

本文介绍了一个基于WinForm的免费人脸识别系统实现方案。系统使用AForge和ViewFaceCore库,支持从USB或电脑自带摄像头采集图像,具备人脸注册和快速识别功能。硬件方面仅需普通摄像头,软件依赖多个NuGet包(如AForge.Video、ViewFaceCore等)。界面包含视频显示、图像预览和功能按钮控件,后台采用SQLServer存储用户面部特征数据。核心代码展示了人脸检测、特征提取和多线程处理逻辑,通过ViewFaceCore实现高效识别。系统具有识别速度快、准确率高的特点,适合Win

一个靠“吃代码”长大的程序猿,专治各种“Bug 饥饿症”。 818

【算法拾光】队列|在每个树⾏中找最⼤值 + 最后⼀块⽯头的重量

本文总结两道经典算法题:一是通过层序遍历(BFS)逐层找最大值,利用队列分层处理,每层维护最大值;二是用大根堆(优先队列)模拟石头碰撞过程,每次取最大两块石头,差值重新入堆,直至剩余一块或无石。前者适用于树的层次遍历,后者适合动态获取最值场景。核心思想:队列用于分层遍历,堆用于高效取极值。

wuqingyigexiaoba的博客 194

Chrome红页误报?手把手教你通过Search Console快速解除安全警告

本文详细解析了Chrome红页误报的常见原因及解决方案,重点介绍如何通过Google Search Console快速解除安全警告。内容涵盖误报诊断、技术修复、申诉技巧及长效预防措施,帮助网站管理员有效应对Google Safe Browsing的误判问题,确保网站安全状态快速恢复。

a1b2c的博客 1001

把 KV Cache 压到 FP4:BitDecoding-FP4 的算法-系统协同设计复盘

长上下文推理卡在KV Cache的显存与带宽上,而现有FP4方案要么精度崩、要么速度废。这篇文章复盘BitDecoding-FP4如何用三个算法补丁加一个系统重构,在B300上把服务吞吐提升2.4–3.0倍,精度几乎不掉。

helloagent的博客 318

Open5GS实战:从零搭建树莓派5G核心网并与商用终端联调

本文详细介绍了如何在树莓派5上从零开始搭建基于Open5GS的5G核心网,涵盖系统准备、依赖安装、源码编译、关键配置修改及服务启动全流程。通过整合SDR基站与商用终端进行端到端联调,为开发者和研究者提供了一个低成本、可实践的5G网络实验平台,助力物联网与边缘计算应用的本地化测试与验证。

weixin_29194255的博客 162

相机标定-张正友棋盘格标定法

(X,Y,Z)为在世界坐标系下一点的物理坐标 ( u , v ) 为该点对应的在像素坐标系下的像素坐标 引入齐次坐标的原因:引入齐次坐标的目的是为了升维,将坐标从二维坐标变为三维坐标。 2.相机成像过程 相机成像系统中,共包含四个坐标系:世界坐标系、相机坐标系、图像坐标系、像素坐标系。 (1)世界坐标系(,,) 描述目标在真实世界中的位置引入的参考坐标系。 (2)相机坐标系(,,) ...

qq_41275123的博客 1万+

【算法拾光】链表 哈希表|K个一组翻转链表 + 两数之和

K个一组翻转链表需分段反转,借助哑节点与前驱指针维护连接,避免断链;两数之和利用哈希表实现O(1)查找,空间换时间,高效定位目标组合。前者考察链表指针操作,后者聚焦映射关系构建,均为面试高频题。

wuqingyigexiaoba的博客 337

零基础实战:云开发家政维修小程序搭建指南

欢迎来到这本关于构建家政维修小程序的电子书!在当今数字化的浪潮中,线上服务已成为我们生活中不可或缺的一部分。从点餐购物到居家维修,人们越来越习惯通过指尖轻点来解决各种需求。正是基于这样的背景,我们希望通过这个系列,手把手地带领大家从零开始,搭建一个功能完备、体验流畅的家政维修平台。这不仅仅是一本技术教程,更是一次将想法变为现实的探索之旅。我们将深入浅出地讲解如何运用云开发等现代技术,实现从用户下单、支付,到客服派单、工人接单,再到订单完成和权限管理等一系列核心业务流程。

低代码布道师的博客 3434

排序算法深度笔记(下):非比较类排序的线性突破与工程边界

排序算法时间复杂度额外空间复杂度稳定性是否基于比较选择排序O(N^2)(O(1))不稳定是冒泡排序O(N^2)(O(1))稳定是插入排序O(N^2)(O(1))稳定是归并排序O(N\log N)(O(N))稳定是快速排序O(N\log N)O(log N)不稳定是堆排序O(N\log N)O(1)不稳定是计数排序(O(N + M))O(M))稳定否基数排序O(N)稳定否桶排序平均O(N) / 最坏O(N^2)

2401_87069538的博客 615

32位快速加法器设计

32位快速加法器设计

好好学习,天天向上 3292

康耐视visionpro-CogCreateSegmentAvgSegsTool操作工具详细说明

康耐视visionpro-CogCreateSegmentAvgSegsTool操作工具详细说明

视觉人机器视觉的博客 1483

优选算法:分治-归并排序

交易逆序对的总数

Wangshijie1015的博客 151

如何用libwebsocket库在stm32F407上实现websocket客户端?

🏆本文收录于 《全栈 Bug 调优(实战版)》 专栏。专栏聚焦真实项目中的各类疑难 Bug,从成因剖析 → 排查路径 → 解决方案 → 预防优化全链路拆解,形成一套可复用、可沉淀的实战知识体系。无论你是初入职场的开发者,还是负责复杂项目的资深工程师,都可以在这里构建一套属于自己的「问题诊断与性能调优」方法论,助你稳步进阶、放大技术价值 。

**My Coding Family** 1137

408 算法 06|堆与 Top-K:最小的 10 个数,到底该用数组还是堆

2022(42) 找 100 万个数里最小的 10 个,官方给了"10 元素有序数组"和"大根堆"两种答案。我把两种都实现并数了比较次数:k=10 时两者几乎打平,k=10000 时堆少 111 倍。另附 2021(42) 计数排序的比较次数与稳定性实测。

m0_56226131的博客 451

老版QQ原生表情gif动态

这是老版的QQ表情,不过全是动态的的,Windows照片查看器是不会动的,可以右击用浏览器打开查看,再说一遍是老版的,只是分享给宁愿用老版也要动态的人。

复分析 阿尔福斯(Ahlfors) 第三版答案 英文

答案分为十章,有部分习题没有答案,全英文复分析阿尔福斯(Ahlfors) 第三版答案

牛客周赛165C题(前后缀性质,递推,排序,枚举)

题目大意:给定长度为 `n` 的数组,选择 `k ∈ [1,n]`,将前 `k` 个元素整体翻转,后面的元素不变。要求操作后的整个数组非递减,统计满足条件的 `k` 的数量。题目链接:https://ac.nowcoder.com/acm/contest/141721/C。时间复杂度:O(nlogn)

2501_94316951的博客 175

VC中sqlite的使用

在VC中对sqlite的使用,实现数据库大批数据的快速插入,读取数据库中的数据在ListCtrl中显示

算法64,字母异位词分组

代码核心思路是:互为字母异位词的字符串,在对其字符进行排序后,一定会得到相同的字符串。因此,可以将排序后的字符串作为哈希表的键(Key)来进行分组。

sheshibenniaodan的博客 219

android 蓝牙通讯demo

android 下蓝牙通讯demo,实现了发现蓝牙,使能蓝牙,通过蓝牙发送消息。

408 算法 04|主元素与三元组最小距离:两种“扫描时做决定“的思路

2013(41) 的主元素用摩尔投票一遍选出候选,但真正的考点是"选完必须验票";2020(41) 的三元组最小距离,三指针每次只挪最小的那个,靠的是 D=2×(max−min) 这个恒等式。两套完整 C 代码加真实输出,附官方评分档位表。

m0_56226131的博客 444

【MPC 算法】Python 实战:模型预测控制(MPC)算法原理与代码实现

本文介绍模型预测控制(MPC)原理与Python实现。MPC通过滚动优化预测未来状态,结合系统模型、目标函数与约束条件,动态生成最优控制序列。以一阶温度系统为例,使用CVXPY构建凸优化问题,实现加热功率的智能调控。代码包含模型预测、目标函数设计(含温度跟踪与控制平滑惩罚)、输入约束及滚动更新机制。仿真结果表明,MPC能快速稳定温度至目标值60℃,且控制动作平滑合理,体现其在复杂系统中的优越性。

赖德发的博客 67

算法61,判定是否为字符重排

两字符串能互为重排 ⇔。

sheshibenniaodan的博客 226

【前缀和+复合离散化】题解:双色矩形(two)_数据结构_算法优化_预处理_二维前缀和_STL_C++算法竞赛

给定一个 $n \times n$ 的黑白棋盘,初始按 $i+j$ 奇偶染色,经 $k$ 个格子颜色翻转后,求子矩形中黑白格子数差的绝对值最大值。由于 $n$ 可达 $10^9$,但 $k \leq 50$,可离散化关键点。枚举所有由修改点构成的矩形及其四边向外扩展(共 $16$ 种扩展方式),利用二维前缀和快速计算变化量,结合奇偶性判断多出的格子颜色,最终取最大绝对差。时间复杂度为 $O(k^4)$,在 $k \leq 50$ 下可行。

YLCHUP的博客 213

算法62&&63,存在重复元素

判断整数数组中是否有任意元素出现至少两次。:判断数组中是否存在两个。,且索引差值的绝对值。

sheshibenniaodan的博客 105

商业综合体公摊电费分摊算法:512 商户双层权重与尾差实测

公摊电费分摊算法是指把公共回路电量按公示权重拆到每户的规则化计算过程,核心不是权重多精巧,而是每张账单都能被租户手算复现。某珠三角商业综合体 512 户商户实测:固定费与变动费按 4:6 加权,尾差用余数递补压到 0.01 千瓦时精度,24 个月 12288 张账单重算差异为零,月度争议从 31 起降到 2 起。适用边界:商铺与租户已独立装表、公共回路已分项计量的商业综合体适用;末端未装表、靠总表减合计倒推公摊的场景不适用,先把表补齐再谈分摊。

185

考拉兹猜想 - 7

本文通过引入虚数单位重构Collatz猜想的迭代逻辑,指出偶数减半与奇数“乘3加1”的过程在虚数坐标下等价于归结操作。关键在于:乘3加1使奇数跃升至偶数,随后持续减半,而虚数单位代表“无限减1”,实现无限向有限的转化。无论初始值如何,减半始终主导,最终归结为4-2-1循环。该视角消解了正向思维中的速度竞争疑虑,证明归结是必然结果。

杨铸人的博客 92

2026华为OD机试真题 10月14日 新系统 【分组字符串排序】

华为OD机试真题 新系统【分组字符串排序】,提供C语言、Java、C++、Python、Go、JavaScript源码实现,并提供完整思路讲解,以及在线OJ模拟做题。华为OD机试真题 【分组字符串排序】多语言题解。

qq_45776114的博客 458

贪心题目:可以到达的最远建筑

为了到达尽可能远的建筑物,在将砖块替换成梯子时,应将高度差最大位置的砖块替换成梯子,此时剩余的砖块数量最多,如果将高度差较小位置的砖块替换成梯子,则剩余的砖块数量较少,能到达的最远建筑物下标不可能更大。开始向后面的建筑物移动,移动过程中维护优先队列存储正高度差,优先队列的队首元素是最大元素,每次遇到正高度差时优先考虑使用砖块,如果砖块数量不够则使用梯子替换最大高度差位置的砖块,直到砖块和梯子都使用完毕时,到达的建筑物即为最远建筑物。如果以最佳方式使用给定的梯子和砖块,返回可以到达的最远建筑物的下标。

stormsunshine的博客 299

强化学习第十一课:完整 Q-Learning 算法

表示有:\[ 5 \]0 1 2 3 40:向左1:向右目标位置:\[ 4 \]

qq_56657939的博客 328
上一篇: MS3251 Analytics Using SAS
下一篇: Software Vulnerabilities Exercise 2
wx____codinghelp
博客等级 码龄2年 771粉丝 115原创
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付:10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值