6CCS3AIN Pacman MDP Solver

AI 应用开发 × 算法面试题:业务场景实战 12 题【TopK、LRU、RRF 融合、相似度、限流算法,含手撕代码与底层答案】 本文聚焦AI应用岗算法面试新趋势,剖析12道融合业务场景的高频题:从海量日志中挖掘慢查询(分治+小顶堆)、亿级去重(布隆过滤器)、向量与关键词融合排序(RRF)、语义缓存设计(带租户隔离与TTL的LRU)到限流策略(令牌桶+滑动窗口),强调“场景复述→算法选型→手撕代码→生产落地”四步答题框架。突出考察点已从纯算法转向工程权衡、业务理解与系统设计能力,如误判容忍度、实时性与内存预算的平衡,助力开发者应对2026年真实技术挑战。 阅读详情

Coursework

1 Introduction

This coursework exercise asks you to write code to create an MDP-solver to work in the Pacman

environment that we used for the practical exercises.

Read all these instructions before starting.

This exercise will be assessed.

2 Getting started

You should download the file pacman-cw.zip from KEATS. This contains a familiar set of files that

implement Pacman, and version 6 of api.py which defines the observability of the environment that

you will have to deal with, and the same non-deterministic motion model that the practicals used.

Version 6 of api.py, further extends what Pacman can know about the world. In addition to

knowing the location of all the objects in the world (walls, food, capsules, ghosts), Pacman can now

see what state the ghosts are in, and so can decide whether they have to be avoided or not.

3 What you need to do

3.1 Write code

This coursework requires you to write code to control Pacman and win games using an MDP-solver.

For each move, you will need to have the model of Pacman’s world, which consists of all the elements

of a Markov Decision Process, namely:

• A finite set of states S;

• A finite set of actions A;

• A state-transition function P(s

0

|s, a);

• A reward function R;

• A discount factor γ ∈ [0, 1];

Following this you can then compute the action to take, either via Value Iteration, Policy Iteration or

Modified Policy Iteration. It is expected that you will correctly implement such a solver and optimize

the choice of the parameters. There is a (rather familiar) skeleton piece of code to take as your

starting point in the file mdpAgents.py. This code defines the class MDPAgent.

There are two main aims for your code:

1 Mallmann-Trenn / McBurney / 6ccs3ain-cw(a) Win hard in smallGrid

(b) Win hard in mediumClassic

To win games, Pacman has to be able to eat all the food. In this coursework, for these objectives,

“winning” just means getting the environment to report a win. Score is irrelevant.

3.1.1 Getting Excellence points

There is a difference between winning a lot and winning well. This is why completing aim (a) and

(b) from previous section allows you to collect up to 80 points in the Coursework. The remaining

20 points are obtained by having a high Excellence Score Difference in the mediumClassic layout,

a metric that directly comes from having a high average winning score. This can be done through

different strategies, for example through chasing eatable ghosts.

A couple of things to be noted. Let W be the set of games won, i.e., |W| ∈ [0, 25]. For any won

game i ∈ W define sw(i) to be the score obtained in game/run i.

• ∆Se in the marksheet is the Excellence Score Difference. You can use the following formula

to calculate it when you test your code and compare the result against the values in Table 3

∆Se =

X

i∈W

(sw(i) − 1500) (1)

Losses count as 0 score and are not considered. If ∆Se < 0, we set it to 0 (you cannot have

a negative excellence score difference).

• Because smallGrid does not have room for score improvement, we will only look at the

mediumClassic layout

• You can still get excellence points if your code performs poorly in the number of wins; marking

points are assigned independently in the two sections

• Note however that marking points are assigned such that it is not convenient for you to directly

aim for a higher average winning score without securing previous sections’s aims (a) and (b)

first

• We will use the same runs in mediumClassic to derive the marks for Table 2 and Table 3.

3.2 Things to bear in mind

Some things that you may find helpful:

(a) We will evaluate whether your code can win games in smallGrid by running:

python pacman.py -q -n 25 -p MDPAgent -l smallGrid

-l is shorthand for -layout. -p is shorthand for -pacman. -q runs the game without the

interface (making it faster).

(b) We will evaluate whether your code can win games in mediumClassic by running:

python pacman.py -q -n 25 -p MDPAgent -l mediumClassic

The -n 25 runs 25 games in a row.

2 Mallmann-Trenn / McBurney / 6ccs3ain-cw(c) The time limit for evlauation is 25 minute for mediumClassic and 5 minutes for small grid.

It will run on a high performance computer with 26 cores and 192 Gb of RAM. The time

constraints are chosen after repeated practical experience and reflect a fair time bound.

(d) When using the -n option to run multiple games, the same agent (the same instance of

MDPAgent.py) is run in all the games.

That means you might need to change the values of some of the state variables that control

Pacman’s behaviour in between games. You can do that using the final() function.

(e) There is no requirement to use any of the methods described in the practicals, though you

can use these if you wish.

(f) If you wish to use the map code I provided in MapAgent, you may do this, but you need to

include comments that explain what you used and where it came from (just as you would for

any code that you make use of but don’t write yourself).

(g) You can only use libraries that are part of a the standard Python 2.7 distribution. This ensures

that (a) everyone has access to the same libraries (since only the standard distribution is

available on the lab machines) and (b) we don’t have trouble running your code due to some

library incompatibilities.

(h) You should comment your code and have a consistent style all over the file.

3.3 Limitations

There are some limitations on what you can submit.

(a) Your code must be in Python 2.7. Code written in a language other than Python will not be

marked.

Code written in Python 3.X is unlikely to run with the clean copy of pacman-cw that we will

test it against. If is doesn’t run, you will lose marks.

Code using libraries that are not in the standard Python 2.7 distribution will not run (in

particular, NumPy is not allowed). If you choose to use such libraries and your code does not

run as a result, you will lose marks.

(b) Your code must only interact with the Pacman environment by making calls through functions

 in Version 6 of api.py. Code that finds other ways to access information about the

environment will lose marks.

The idea here is to have everyone solve the same task, and have that task explore issues with

non-deterministic actions.

(c) You are not allowed to modify any of the files in pacman-cw.zip except mdpAgents.py.

Similar to the previous point, the idea is that everyone solves the same problem — you can’t

change the problem by modifying the base code that runs the Pacman environment. Therefore,

you are not allowed to modify the api.py file.

(d) You are not allowed to copy, without credit, code that you might get from other students or

find lying around on the Internet. We will be checking.

This is the usual plagiarism statement. When you submit work to be marked, you should only

seek to get credit for work you have done yourself. When the work you are submitting is code,

3 Mallmann-Trenn / McBurney / 6ccs3ain-cwyou can use code that other people wrote, but you have to say clearly that the other person

wrote it — you do that by putting in a comment that says who wrote it. That way we can

adjust your mark to take account of the work that you didn’t do.

(e) Your code must be based on solving the Pacman environment as an MDP. If you don’t submit

a program that contains a recognisable MDP solver, you will lose marks.

(f) The only MDP solvers we will allow are the ones presented in the lecture, i.e., Value iteration,

Policy iteration and Modified policy iteration. In particular, Q-Learning is unacceptable.

(g) Your code must only use the results of the MDP solver to decide what to do. If you submit

code which makes decisions about what to do that uses other information in addition to what

the MDP-solver generates (like ad-hoc ghost avoiding code, for example), you will lose marks.

This is to ensure that your MDP-solver is the thing that can win enough games to pass the

functionality test.

4 What you have to hand in

Your submission should consist of a single ZIP file. (KEATS will be configured to only accept a

single file.) This ZIP file must include a single Python .py file (your code).

The ZIP file must be named:

cw <lastname> <firstname>.zip

so my ZIP file would be named cw mallmann-trenn frederik.zip.

Remember that we are going to evaluate your code by running your code by using variations on

python pacman.py -p MDPAgent

(see Section 5 for the exact commands we will use) and we will do this in a vanilla copy of the

pacman-cw folder, so the base class for your MDP-solving agent must be called MDPAgent.

To streamline the marking of the coursework, you must put all your code in one file, and this file

must be called mdpAgents.py,

Do not just include the whole pacman-cw folder. You should only include the one file that includes

the code you have written.

Submissions that do not follow these instructions will lose marks. That includes submissions which

are RAR files. RAR is not ZIP.

5 How your work will be marked

See cw-marksheet.pdf for more information about the marking.

There will be six components of the mark for your work:

(a) Functionality

We will test your code by running your .py file against a clean copy of pacman-cw.

As discussed above, the number of games you win determines the number of marks you get.

Since we will check it this way, you may want to reset any internal state in your agent using

4 Mallmann-Trenn / McBurney / 6ccs3ain-cwfinal() (see Section 3.2). For the excellence marks, we will look at the winning scores for

the mediumClassic layout.

Since we have a lot of coursework to mark, we will limit how long your code has to demonstrate

that it can win. We will terminate the run of the 25smallGrid games after 5 minutes, and

will terminate the run of the 25 mediumClassic games after 25 minutes. If your code has

failed to win enough games within these times, we will mark it as if it lost. Note that we will

use the -q command, which runs Pacman without the interface, to speed things up.

(b) Code not written in Python will not be marked.

(c) Code that does not run in our test setting will receive 0 marks. Regardless of the reason.

(d) We will release the random seed that we use for marking. Say the seed is 42, then you need

to do the following to verify our marking is correct:

1) fix the random seed to 42 (int, not string type) at line 541 of pacman.py. (not ’42’)

2) download a fresh copy of the new api (to avoid using files you modified yourself)

3) run python pacman.py -q -f -n 25 -p MDPAgent -l mediumClassic

4) you should get the same result as us. If not repeat step 3) again. Should the outcome be

different, then you didn’t fix the random seed correctly. Go back to 1)

A copy of the marksheet, which shows the distribution of marks across the different elements of

C++ 极致工程美学:拆解 Meta Faiss 量化器的 AVX2 向量化与有偏收缩算法 在 Meta 向量检索库 Faiss 的头文件 内部,标量量化器为反量化系数预留了两个并列枚举:无偏(UNBIASED)与有偏(BIASED)。按照经典数理统计教材的直觉惯性,无偏估计一向被奉为无损推断的金科玉律;但在 100 万条 768 维真实特征向量的检索压测中,把缩放模式切到有偏估计后,重构均方误差(MSE)稳定下降 19.60%,P99 长尾失真压低 35.05%,最大极端离群误差更是被削减了 81.46%。向量检索面对的是单次离散几何判决,没有任何大数定律能抹平单次重构方差。本文聚焦 Fais 阅读详情

相关推荐

手把手教你用GEM-2多频电磁仪找地下管道(附真实数据图)

本文详细解析了使用GEM-2多频电磁仪精准定位地下管线的全流程,包括设备调试、频率选择、现场扫描技巧及数据解读。通过真实案例展示了如何应对复杂干扰环境,提升管线检测效率与准确性,特别适合市政工程和地下管线普查。

weixin_42565652的博客 404

【算法拾光】队列|在每个树⾏中找最⼤值 + 最后⼀块⽯头的重量

本文总结两道经典算法题:一是通过层序遍历(BFS)逐层找最大值,利用队列分层处理,每层维护最大值;二是用大根堆(优先队列)模拟石头碰撞过程,每次取最大两块石头,差值重新入堆,直至剩余一块或无石。前者适用于树的层次遍历,后者适合动态获取最值场景。核心思想:队列用于分层遍历,堆用于高效取极值。

wuqingyigexiaoba的博客 194

深信服aCloud虚拟存储VS3.0实战:从数据分片到仲裁机制的全流程解析

本文深度解析了深信服aCloud虚拟存储VS3.0的核心架构与生产级实践。从数据分片、条带化写入、多副本机制到智能缓存分层,详细阐述了其如何实现高性能与高可靠性的平衡。同时,深入剖析了动态均衡、平滑扩容以及故障恢复与仲裁机制,为运维人员和技术决策者提供了在超融合基础架构(HCI)中部署和管理虚拟存储的实战指南。

net55的博客 588

把 KV Cache 压到 FP4:BitDecoding-FP4 的算法-系统协同设计复盘

长上下文推理卡在KV Cache的显存与带宽上,而现有FP4方案要么精度崩、要么速度废。这篇文章复盘BitDecoding-FP4如何用三个算法补丁加一个系统重构,在B300上把服务吞吐提升2.4–3.0倍,精度几乎不掉。

helloagent的博客 318

【Nature Communications】超高介电常数材料 Hf0.5Zr0.5O2(HZO)

复旦大学AnQuanJiang团队在《NatureCommunications》发表研究,通过原子层沉积技术制备Hf0.5Zr0.5O2(HZO)薄膜电容器,结合等离子体处理获得突破性性能:介电常数达921,储能密度584J/cm³,效率近100%。研究揭示了铁电性消失后氧空位有序排列产生超高介电常数的机制,并通过同步辐射衍射和STEM分析证实了微观结构转变。该材料在85-300K温度范围内保持稳定,经1万亿次循环后仍能维持100μC/cm²电荷密度,为高密度DRAM和CMOS器件集成提供了新方案。

荷塘阅色 2469

STM32G0实战:手把手教你用国产GD25Q16 Flash存储传感器数据(附完整驱动)

本文详细介绍了如何使用国产GD25Q16 Flash芯片与STM32G0系列MCU实现传感器数据存储的完整方案。从SPI接口配置、驱动层实现到存储系统架构设计,提供了实战代码和优化策略,帮助开发者快速构建高性价比的物联网数据存储系统。

weixin_30733003的博客 108

【算法拾光】链表 哈希表|K个一组翻转链表 + 两数之和

K个一组翻转链表需分段反转,借助哑节点与前驱指针维护连接,避免断链;两数之和利用哈希表实现O(1)查找,空间换时间,高效定位目标组合。前者考察链表指针操作,后者聚焦映射关系构建,均为面试高频题。

wuqingyigexiaoba的博客 337

读懂 ABAP CDS 注解的直接、继承与派生

本文系统梳理了ABAP CDS注解的评估逻辑,重点解析直接注解(Direct)、继承注解(Inherited)和派生注解(Derived)的差异及其在CL_DD_DDL_ANNOTATION_SERVICE中的处理方式。通过视图注解@Metadata.ignorePropagatedAnnotations可调控注解传播行为,避免下游语义影响上层视图。文章还提供了基于标准表的完整示例(包含底层视图、消费视图和ABAP报告程序),帮助开发者在实际系统中验证三层注解的叠加效果与阻断机制。关键点包括:派生注解的数据

2007 年 ~ 2025 年,深耕 SAP 技术 18 年 247

排序算法深度笔记(下):非比较类排序的线性突破与工程边界

排序算法时间复杂度额外空间复杂度稳定性是否基于比较选择排序O(N^2)(O(1))不稳定是冒泡排序O(N^2)(O(1))稳定是插入排序O(N^2)(O(1))稳定是归并排序O(N\log N)(O(N))稳定是快速排序O(N\log N)O(log N)不稳定是堆排序O(N\log N)O(1)不稳定是计数排序(O(N + M))O(M))稳定否基数排序O(N)稳定否桶排序平均O(N) / 最坏O(N^2)

2401_87069538的博客 615

[SC]SystemC在CPU/GPU验证中的应用(六)

本文提供了一个SystemC学习路线图,包含50个由浅入深的编程实例,涵盖从基础模块到复杂SoC建模的全过程。示例包括:基础模块(时钟、计数器)、通道(FIFO、握手)、TLM2.0接口、总线模型(AXI-Lite)、CPU/GPU内核实现、功耗分析、断言验证等。重点展示了SystemC在硬件验证中的关键应用:VCD波形生成、UVM-SystemC框架、Verilog DPI协同仿真、Python集成、NoC路由器和完整SoC建模(CPU+Cache+DRAM+互连)。每个示例提供完整代码和详细注释,建议分

元直的博客 1496

C# Blazor Server工业监控Web UI:实时数据推送、历史曲线、报警记录,跨浏览器访问(踩坑总结+生产可用)

坑点现象解决方案StateHasChanged()太频繁服务器CPU占用高,浏览器卡顿别100ms刷新一次,1秒刷新一次足够并发高时服务器卡多个用户同时访问,服务器响应慢用Redis做分布式缓存,或者加服务器做负载均衡断线重连后数据丢失浏览器刷新后,实时数据不更新用InfluxDB存历史数据,刷新后重新查询工业现场网络不稳定Blazor Server频繁断线用SQLite本地缓存,断网时数据不丢,联网后补发历史曲线查询慢查询过去7天的数据,要等很久。

这里只有干货:从 Modbus 通信到 MES 对接,从 YOLO 训练到工控机部署,带你搞定工业软件开发全流程。 312

优选算法:分治-归并排序

交易逆序对的总数

Wangshijie1015的博客 151

1篇1章3节:R 语言的产生与发展轨迹(更新2024/08/14)

药物的研究阶段包括靶标确定、生物学模型建立、新化学实体的发现和优化新化学实体四个关键环节。每个环节都在药物研发过程中发挥着至关重要的作用。通过科学严谨的方法和技术手段,研究人员能够在药物研发过程中不断探索和优化,从而发现和开发出具有高效、安全性的候选药物,为疾病治疗提供更多的选择和希望。

MD Code 2669

408 算法 06|堆与 Top-K:最小的 10 个数,到底该用数组还是堆

2022(42) 找 100 万个数里最小的 10 个,官方给了"10 元素有序数组"和"大根堆"两种答案。我把两种都实现并数了比较次数:k=10 时两者几乎打平,k=10000 时堆少 111 倍。另附 2021(42) 计数排序的比较次数与稳定性实测。

m0_56226131的博客 451

计算机控制技术及应用

《计算机控制技术及应用》通过总结作者的科研成果和吸收国内外的先进理论、方法和技术,以计算机控制系统的网络化、开放化、智能化和集成化发展趋势为主线索,主要介绍计算机控制系统的基本概念和计算机控制系统的典型形式,计算机控制系统的数学描述与基本分析方法,以及数字PID控制器、最少拍控制系统、达林算法、状态变量设计法等设计方法及其各种改进算法。重点突出计算机控制系统输入输出通道接口的设计方法和典型控制系统的设计案例与实现方法等。

VB摄像头监控

该VB模块可对摄像头捕获的画面进行录像等,可在创建的窗口进行控制

牛客周赛165C题(前后缀性质,递推,排序,枚举)

题目大意:给定长度为 `n` 的数组,选择 `k ∈ [1,n]`,将前 `k` 个元素整体翻转,后面的元素不变。要求操作后的整个数组非递减,统计满足条件的 `k` 的数量。题目链接:https://ac.nowcoder.com/acm/contest/141721/C。时间复杂度:O(nlogn)

2501_94316951的博客 175

IMSL CNL 7.0.0 (x86-64)Crack

IMSL CNL 7.0.0 (x86-64)Crack intel的算法库破解文件,这是C++版本的破解文件

算法64,字母异位词分组

代码核心思路是:互为字母异位词的字符串,在对其字符进行排序后,一定会得到相同的字符串。因此,可以将排序后的字符串作为哈希表的键(Key)来进行分组。

sheshibenniaodan的博客 219

win10 跳过U8环境检测的IE WebControls.rar

win10 安装u8的时候 提示IE WebControls 不通过,这个工具可以欺骗u8的检查。 跳过U8环境检测的IE WebControls。提示导入失败后,重新检测通过。

408 算法 04|主元素与三元组最小距离:两种“扫描时做决定“的思路

2013(41) 的主元素用摩尔投票一遍选出候选,但真正的考点是"选完必须验票";2020(41) 的三元组最小距离,三指针每次只挪最小的那个,靠的是 D=2×(max−min) 这个恒等式。两套完整 C 代码加真实输出,附官方评分档位表。

m0_56226131的博客 444

【MPC 算法】Python 实战:模型预测控制(MPC)算法原理与代码实现

本文介绍模型预测控制(MPC)原理与Python实现。MPC通过滚动优化预测未来状态,结合系统模型、目标函数与约束条件,动态生成最优控制序列。以一阶温度系统为例,使用CVXPY构建凸优化问题,实现加热功率的智能调控。代码包含模型预测、目标函数设计(含温度跟踪与控制平滑惩罚)、输入约束及滚动更新机制。仿真结果表明,MPC能快速稳定温度至目标值60℃,且控制动作平滑合理,体现其在复杂系统中的优越性。

赖德发的博客 67

算法61,判定是否为字符重排

两字符串能互为重排 ⇔。

sheshibenniaodan的博客 226

【前缀和+复合离散化】题解:双色矩形(two)_数据结构_算法优化_预处理_二维前缀和_STL_C++算法竞赛

给定一个 $n \times n$ 的黑白棋盘,初始按 $i+j$ 奇偶染色,经 $k$ 个格子颜色翻转后,求子矩形中黑白格子数差的绝对值最大值。由于 $n$ 可达 $10^9$,但 $k \leq 50$,可离散化关键点。枚举所有由修改点构成的矩形及其四边向外扩展(共 $16$ 种扩展方式),利用二维前缀和快速计算变化量,结合奇偶性判断多出的格子颜色,最终取最大绝对差。时间复杂度为 $O(k^4)$,在 $k \leq 50$ 下可行。

YLCHUP的博客 213

算法62&&63,存在重复元素

判断整数数组中是否有任意元素出现至少两次。:判断数组中是否存在两个。,且索引差值的绝对值。

sheshibenniaodan的博客 105

商业综合体公摊电费分摊算法:512 商户双层权重与尾差实测

公摊电费分摊算法是指把公共回路电量按公示权重拆到每户的规则化计算过程,核心不是权重多精巧,而是每张账单都能被租户手算复现。某珠三角商业综合体 512 户商户实测:固定费与变动费按 4:6 加权,尾差用余数递补压到 0.01 千瓦时精度,24 个月 12288 张账单重算差异为零,月度争议从 31 起降到 2 起。适用边界:商铺与租户已独立装表、公共回路已分项计量的商业综合体适用;末端未装表、靠总表减合计倒推公摊的场景不适用,先把表补齐再谈分摊。

185

考拉兹猜想 - 7

本文通过引入虚数单位重构Collatz猜想的迭代逻辑,指出偶数减半与奇数“乘3加1”的过程在虚数坐标下等价于归结操作。关键在于:乘3加1使奇数跃升至偶数,随后持续减半,而虚数单位代表“无限减1”,实现无限向有限的转化。无论初始值如何,减半始终主导,最终归结为4-2-1循环。该视角消解了正向思维中的速度竞争疑虑,证明归结是必然结果。

杨铸人的博客 92

2026华为OD机试真题 10月14日 新系统 【分组字符串排序】

华为OD机试真题 新系统【分组字符串排序】,提供C语言、Java、C++、Python、Go、JavaScript源码实现,并提供完整思路讲解,以及在线OJ模拟做题。华为OD机试真题 【分组字符串排序】多语言题解。

qq_45776114的博客 458

贪心题目:可以到达的最远建筑

为了到达尽可能远的建筑物,在将砖块替换成梯子时,应将高度差最大位置的砖块替换成梯子,此时剩余的砖块数量最多,如果将高度差较小位置的砖块替换成梯子,则剩余的砖块数量较少,能到达的最远建筑物下标不可能更大。开始向后面的建筑物移动,移动过程中维护优先队列存储正高度差,优先队列的队首元素是最大元素,每次遇到正高度差时优先考虑使用砖块,如果砖块数量不够则使用梯子替换最大高度差位置的砖块,直到砖块和梯子都使用完毕时,到达的建筑物即为最远建筑物。如果以最佳方式使用给定的梯子和砖块,返回可以到达的最远建筑物的下标。

stormsunshine的博客 299

强化学习第十一课:完整 Q-Learning 算法

表示有:\[ 5 \]0 1 2 3 40:向左1:向右目标位置:\[ 4 \]

qq_56657939的博客 328
上一篇: MS3251 Analytics Using SAS
下一篇: Software Vulnerabilities Exercise 2
wx____codinghelp
博客等级 码龄2年 771粉丝 115原创
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付:10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值