CS 551 Systems Programming

【信息科学与工程学】【数据科学】数据科学领域——第三篇 数学08 几何学00总篇 编号英文名称中文名称简要说明/归属1Foundations of Geometry几何基础公理化方法, Hilbert《几何基础》2Euclidean Geometry (Synthetic)欧几里得几何(综合法)《几何原本》传统3Analytic Geometry解析几何坐标方法, Descartes, Fermat4Plane Geometry平面几何三角形、圆、共点共线等经典问题5Solid Geometry立体几何三维空间中的图形与度量6Non-Euclidean Geometry非欧几何平行公设不 阅读详情

CS 551 Systems Programming, Fall 2024

Programming Project 2

In this project we are going to simulate the MapReduce framework on a single machine using

multi-process programming.

1 Introduction

In 2004, Google (the paper “MapReduce: Simplified Data Processing on Large Clusters” by J.

Dean and S. Ghemawat) introduced a general programming model for processing and generating

large data sets on a cluster of computers.

The general idea of the MapReduce model is to partition large data sets into multiple splits,

each of which is small enough to be processed on a single machine, called a worker. The data

splits will be processed in two phases: the map phase and the reduce phase. In the map phase, a

worker runs user-defined map functions to parse the input data (i.e., a split of data) into multiple

intermediate key/value pairs, which are saved into intermediate files. In the reduce phase, a

(reduce) worker runs reduce functions that are also provided by the user to merge the intermediate

files, and outputs the result to result file(s).

We now use a small data set (the first few lines of a famous poem by Robert Frost, see Figure

1) to explain to what MapReduce does.

Figure 1: A small data set to be processed by MapReduce.

To run MapReduce, we first split the dataset into small pieces. For this example, we will split

the dataset by the four lines of the poem (Figure 2).

Figure 2: Partitioning the input data set into multiple splits.

The MapReduce framework will have four workers (in our project, the four workers are four

processes that are forked by the main program. In reality, they will be four independent machines)

to work on the four splits (each worker is working on a split). These four map worker will each

run a user-defined map function to process the split. The map function will map the input into

a series of (key, value) pairs. For this example, let the map function simply count the number of

each letter (A-Z) in the data set.

Figure 3: The outputs of the map phase, which are also the inputs to the reduce phase.

The map outputs in our example are shown in Figure 3. They are also the inputs for the

reduce phase. In the reduce phase, a reduce worker runs a user-defined reduce function to merge

the intermediate results output by the map workers, and generates the final results (Figure 4).

Figure 4: The final result

2 Simulating the MapReduce with multi-process programming

2.1 The base code

Download the base code from the Brightspace. You will need to add your implementation into

this base code. The base code also contains three input data sets as examples.

2.2 The working scenario

In this project, we will use the MapReduce model to process large text files. The input will be a

file that contains many lines of text. The base code folder contains three example input data files.

We will be testing using the example input data files, or data files in similar format.

A driver program is used to accept user inputs and drive the MapReduce processing. The

main part of driver program is already implemented in main.c. You will need to complete the

mapreduce() function, which is defined in mapreduce.c and is called by the driver program.

A Makefile has already been given. Running the Makefile can give you the executable of the driver

program, which is named as “run-mapreduce”. The driver program is used in the following way:

./run-mapreduce "counter"|"finder" file_path split_num [word_to_find]

where the arguments are explained as follows.

• The first argument specifies the type of the task, it can be either the “Letter counter” or

the “Word conter” (explained later).

• The second argument “file path” is the path to the input data file.

• The third argument “split num” specifies how many splits the input data file should be

partitioned into for the map phase.

• The fourth argument is used only for the “Word finder” task. This argument specifies the

word that the user is trying to find in the input file.

The mapreduce() function will first partition the input file into N roughly equal-sized splits,

where N is determined by the split num argument of the driver program. Note that the sizes of

each splits do not need to be exactly the same, otherwise a word may be divided into two different

splits.

Then the mapreduce() forks one worker process per data split, and the worker process will

run the user-defined map function on the data split. After all the splits have been processed, the

first worker process forked will also need to run the user-defined reduce function to process all the

intermediate files output by the map phase. Figure 5 below gives an example about this process.

split 0

split 1

split 2

Driver

Program

map

worker 0

reduce

worker

map

worker 2

map

worker 3

“mr-0.itm”

“mr-1.itm”

“mr-2.itm”

“mr-3.itm”

map

worker 1

(1) partition

(2) fork

(3) userdefined

map

(5) userdefined

reduce

“mr.rst”

Input

data file

Intermediate

files

Result

file

PID=1001

PID=1002

PID=1003

PID=1004

PID=1001

split 3

Figure 5: An example of the working scenario.

2.3 The two tasks

The two tasks that can be performed by the driver program are described as follows.

The “Letter counter” task is similar to the example we showed in Section 1, which is counting

the number of occurrence of the 26 letters in the input file. The difference is the intermediate file

and the final result file should be written in the following format:

A number-of-occurrences

B number-of-occurrences

...

Y number-of-occurrences

Z number-of-occurrences

The “Word finder” task is to find the word provided by user (specified by the “word to find”

argument of the driver program) in the input file, and outputs to the result file all the lines that

contain the target word in the same order as they appear in the input file. For this task, you

should implement the word finder as a whole word match, meaning that the function should only

recognize complete words that match exactly(case-sensitive) with the specified search terms. And

if multiple specified words are found in the same line, you only need to output that line once.

2.4 Other requirements

• Besides the mapreduce() function defined in mapreduce.c, you will also need to complete the map/reduce functions of the two tasks (in usr functions.c.)

• About the interfaces listed in “user functions.h” and “mapreduce.h”:

– Do not change any function interfaces.

– Do not change or delete any fields in the structure interfaces (but you may add additional fields in the structure interface if necessary).

The above requirements allow the TA to test your implementations of worker logic and user

map/reduce functions separately. Note that violation to these requirements will result in 0

point for this project.

• Use fork() to spawn processes.

• Be careful to avoid fork bomb (check on Wikipedia if you are not familiar with it). A fork

bomb will result in 0 point for this project.

• The fd in the DATA SPLIT structure should be a file descriptor to the original input data

file.

• The intermediate file output by the first map worker process should be named as “mr-0.itm”,

the intermediate file by the second map worker process should be named as “mr-1.itm”, ...

The result file is named as “mr.rst” (already done in main.c).

• Program should not automatically delete the intermediate files once they are created. They

will be checked when grading. But your submission should not contain any intermediate

files as they should be created dynamically.

3 Submit your work

Compress the files: compress your README file, all the files in the base code folder, and

any additional files you add into a ZIP file. Name the ZIP file based on your BU email ID. For

example, if your BU email is “abc@binghamton.edu”, then the zip file should be “proj2 abc.zip”.

Submission: submit the ZIP file to Brightspace before the deadline.

3.1 Grading guidelines

(1) Prepare the ZIP file on a Linux machine. If your zip file cannot be uncompressed, 5 points

off.

(2) If the submitted ZIP file/source code files included in the ZIP file are not named as specified

above (so that it causes problems for TA’s automated grading scripts), 10 points off.

(3) If the submitted code does not compile:

1 TA will try to fix the problem (for no more than 3 minutes);

2 if (problem solved)

3 1%-10% points off (based on how complex the fix is, TA’s discretion);

4 else

5 TA may contact the student by email or schedule a demo to fix the problem;

6 if (problem solved)

7 11%-20% points off (based on how complex the fix is, TA’s discretion);

8 else

9 All points off;

So in the case that TA contacts you to fix a problem, please respond to TA’s email promptly

or show up at the demo appointment on time; otherwise the line 9 above will be effective.

(4) If the code is not working as required in this spec, the TA should take points based on the

assigned full points of the task and the actual problem.

(5) Lastly but not the least, stick to the collaboration policy stated in the syllabus: you may

discuss with your fellow students, but code should absolutely be kept private.

【信息科学与工程学】计算机科学与自动化——第二十四篇 编译器10——编译原理与词法分析02 链接时优化:全局DCE(579)、LTO(580)安全编译:CFI(581、586)、ASan(582)、UBSan(583)、MSan(584)、TSan(585)、CET(587)、PAC(588)、MTE(589)、SFI(590)、Wasm沙箱(591)侧信道与秘密:侧信道缓解(592)、常量时间(593)、秘密清理(594)内存安全:ASLR(595)、栈保护(596)、整数溢出检测(597)、除零检测(598)、空指针检测(599)、数组边界检查(600)至此编号达到600。 阅读详情

相关推荐

如何处理动态语言(Python / JS)的“隐形”调用链?

本文揭示动态语言(Python/JS)中静态AST解析导致调用链“断链”的核心问题,提出通过运行时追踪补全图谱的解决方案。基于Code-Graph-RAG的cgr trace机制,分别利用pytest插件与V8 Profiler捕获真实调用路径,将动态边合并至静态图谱,并在权重体系中提升动态边优先级。实践表明,该方法可显著提升图谱覆盖率,实现“结构+行为”双视角理解代码,为MCP与Rerank系统提供可靠支持。

xianyinsuifeng的博客 425

Web 版翻译者资料库

本项目是桌面版《翻译者资料库》的 Web 移植版,基于 Spring Boot 3.3、Thymeleaf、Bootstrap 5.3 与 MySQL 8.0 构建。它面向翻译写作提供资料、词汇、句库、文本工具、对照制作、查找、备份恢复及旧库迁移等功能,完整继承桌面版核心业务规则,支持一键启动与离线访问。

howard2005的专栏 199

SpringBoot+Vue+MySQL 高校学科竞赛平台 完整可运行版(源码+文档)

本项目基于SpringBoot+Vue+MySQL构建高校学科竞赛管理平台,覆盖学生、教师、领队教师、管理员四大角色,实现竞赛发布、报名、刷题、评分、公示等全流程闭环管理。系统架构清晰,技术主流,配套完整源码、数据库脚本、万字论文、开题报告、答辩PPT及部署文档,支持一键运行,适用于毕业设计、课程设计及院系二次开发,真正实现“开箱即用”。

招风的黑耳优快云 323

副队长BootStrap5探究(2)--按钮与按钮组

本文系统讲解Bootstrap 5按钮组件的使用与底层原理。通过CDN引入框架,掌握btn基础类及primary、success等主题色应用,学习轮廓按钮(btn-outline-*)、尺寸控制(btn-lg/sm)、块级布局(d-block)和禁用状态(disabled)等实用功能。结合源码解析,深入理解其基于CSS的圆角、过渡动画、伪类交互实现机制,实现“快速上手+原理掌握”的双重目标,助力前端技能进阶。

编程副队长 708

SpringBoot+Vue 社区医院信息平台 完整源码+全套文档 开箱即用

本文介绍一套基于SpringBoot+Vue+MySQL的社区医院信息平台,覆盖患者、医生、护士、药品管理员、管理员五大角色,实现从挂号就诊、开方注射、取药结算到数据统计的全流程闭环管理。系统采用成熟技术栈,架构清晰,权限分明,配套完整源码、数据库脚本、万字论文、答辩PPT及部署文档,可直接用于毕业设计或基层医疗信息化落地,助力高效开发与快速交付。

招风的黑耳优快云 541

Full Range && Limit Range

Full Range && Limit Range1 Full range和Limit Range定义2 为什么会需要Limit Range3 用Full range去显示limit Range图片4 YUV转RGB5 额外 1 Full range和Limit Range定义 我们知道,对于RGB和YUV的色彩空间来说,有个位深(bit-depth)的概念;位深是指记录图像每个像素的色彩通道时所使用的bit数,常见的RGB888,YUV420-8bit。以RGB888为例,其代表R G B

zjhcj1的博客 1万+

微信小程序-wxs

在微信开发者工具中右键创建.wxs后缀的文件。每个.wxs文件也是一个独立模块,同样使用导出内容。示例(文件return d;FOO: foo,bar: bar(外联定义参考官方文档)

dreams_dream的博客 380

python轮胎缺陷检测_基于深度学习的轮胎缺陷无损检测与分类技术研究

基于深度学习的轮胎缺陷无损检测与分类技术研究【摘要】:随着汽车行业快速发展及汽车保有量的持续增长,大大增加了对轮胎的需求,而轮胎质量在很大程度上决定了轮胎的使用寿命,也是在行驶过程中对人们生命财产安全的保障,同时对轮胎进行在线质量检测也便于轮胎生产企业及时发现生产线中存在的不合格产品,以此来调整机械设备甚至轮胎的生产工艺,避免造成生产材料的浪费。因此,检测轮胎质量是轮胎生产过程中必不可缺少的一部分...

weixin_39689428的博客 1428

汽车租赁系统(协同过滤算法、支付宝沙盒支付、 腾讯地图接入、WebSocket在线沟通、ECharts运营分析、三角色协同、还车费用处理)

本系统基于B/S架构,采用前后端分离设计,集成SpringBoot3、Vue2、MySQL等技术,实现用户、商家、管理员三角色协同管理。亮点包括:车型加权推荐与话题协同过滤算法提升个性化体验;支付宝沙盒支付、腾讯地图定位、WebSocket在线沟通保障流程闭环;还车费用智能计算与ECharts多维数据可视化助力运营分析,全面支持租车全生命周期管理。

paopaokaka_luck的博客 171

告别零碎资料!手把手教你搞定ASTER L1T数据的预处理全流程(附ENVI实操)

本文详细介绍了ASTER L1T数据的预处理全流程,包括数据获取、质量检查、ENVI实操步骤(辐射定标、大气校正、几何精校正)以及热红外波段深度处理。特别适合地信领域初学者和研究人员,帮助快速掌握ASTER L1T数据处理技巧,提升定量反演研究效率。

weixin_30385925的博客 357

三相电机控制入门:端电压、相电压、线电压到底怎么测?(附实测波形对比)

本文为三相电机控制初学者提供了端电压、相电压和线电压的完整测量指南。通过厘清三者的物理定义与测量方法,详细讲解了如何搭建虚拟星形点电路,并对比分析了方波、SPWM和SVPWM三种主流控制模式下关键电压波形的核心特征与差异,帮助工程师掌握正确的示波器测量与波形解读技巧。

oauth7security的博客 660

设计模式(unity游戏研发版)

当你使用图形用户界面类时通常会遇到一个问题。比如,你创建了自定义按钮类并允许客户端在按钮中注入自定义代码,这样当用户按下按钮时就会触发这些代码。观察者模式允许任何实现了订阅者接口的对象订阅发布者对象的事件通知。你可在按钮中添加订阅机制,允许客户端通过自定义订阅类注入自定义代码。在Unity中,这就像UnityEvent在Inspector中绑定多个回调。

m0_63330263的博客 135

IGV实战指南:从数据导入到Peak可视化的全流程解析

本文提供了一份IGV实战指南,详细解析了从数据导入到Peak可视化的全流程。内容包括软件安装、参考基因组加载、测序数据处理、信号调整以及如何精准定位和可视化Peak区域,并分享了导出出版级矢量图的高级技巧,帮助用户高效完成基因组数据可视化分析。

weixin_29212867的博客 150

实战项目:哈米音乐(前六章)

本文整理哈米音乐实战项目前六章核心知识点,涵盖技术选型、数据库建模、Maven聚合工程、MyBatis逆向工程、SSM架构整合、泛型反射抽取公共Dao与Service、文件流代码生成器,附核心易错点速查表,便于复习备考。

既然你都点开这里了,那可以给我点个关注吗?可以吗?可以吗? 305

离散数学 --- 图论基础 --- 图的同构,通路与回路,可达性与最短通路

同一个图(这里的图是抽象的数学定义)可以有不同的图形表示方法1.重数:两点之间的平行边的个数1.得到 n!的过程,一个图中的一个结点在另一个图中对应的结点有n种可能(黄框中定义的图来讨论),这个对应好后下一个结点有 n - 1 种可能,再下一个有n-2种,直到最后一个为1,所有可能的结果就等于 n * (n - 1) *(n - 2)*....*1 = n!2.虽然说找对应结点很困难,但不是没有规律可循的。比如1.两个相互对应的结点的度数要相同;

qq_51947882的博客 7444

CloudCompare 二次开发(17)——点云添加均匀分布的随机噪声

使用CloudCompare编程实现点云添加随机噪声。

点云侠的博客 840

【技术专题】JavaScript 网页技术 - JavaScript 流程控制

小锋老师推出《一天学会JavaScript(无废话版)》视频教程,涵盖基础语法、流程控制、函数、数组对象、DOM操作及ES6新特性。内容精炼,配套课件与源码打包下载,助力快速入门。重点讲解if/else、switch、for/while循环、for...of/in遍历及break/continue使用场景,并通过成绩判断实例演示实际应用。适合零基础学习者高效掌握核心技能。

java1234的博客 167

指纹浏览器降维打击(下):Cloudflare Turnstile 与 hCaptcha 的底层交互逻辑

本文将深度拆解:Turnstile 与 hCaptcha 的 Iframe 沙箱通信机制,动态混淆 JS 与 PoW(工作量证明)的物理破解法则,以及如何通过 C++ 底层事件合成与环境基因强绑定,实现验证码的“无感过验”。

数据知道的博客 425

基于 FastAPI+LangChain+LangGraph+Agent+Vue3+MySQL 的房产数据采集与房价预测分析平台

本文介绍一个完整的「房产数据采集+AI房价预测」全栈项目,涵盖FastAPI异步后端、LangGraph Agent智能问答、爬虫采集、机器学习预测与Vue3前端,实现从数据采集到AI交互的闭环。项目融合多技术栈,具备真实业务场景与生产级设计(如降级策略、流式输出、多模型对比),兼具简历亮点与答辩演示优势,可系统性提升工程能力,并附高频面试题解析,是理想的技术综合实践项目。

深耕开发,专注编程实战。分享源码解析、项目搭建、问题排查笔记,沉淀技术,持续更新。 252

各省电商指数数据(1990-2022).xlsx

1、资源内容地址:https://blog.csdn.net/2301_79696294/article/details/1434701112、数据特点:今年全新,手工精心整理,放心引用,数据来自权威,且标注《数据来源》,相对于其他人的控制变量数据准确很多,适合写论文做实证用 ,不会出现数据造假问题3、适用对象:大学生,本科生,研究生小白可用,容易上手!!!3、课程引用: 经济学,地理学,城市规划与城市研究,公共政策与管理,社会学,商业与管理

IQOO9【pd2171】解锁bl工具+root详细图文教程

IQOO9【pd2171】解锁bl工具+root详细图文教程1---资源内带详细的图文教程步骤2---资源内带相关解锁bl驱动与解锁工具3---按步骤操作即可解锁bl 然后按步骤获取面具root权限4---从此走向vivo机型玩机之路。安装框架 模块等等5----需要的友友建议看图文教程3遍后在操作6----建议备份重要资料后在操作。7----任何的刷写操作都有风险。友友们请谨慎操作

副队长BootStrap5探究(11)--单选框 radio

本文详解Bootstrap 5中单选框(radio)的使用与底层实现原理。通过引入Bootstrap CSS,掌握基础用法、默认选中、禁用状态及横向排列等特性。核心在于隐藏原生input,利用~兄弟选择器配合伪元素::before和::after绘制圆形外圈与实心圆点,实现统一视觉风格。重点解析了position、z-index、opacity及:checked状态控制等CSS技巧,巩固前端样式设计能力。

编程副队长 298

AI智能体在顺丰运营环节的应用.pptx

AI智能体在顺丰运营环节的应用.pptx

最新大数据毕业设计选题推荐-基于大数据的快餐营养摄入与运动消耗关联性分析与可视化-大数据-Spark-Hadoop-Bigdata

本文介绍《基于大数据的快餐营养摄入与运动消耗关联性分析与可视化》系统,采用Hadoop+Spark框架实现数据存储与计算,结合Python+Django与Java+Spring Boot双后端,通过MySQL存储数据,前端使用Vue+ElementUI+Echarts完成可视化。系统涵盖营养摄入、消费画像、运动消耗等分析功能,实现快餐订单数据与运动消耗的关联分析,为计算机专业毕业设计提供技术完整、可落地的大数据项目参考。文末可获取源码。

IT研究室的博客 279

基于C51单片机教程仿真实例源码50例,DSN后缀仿真源码文件.zip

基于C51单片机教程仿真实例源码50例,DSN后缀仿真源码文件:74LS139选通4位LED数码管动态显示 74LS595串行传送8位LED数码管动态显示PNP晶体管选通3位共阳LED数码管动态显示 74HC139选通4位LED动态显示74HC138+377选通8位LED动态显示74HC138+164选通8位共阴LED数码管动态显LCD 1602液晶显示屏显示模拟电子钟(秒时基由80C51定时器产生)1302实时时钟(1602液晶屏显示)

HTML5 History API 详解:前端路由底层原理

pushState改 URL 不刷新popstate接浏览器导航服务端必须兜底 SPA 入口前端路由的本质不是“魔法”,而是:把「URL 字符串」变成「组件渲染函数」的映射表,再让浏览器历史栈和这张表保持同步。

2603_94941239的博客 223

2026 年前端框架排行榜

前端框架是用于构建用户界面的工具集合,通过预设组件、网格系统和样式规范提升开发效率与可访问性。2026年主流框架包括:React(生态领先,适合全栈)、Next.js(SSR/SEO优化,主流选择)、Vue(学习友好,中小团队首选)、Angular(企业级稳定)、Svelte与Astro(性能突出)。趋势显示,服务端渲染、TypeScript、AI工具融合及轻量化成为关键方向。就业建议:优先学React+Next.js;追求易用选Vue;企业级项目可学Angular;追求极致性能可尝试Svelte或Astr

lxttzlove的博客 316

深入理解 ES6 Reflect:JavaScript 元编程的基石

Reflect是ES6引入的内置对象,旨在统一JavaScript底层对象操作的函数式接口。它与Proxy协同,提供一致的、可预测的行为,弥补了Object方法在失败处理、参数校验和语义一致性上的不足。通过13个静态方法,Reflect将get、set、delete等操作标准化,支持返回布尔值而非抛错,便于元编程与错误处理。其核心价值在于为代理(Proxy)提供默认行为的调用方式,确保this指向正确、避免递归,并提升代码可读性与健壮性。理解Reflect,是掌握现代JS框架(如Vue3响应式系统)原理的关

vvilkin的学习备忘 550
上一篇: COMP4134 dvanced Algorithms and Data Structures
下一篇: BEA3026 Financial Modelling
wx____codinghelp
博客等级 码龄2年 771粉丝 115原创
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付:10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值