MAST90083: Computational Statistics and Data Science

3天干完6个月的活,Claude把模型搬上AMD 一条挺刺激的消息:Anthropic 给投资人的材料里有一张幻灯片,写着它的 Claude Mythos Preview 只花了大约​,就摸清了怎么让另一个 Claude 模型在 AMD 的 MI355X 上跑起来。同一处还写了个对照数字:过去给一块新芯片做推理适配,通常要几十名工程师、​。3 天对 6 个月,这个比值太扎眼,当天中文互联网上就冒出一堆「英伟达的护城河塌了」的说法。我把能翻到的材料都翻了一遍,越看越觉得,塌掉的那一小块是真的,可塌的位置和很多人想的不一样。 阅读详情

School of Mathematics and Statistics

MAST90083: Computational Statistics and Data Science

Assignment 1

Due date: No later than 08:00am on Monday 9th September 2024

Weight: 20%

Question 1 Linear Regression

In this question, we will apply linear regression, ridge regression and Lasso to the R built-in dataset Hitters and compare their performance. Type ?Hitters to see the description of this data. Our goal is to predict the response variable salary using these three regression models. To complete this task, you’ll need to install two packages: ISLR and glmnet. (Note: When using random functions in this exercise, please set the random number generator with set.seed(10) to ensure reproducibility.)

1. Data preparation: Load the Hitters dataset from library(ISLR). Remove all those rows from Hitters dataset that have entry NA in the salary column. For a design matrix construction, include all variables in Hitters dataset except the salary and store them in variable X (matrix of covariates). Also, read the salary variable and store it in variable y (response).

2. Linear regression model: First, we consider the full model that includes all the po-tential variables and do the following tasks: (1) compute the least square estimates ˆβLS of β and the corresponding p-values manually using the sampling distribution property of the least squares estimator and check with the R built-in function lm(). From these p values, which variable (variables) can you consider significant (or meaningful) ? (2) Now we consider a smaller model using those variables that have p values smaller than 0.05 in (1). Compute the ˆβLS1 and the corresponding p-values for this smaller model. (3) Compare these two models using F-statistics to test whether the smaller model provides a better characterization of the linear relationship between X and y. First compute manually the F statistics and associated p-value and then check with the F statistics obtained from R built-in function anova(). What conclusions can you make from this comparison? Can you reject the hypothesis that the smaller model is correct?

3. Ridge regression: Generate a sequence of 100 values of λ between 105 and 10−2 and use the glmnet function. For ridge regression, we set α = 0 and do the following tasks: (1) compute estimates of the ridge regression coefficients for the 100 different λ values. Then, observe the set of coefficients for two extreme values of λ i.e. 105 and also for 10−2 . For which value of λ among these two, the ridge coefficient values are closer to zero? (2) Draw a plot of l2-norm of the ridge coefficient values (excluding the intercept’s coefficient value) against the logarithm of the λ values. From this plot, can you decide the optimal λ value between 105 and 10−2 , or is it better to use the mean square error (MSE) plot against the λ values? justify your response. (3) Use the cv.glmnet function to perform. ten fold cross validation (CV). First, we randomly pick half samples from x for all variables and also the corresponding samples from y to construct a training dataset (use set.seed(10) for random number generator). The rest of the samples are saved for the testing dataset. Using this training dataset, plot the cross validation results, and find the best λ (the one that results in smallest CV error) value and its corresponding MSERR value obtained using the testing dataset. You can use predict() function here. Now refit the ridge regression model on the full data set using the λ chosen by CV. Examine the ridge regression coefficients, are they all present, similar to the linear regression case?

4. Lasso: This time we set α = 1 and again (1) plot the cross validation results, and find the best λ value using the same training set used in ridge regression above and its corresponding MSELasso value using the same testing set used in ridge regression model above. (2) Now obtain the coefficients again using the best λ that we just selected using CV. Are all the coefficients selected again?

5. Here we consider a regularization technique that combines both the ridge regression penalty and lasso penalty, which can be written as 2/1−α||β||2 2 + α|β|1, 0 ≤ α ≤ 1 (you can see that when α = 1, it corresponds to the Lasso penalty and when α = 0, it corresponds to the ridge regression penalty).

Now, use this penalty to combine the benefits of ridge regression and Lasso by finding a best α. This still can be achieved using glmnet(). Type ?glmnet() to see the details of each argument in this function. Let α ∈ [0, 1] be a set of 11 different values obtained using seq(0,1,by=0.1). Then, (1) for each of this αi , i=1,...,11. apply the ten fold cross-validation to select the best λ values and compute the associated MSEi using the same testing set used in ridge regression and Lasso above. (2) Select the best α as the one that results in the smallest MSE and estimate the regression coefficients again using the best α and the corresponding best λ that where selected. (3) Are all the coefficients selected again? Compare this results with the coefficients obtained from ridge regression in 3 and Lasso in 4, what do you observe? (Hint: You can check whether your code is correct or not by checking MSE1 = MSERR and MSE11 = MSELasso)

Question 2

In this question, we will analyze the weather stations.csv dataset, which contains daily average pressure readings (measured in inches) from 17 weather stations located in the central part of Oklahoma State, USA. The data covers the period from May 2, 2017, to September 30, 2017, covering a total of 152 days. In other words, we have 17 time series, each representing daily average pressure over 152 days in 17 different weather stations. In addition to the pressure data, we also have geographic information for these 17 weather stations stored in the file geoinfo stations.csv. We are interested in three spatial variables: NLAT (North Latitude),ELON (East Longitude) and ELEV (Elevation in meters) for each weather station in this file.

1. At weather station j, j = 1, · · · , 17, the daily average pressure data at time t, t = 1, · · · , 152 is yj,t. For each time series, we consider the following model of order p

Provide a linear matrix form. for model (1) where the parameters are presented in a p×1 vector ϕ = (ϕ1, · · · , ϕp) T and a 3 × 1 vector β = (β1, β2, β3) T , derive the expressions of least square estimators of ϕj and βj . (Note a linear matrix form. for model includes the expression for observed time series yj , design matrices and the error term in vector form.)

2. First, only consider the centered daily pressure data and don’t include the geographic variables in the model. In this case the time series can be expressed using the following model

This also known as autoregression (AR) model with order p. Now we aim to use AIC and BIC to find the best order p in model (2) for each time series j, j = 1, · · · , 17. This means, for time series j, use the R built-in function AIC() and BIC() to compute the AIC and BIC values for different order p, say p=seq(1,15,by=1) then select the best order p using AIC and BIC. Repeat these steps for all the 17 time series. Draw two plots (one for AIC and the other for BIC) and each of them should have 17 curves. Each curve represent the AIC (or BIC) values against the order p for the time series j and provide your best order p for the 17 time series. (Hint: You can use arima() function in R to fit AR(p) model and get the log-likelihood for computing AIC and BIC)

3. Now we use the same best order p ⋆ obtained from 2 for each of the 17 time series in order to perform. the following analysis. From 2, we can get 17 pairs of coefficients (ϕj , ηˆj ), for the 17 time series, or 17 sets of parameters in total so that each time series is modeled as

It is aimed to have a single model for all stations time series. For this purpose, it is assumed that ˆηj depends on the spatial covariates, NLATj , ELONj and ELEVj . This assumption can be checked if it is reasonable by fitting the following linear regression model

ηˆ = γ0 + γ1NLAT + γ2ELON + γ3ELEV.

where ηˆ, NLAT, ELON, ELEV are a 17 × 1 vector and ηˆ obtained from 2. Perform. a t-test to determine whether all three spatial covariates should be included. Then, remove any insignificant covariate and use the F-test to assess whether the smaller model is better. Compare the results of the t-test and F-test. What conclusions can you make ?

如何处理动态语言(Python / JS)的“隐形”调用链? 本文揭示动态语言(Python/JS)中静态AST解析导致调用链“断链”的核心问题,提出通过运行时追踪补全图谱的解决方案。基于Code-Graph-RAG的cgr trace机制,分别利用pytest插件与V8 Profiler捕获真实调用路径,将动态边合并至静态图谱,并在权重体系中提升动态边优先级。实践表明,该方法可显著提升图谱覆盖率,实现“结构+行为”双视角理解代码,为MCP与Rerank系统提供可靠支持。 阅读详情

相关推荐

Playwright: Pytest vs Library vs LibraryAsync:三者如何选?

Python中Playwright的3个Target:Pytest(推荐用于E2E自动化测试,支持报告、并行、重试)、Library(同步脚本,适合爬虫与简单任务,需手动管理浏览器)、LibraryAsync(异步脚本,适用于高并发批量操作)。选择依据:测试项目选Pytest,轻量脚本选Library,高并发场景选LibraryAsync。

SmallBull的博客 229

MAST90083: Computational Statistics and Data Science 1R

555。

qq__99515681的博客 1053

QT实现HTTP文件(夹)下载的完整流程与实战技巧

本文详细介绍了使用QT框架实现HTTP文件和文件夹下载的完整流程与实战技巧。通过QNetworkAccessManager等核心类,讲解了从单文件下载、大文件处理、进度显示、暂停/取消功能,到复杂的文件夹下载策略。文章重点剖析了断点续传、错误处理与网络健壮性等关键问题,为C++桌面应用集成稳定高效的下载功能提供了专业指导。

weixin_29239279的博客 280

本地化部署 DeepSeek,构建单细胞与空间转录组 AI 分析平台

本地化部署 DeepSeek,构建单细胞与空间转录组 AI 分析平台

weixin_53637133的博客 187

红外热成像仪原理及应用

目录什么是红外热成像背景介绍热成像技术原理热成像分类红外热成像和夜视仪之间的区别红外热像仪夜视仪红外照明相机对比红外热像仪都能看透什么如何选择制冷型和非制冷型红外热像仪制冷型红外热像仪非制冷型红外热像仪实例对比成像速度空间分辨率灵敏度光谱滤波同步何如选择合适的红外热像仪像素测温范围和被测物温度分辨率空间分辨率温度稳定性热像仪的距离系数比常见热成像仪关键参数量程视场角 (FOV)红外分辨率热灵敏度(NETD)焦距光谱范围精度内置优化分析NTED(热灵敏度)什么是NETD如何测量NETD?NETD如何影响测量?

zhy295006359的博客 2万+

workbench 流固耦合_基于ANSYS 2019R1全解一款双吸泵的双向流固耦合方法

一、导读对于旋转机械来说,传统设计从理论计算到手工木模图,再到模型泵的加工制造,最后进行相关性能试验。当性能试验与预期效果差距较大的时候还需要修改水力模型。这种传统的设计不仅设计周期长,而且成本也高,在没有成熟的水力模型参考的情况下做设计风险也较大。而日益强大的CFD技术为旋转机械行业的发展提供了非常有力的工具。大量工程实践证明像CFX、Fluent以及Pumplinx等流产分析软件具有较高的计算...

weixin_39604280的博客 4762

【第 009 讲】全面掌握 Python 注释:注释的定义与作用 | 单行注释 | 三引号与多行注释 | 注释使用的最佳实践

本文系统讲解 Python 注释的完整知识体系:从注释的定义与作用讲起,梳理单行注释的书写位置、临时屏蔽用法与 PEP 8 排版规范,剖析注释被忽略的底层机制;进而说明三引号字符串的本质与嵌套问题;最后给出注释使用的三条实践原则,帮助读者写好可读性高的注释。

大模型全栈工程师 · Thanks_ks 1175

AgentScope Java 2.0 最佳实践:9 个完整示例手把手带你开发 Java‑Agent

本文基于9 个完整示例,深入剖析从基础对话(BasicChatExample)、工具调用(@Tool 注解)、结构化输出(POJO Schema)、ToolGroup 自主管理、MCP 外部工具集成、Hook 生命周期监控、Spring Boot SSE 流式 Web、JsonSession 会话持久化到中断恢复的全链路实战教程。

program哲学 1685

AI全栈实战 | 3.9-01 爬虫入门:requests 加 BeautifulSoup 三分钟跑通,但 90% 的人栽在这 4 个“礼貌“问题上

⚠️:本文内容仅用于。请遵守目标站点的服务条款与相关法律法规,不得对目标服务造成负担,不得采集、传播个人隐私数据。技术是中性的,边界在你手里。爬虫的入门技术(requests 发请求 + BeautifulSoup 解析)三分钟就能学会。真正区分"会用爬虫"和"会做爬虫的人"的,是这篇我们跑通"请求→解析→存储"的最小闭环,然后重点把"合规与礼貌"这根弦给你绷上。

源码获取 | 运行部署 | 定制二开修改,如果回复不及时请耐心等待,请见谅。 442

基于物联网技术设计的微型消防小车控制系统(STM32+ESP8266局域网)(181)

基于物联网技术的小微型消防车控制系统。通过远程控制功能,用户能够灵活操作消防车的行驶,方便快速抵达火灾现场。环境感知功能实时采集烟雾浓度、温湿度信息,为灭火决策提供科学依据。火源定位功能采用自动和手动两种模式,准确迅速地定位火源,并通过报警提示用户。灭火功能则根据火源定位结果,自动或手动启动干粉灭火器进行高效灭火。这些功能的结合实现了对小型火灾的全面应对,提升了消防效率与安全性,减少了火灾带来的损失。通过本项目的设计与实现,为消防领域提供了一种创新且实用的解决方案。

3084

学Simulink——基于故障诊断与容错控制的场景实例:并网逆变器在短路故障下的动态响应及恢复策略

本文详细介绍了使用Simulink进行并网逆变器短路故障模拟与保护策略设计的方法。通过构建高精度逆变器模型,模拟IGBT击穿和桥臂直通故障,分析动态响应特性。重点设计了基于Stateflow的微秒级快速保护逻辑,包括过流检测、PWM闭锁和故障报警。同时提出完整的恢复策略流程,涵盖放电、去磁、自检和软启动等关键步骤。仿真结果表明,该方法能在200μs内有效限制短路电流,保护器件安全,并实现系统可靠恢复。文章还探讨了硬件保护协同、多级保护和预测保护等高级策略,为新能源系统的安全运行提供了重要参考。

amy_mhd的博客 144

Python 的 yield from 到底转发了什么:拆开 send、throw 和那个 return 值

很多人把yield from记成「省一层 for 循环的语法糖」。写和写,结果看着一样,于是就当它只是少敲几个字。这个理解会在三个地方翻车:调用方.send()的值送不进去、子生成器return的值拿不到、异常和close()传不下去。yield from真正做的事是,外层生成器退成一个透明通道。本文所有输出来自本机 CPython 3.13.14 实测。yield from不是循环,是委托——nextsendthrowclose四个方向的流量全部直通子生成器,并且子生成器return。

小小张说故事的博客 163

开源数字人、虚拟直播部署教程

对,你无看错,是欧美的。在直播带货案例里,我们把模型导入到ue4,我们给模型在ue4里预设了四个动作(站着、站着说话、坐着、坐着说话),三个表情(平静、开发、愤怒),三个镜头(全局、近矩说话、看显示器播放商品展示)。我们只需要把目标主播的直播给录制下来,提取样本数据,然后训练一个从粉丝的不同刺激的组合,到主播的不同响应方式的多元逻辑回归的数学模型参数即可。

南七小僧的学海无涯 4564

从一段视频到可漫游的 4D 有声场景:思路拆解与最小复现手记

只需一段手机视频,你就能生成可"走进去"的 4D 有声场景:转头时狗叫声方向随之变化,走近时声音更清晰。本文拆解 Video-to-4DAVScene 的核心思路,手把手带你用几十行 Python 搭出最小管线,把视频中的运动物体变成带空间位置的声源,并附上三个真实踩坑的排障经验,代码可直接跑。🎧

helloagent的博客 246

RT-DETR目标检测模型性能评价指标详解,涉及混淆矩阵、F1-Score、IoU、mAP、参数量、计算量等,一文打尽所有评价指标

本文主要讲解的是计算机视觉网络模型的性能评价指标,这是每一篇论文中都需要展示的内容。熟悉每一个评价指标也能够帮助我们在训练和改进模型过程中,更好的观察模型的变化效果。

Limiiiing的博客 3366

SAP B1中文培訓教材_SDK

公司内部培训资料,对开发工具 ,DIAPI, UIAPI,Add-ON有详细解说

【每天一个 Python 小案例30】Python 基础知识综合小项目:我的工具盒

作为系列收官,本篇把前 29 篇的知识点合成一个能直接用的命令行工具"我的工具盒":计算器(用 ast 解析表达式,不碰 eval)、单位换算与温度转换(一张换算表 + 特例分支)、记账与分类统计(Decimal + JSON 持久化)、以及用"字典当命令表"把菜单和业务逻辑解耦的主循环。文中对比了 eval 的安全隐患与 ast 白名单的做法,解释了 lambda 闭包在字典里的写法与晚绑定陷阱,并在最后给出整个系列的知识地图,把变量、数据结构、控制流、函数、类、异常、文件、标准库八块内容对应到具体篇目。

xinjiyaya的博客 432

Outlook Express 6 (OE6) for Win7/Win8/Win10

英文版 Outlook Express6.0, 兼容Win7/Win10/WinXP,解压就可用。

双路径优化+双容差校准

默认使用向量化路径处理常规数据,当检测到内存非连续或需要极致短路性能时,自动切换至 Numba JIT 路径。“当 CI 宽度中位数为 $W$ 时,实际判定容差为 $10^{-12} + 10^{-9} \times W$"针对表征谱算子 2 的实际输出量纲(例如 $10^{-3}$ 至 $10^3$),调整。准确标记了哪些数组被复制。非短路、静默拷贝风险、量纲耦合容差),本方案重构了核心逻辑。,并完善审计元数据以记录内存拷贝状态。, 确保在不同量纲下判定标准的一致性。,否则需排查上游数据生成逻辑。

2301_77211362的博客 202

IMSL CNL 7.0.0 (x86-64)Crack

IMSL CNL 7.0.0 (x86-64)Crack intel的算法库破解文件,这是C++版本的破解文件

lwip-2.0.3

lwip是瑞典计算机科学院(SICS)的Adam Dunkels 开发的一个小型开源的TCP/IP协议栈。实现的重点是在保持TCP协议主要功能的基础上减少对RAM 的占用。

yolo 和 deft 算法的区别

你应该是打错了:是,不是 defrat;

曾大强博客 275

蓝桥杯·全国青少年信息素养大赛·电子学会Python/C++考级 备考:莫队算法离线区间查询详解

备赛进行时:第十八届蓝桥杯软件赛报名已于 2026-10-08 10:00 开启、12-18 14:00 截止,省赛定在 2027-04-10(C/C++·Python 程序设计 9:00–13:00);中国电子学会 2026 年 12 月软件编程(C/C++ 一级至十级、Python 一级至六级)考级报名 9/23–12/2 16:00、统考 12-12;全国青少年信息素养大赛 2026—2027 赛季算法应用主题赛也已进入复盘练兵窗口。

IT_Scratch的博客 184

两张表合并之后,行数悄悄多了一倍

pandas merge 行数异常膨胀源于右表键不唯一,导致数据静默复制。实测显示,未聚合的右表(756行)与左表合并后行数增至504行,合计膨胀3.5倍,但无报错。validate 参数仅在检查键唯一性时有效,one_to_one 和 many_to_one 可拦截问题,但需结合 duplicated() 前置检查。正确做法为先按键聚合右表,再合并,避免重复。建议合并前后核对行数、合计,并使用 indicator=True 排查匹配情况,确保数据一致性。

patrickstar231的博客 252

我用 200 行代码写了个“代码屎山检测器“,扫完公司的老项目后我沉默了

这个工具不能帮你把屎山铲平,但至少能让你知道自己面对的是多大的山,也能在代码评审时用数据说话——“这个 PR 把圈复杂度从 12 推到了 18,能不能拆一下?毕竟,不能量化的东西,就没法改进。

xixiaoyunya的博客 277

智慧农业实战:育秧车间秧盘计数与苗情分级的Python实现

首先,建立统一的数据结构,保存每盘秧苗的编号、批次、覆盖度、叶色得分和播种天数。A = "壮苗"B = "合格苗"C = "弱苗"@dataclassdays: int。

荆棘鸟智能的博客 138

Java Arrays.asList 常见坑详解|UnsupportedOperationException、数组联动、基本类型 bug

解析 Arrays.asList 常见开发陷阱,演示 UnsupportedOperationException、底层数组引用共享、基本类型数组问题,区分 Arrays 内部 ArrayList 与 java.util.ArrayList,给出数组转 List 的各种场景正确代码。

weixin_42617033的博客 511

【每天一个 Python 小案例29】Python 实现商品购物车

本篇实现一个购物车:商品目录、加入商品、修改数量、移除商品、查看小计、计算满减与 VIP 折扣、结算扣库存。核心设计是"目录与购物车分离"——目录存商品主数据,购物车只存 {商品编号: 数量};金额一律用 Decimal 并做 ROUND_HALF_UP 舍入。折扣部分讲清阶梯满减的匹配顺序(阈值从高到低、命中即停)与叠加顺序(先满减、后打折),并用一次完整结算演示 706 元如何变成 613.70 元。

xinjiyaya的博客 220

Python的字典

Python字典是存储键值对的数据结构,通过键(key)快速查找对应的值(value)。可使用{}定义,支持增删改查操作:通过dict[key]查询或修改值,get()方法安全获取并设置默认值,del dict[key]删除键值对,keys()、values()和items()用于遍历。items()结合双变量循环可高效遍历所有键值对,提升代码可读性与灵活性。

qq_43010954的博客 199

黄仁勋走了,护城河留下了什么

11 月 12 日,英伟达董事会批准黄仁勋辞去总裁、CEO 和董事职务,即刻生效,CFO 克雷斯出任临时 CEO;股价开盘一度跌 14.35%,收盘只跌 4.34%,市值单日蒸发 1820 亿美元;本文把「创始人走了护城河就没了」这个流行说法拆开——CUDA 还在,但 CUDA 正在被 Claude 三天凿穿,真正决定下一段路的是三个可以核对的东西:治理结构、决策机制、以及下一个百亿美元由谁拍板。

领码SPARK - 以无代码之星火,燎原数字之未来! 188
上一篇: comp10002 Foundations of Algorithms
下一篇: COMP3506/7505 Algorithms and Data Structures
wx__codinghelp
博客等级 码龄2年 514粉丝 61原创
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付:10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值