Kaggle’s WordPress Challenge: The Like Graph

印尼稻米收成与粮食安全数据集2026年 本数据集收录印度尼西亚2026年水稻收成与粮食安全遥测数据,包含10000条稻米农业收成产量与粮食安全监测记录,围绕印尼国家粮食安全体系中的核心作物水稻展开。稻米是印尼逾两亿人口的主粮,其收成波动直接影响国家粮食安全与进口政策,本数据集将种植区域、收成产量、气候与土壤相关遥测指标整合为结构化表格,可支撑粮食产量预测、收成影响因素分析、区域粮食安全评估与农业政策模拟等研究任务。典型应用场景包括:农业时间序列产量预测模型构建、气候因子与水稻收成的关联分析、省份与地区层面的粮食自给率测算、粮食安全风险预警指标设计,也可作为农业经济、粮食政策与数据科学交叉课程的教学素材。数据为2026年最新版,规模约1.2MB、单CSV结构规整,字段命名清晰,适合Python、Pandas、Scikit-learn等主流工具链直接使用,便于开展探索性分析、可视化与机器学习全流程练习。重要说明:本数据为遥测整理数据,分析结论以数据集文档口径为准,农业决策请结合官方统计综合研判。许可:CC0公有领域,可自由使用与二次分发。 立即下载

I’d like to start this blog by discussing my first Kaggle data science competition – specifically, the “GigaOM WordPress Challenge”.   This was a competition to design a recommendation engine for WordPress blog users; i.e. predict which blog posts a WordPress user would ‘like’ based on prior user activity and blog content.   This  post will focus on how my engine used the WordPress social graph to find candidate blogs that were not in the user’s direct ‘like history’ but were central in their ‘like network.’

My Approach

My general approach – consistent with my overkill analytics philosophy – was to abandon any notions of elegance and instead blindly throw multiple tactics at the problem.   In practical terms, this means I hastily wrote ugly Python scripts to create data features, and I used oversized RAM and CPU from an Amazon EC2 spot instance to avoid any memory or performance issues from inefficient code.   I then tossed all of the resulting features into a glm and a random forest, averaged the results, and hoped for the best.   It wasn’t subtle, but it was effective. (Full code can be found here if interested.)

The WordPress Social Graph

From my brief review of other winning entries, I believe one unique quality of my submission was its limited use of the WordPress social graph.   (Fair warning:  I may abuse the term ‘social graph,’ as it is not something I have worked with previously.)   Specifically, a user ‘liking’ a blog post creates a link (or edge) between user nodes and blog nodes, and these links construct a graph connecting users to blogs outside their current reading list:


Defining user-blog relationships by this ‘like graph’ opens up a host of available tools and metrics used for social networking and other graph-related problems.

Node Distance, a.k.a. Two Degrees of Separation

The simplest of these graph metrics is the concept of node distance within graphs.  In this case, node distance is the smallest number of likes required to traverse between a particular user node and a particular blog node.   In the diagram above, for example, User A and Blog 4 have a node distance of 3, while User C and Blog 5 have a distance of 5.

The chart below breaks down likes from the last week of the final competition training data (week 5) by the node distance between the user and the liked blog within their prior ‘like graph’ (training data weeks 1-4):

As you can see, nearly 50% of all new likes are from blogs one ‘edge’ from the user – i.e., blogs the user had already liked in the prior four weeks.    These ‘like history’ blogs are a small, easily manageable population for a recommendation engine, and there are many relevant features that can be extracted based on the user’s history with the blog.   Therefore, the like history set was the primary focus of most contest submissions (including mine).

However, expanding the search for candidates one more level – to a distance of 3 edges/likes traversed – encompasses 90% of all new likes.   A ‘distance 3’ blog wold be a blog that is not in the subject’s immediate like history but that is in the history of another user who had liked at least one blog in common with the subject.   This is admittedly a large space (see below), but I think it significant that >90% of a user’s liked blogs in a given week can be found by traversing through just one common reader in the WordPress graph.   Finding the key common readers and common blogs, therefore, is a promising avenue for finding recommendation candidates that are new to the subject user.

Node Centrality, a.k.a. Finding The Common Thread

As referenced above, the main problem with using the distance 3 blogs as recommendation candidates is that the search space is far too large – most users have tens of thousands of blogs in their distance 3 sets:

As seen from the above chart, while a significant portion of users (~20%) have a manageable distance 3 blog set (1,000 to 2,000 blogs), the vast majority have tens of thousands of blogs within that distance.   (Some post-competition inspection shows that many of these larger networks are caused by a few ‘hyper-active’ users in the distance 3 paths.  Eliminating these outliers could be a reasonable way to create a more compact distance 3 search set.)

One could just ignore the volume issues and run thousands of distance 3 blog candidates per user through the recommendation engine.  However, calculating the features and training the models for this many candidate blogs would be computationally intractable (even given my inclination for overkill).  To get a manageable search space, one needs to find a basic, easily calculable feature that identifies the most probable liked blogs in the set.

The metric I used was one designed to represent node centrality, a measure of how important a node is within a social graph.  There are many sophisticated, theoretically sound ways to measure node centrality, but implementing them would have required minutes of exhaustive wikipedia reading.   Instead, I applied a highly simplified calculation designed to measure a blog node’s three-step centrality within a specific user’s social graph:

  • Step 1(a):  Calculate all three-step paths from the subject user in the graph (counting multiple likes between a user and blog and multiple possible paths);
  • Step 1(b): Aggregate the paths by the end-point blog; and
  • Step 1(c): Divide the aggregated paths by the total paths in step 1(a).
  • Step 2: ???
  • Step 3: Profit.

The metric is equivalent to the probability of reaching a blog in three steps from the subject user, assuming that at each outbound like/edge has an equal probability of being followed.   It is akin to Google’s original PageRank, except only the starting user node receives an initial ‘score’ and only three steps are allowed when traversing the graph.

I don’t know if this is correct or theoretically sound, but it worked reasonably well for me – substantially lifting the number of likes found when selecting candidates from the distance 3 graph:   

As shown above, if you examine the first 500 distance 3 blogs by this node centrality metric, you can find over 20% of all the likes in the distance 3 blog set.   If you selected 500 candidates by random sample, however, only 3% of the likes from this population would be found.   While I am certain this metric could be improved greatly by using more sophisticated centrality calculations, the algorithm above serves as a very useful first cut.

Some Very Simple Code

I’d feel remiss not putting any code in this post.   Unfortunately, there was a lot of bulky data handling code I used in this competition to get to the point where I could run the analysis above, so posting the code that produced this data would require a lot of extra files.  I’d happily send it all to anyone interested, of course, just e-mail me.

However, in the interest of providing something, below is a quick node distance algorithm in Python that I used after the fact to calculate node distances in the like graph.  This is just a basic breadth-first search implemented in Python, with the input graph represented as a dictionary with node names as keys and sets of connected node names as values:

def distance(graph, start, end):

	# return codes for cases where either the start point 
	# or end point are not in the graph at all
	if start not in graph: return -2
	if end not in graph: return -3

	# set up a marked dictionary to identify nodes already searched
	marked = dict((k, False) for k in graph)
	marked[start] = True

	# set a FIFO queue (just a python list) of (node, distance) tuples
	queue = [(start, 0)]

	# as long as the queue is full...
	while len(queue):
		node = queue.pop(0)

		# if the next candidate is a match, return the candidate's distance 
		if node[0] == end:
			return node[1]

		# otherwise, add all the nodes connected to the candidate if not already searched
		# mark them as searched (added to queue) and associate them with candidate distance + 1
		else:
			nextnodes = [nn for nn in graph.get(node[0], set()) if marked[nn] == False]
			queue.extend((nn, node[1]+1) for nn in nextnodes)
			marked.update(dict((nn, True) for nn in nextnodes))

	# if you fall through, return a code to show no connection
	return -1

Conclusion

In the end, this node centrality calculation served as a feature in my recommendation engine’s ranking and – more importantly – as a method of identifying the candidates to be fed into the engine.   I have not done the work to see how much this feature and selection method added to my score, but I know as a feature it added 2%-3% to my validation score, a larger jump than many other features.   Moreover, my brief review of the other winner’s code leads me to think this may have been a unique aspect of my entry – many of the other features I used were closely correlated elements in the other’s code.

More crucially, for actual implementation by Automattic the ‘like graph’ is a more practical avenue for a recommendation engine, and is probably what the company uses to recommend new posts.   Most of the work we did in the competition differentiated between posts from blogs the user was already reading – useful, but not a huge value-add to the WordPress user.   Finding unseen blog posts in which a user may have interest would be a more relevant and valuable tool, and finding new blogs for the user with high centrality in their social graph is a reasonable way to find them.   From my work on the competition, I believe these methods would be more promising avenues than NLP and topic-oriented methods.

The above posts covers all of my thinking in applying network concepts to the WordPress challenge problem, but I am certain  I only scratched the surface.   There are a host of other metrics that make sense to apply (such as eccentricity, closeness and betweenness).   If you work for WordPress/Automattic (and that is the only conceivable reason you made it this far), I’d be happy to discuss additional ideas, either on this blog or in person.

四分之一汽车模型主动悬架设计:在MATLAB_Simulink中实现比例积分控制、前馈扰动抑制和扰动估计,以优化乘坐舒适性和道路附着力。.rar 立即下载

相关推荐

Delphi 13.2控件之msquic4d.rar

Delphi 13.2控件之msquic4d.rar

The value of attribute "description" associated with an element type "target" must not contain the '<' character.

<br />The value of attribute "description" associated with an element type "target" must not contain the <br /> '<' character.<br /> <br /> <br />因为乱码引起的。重新把这个文件保存为UTF-8编码

Democreen 1万+

企业数字化转型AI大模型赋能智能体平台及AI场景应用规划方案(生态合作版).pptx

企业数字化转型AI大模型赋能智能体平台及AI场景应用规划方案(生态合作版).pptx

ANSYS常见问题汇总

ANSYS常见的问题 NO.0001、ESYS is not valid for line element. 原因:是因为我使用LATT的时候,把“–”的那个不小心填成了“1”。经过ANSYS的命令手册里说那是没有用的项目,但是根据我的理解,这些所谓的没有用的项目实际上都是ANSYS在为后续的版本留接口。对于LATT,实际上那个项目可能就是单元坐标系的设置。当我发现原因后,把1改成0——即使用全局直角坐标系,就没有WARNING了。当然,直接空白也没有问题。 NO.0002、使用*TREAD的时候,有的时候

ambious_的博客 2万+

智慧环保生态保护行业可信数据空间应用规划方案(技术方案版).pptx

智慧环保生态保护行业可信数据空间应用规划方案(技术方案版).pptx

气象服务行业可信数据空间应用规划方案(技术方案版).pptx

气象服务行业可信数据空间应用规划方案(技术方案版).pptx

Selenium错误提示

推荐的导入风格如下: from selenium import webdriver   然后,你可以这样访问所有的类: webdriver.Firefox webdriver.FirefoxProfile webdriver.Chrome webdriver.ChromeOptions webdriver.Ie webdriver.Opera webdriver.Phanto

沐风麟的博客 2万+

机器视觉检测实战教程 19页成品课程光源选型Halcon标定海康相机VisionPro

内容概要:机器视觉检测实战教程,19页6章成品课程PDF:视觉检测光源选型对照表+五个翻车案例、Halcon模板匹配create_shape_model七参数整定、视觉标定三类(9点/旋转/手眼)、海康工业相机C#取图黑图丢帧排坑、VisionPro QuickBuild部署License迁移C#宿主、远心镜头选型。含C#/Halcon代码与现场案例。适用人群:机器视觉工程师、自动化检测开发、企业内训。使用场景:视觉方案设计、选型、标定部署、团队培训。原创整理。

使用element过程中出现的问题及解决方法(1)

主要内容为日常使用饿了么UI过程中犯错和解决方案。ps:以后会更新

接着奏乐接着舞的博客 2615

【医疗器械制造】基于酸碱腐蚀环境的一次性乳胶手套铝合金手模失效机理分析与量产风险控制

内容概要:本文围绕一次性医用乳胶手套生产中使用的铝合金手模在酸碱环境下的失效风险展开系统性分析,重点探讨其在外表面腐蚀、清洗循环、氯介质侵入及表面粗糙度控制等方面的工程挑战。研究表明,尽管市场上已有带涂层的铝合金手模供应,但其能否在强碱、酸洗、毛刷摩擦、冷热交变和氯共存的复杂工况下长期维持Ra 1.0—2.5 μm的表面质量并实现稳定脱模,仍需经过严格的多阶段验证。常规阳极氧化+PTFE涂层方案已被实测排除,而316L不锈钢筒体在湿氯环境中出现的三个月内腐蚀失效案例提示必须重视内腔隔离设计。文章强调,采购决策应基于“小批量试制—离线加速试验—影子生产验证—条件性放行”的四阶段准入机制,结合表面参数、工艺相容性、手套成品性能与全生命周期成本进行综合评估,避免依赖供应商宣传数据。; 适合人群:从事医疗器械制造、材料工程、设备采购与工艺质量管理的技术人员及决策管理者,尤其是涉及乳胶手套生产线优化与手模选型的相关从业人员。; 使用场景及目标:①评估铝合金手模替代陶瓷手模的技术可行性与风险边界;②建立手模选型的系统验证流程,防范因表面失效导致的脱模异常、手套缺陷与非计划停机;③指导采购合同中技术条款的制定,确保供应商交付具备可验证数据支撑; 阅读建议:此资源以工程实证为导向,建议读者结合自身产线工艺参数,重点关注外表面Ra控制、内腔双密封设计、影子生产验证方法及失效链追溯体系的构建,在试制阶段严格执行多维度联合判定标准,审慎推进规模化替代。

【新能源电力系统】项目介绍 MATLAB实现基于GWO-VMD-SSA-LSTM灰狼优化算法(GWO)结合变分模态分解(VMD)、麻雀搜索算法(SSA)及长短期记忆网络(LSTM)进行辐照度-温度驱动

内容概要:本文提出了一种融合灰狼优化算法(GWO)、变分模态分解(VMD)、麻雀搜索算法(SSA)和长短期记忆网络(LSTM)的复合模型(GWO-VMD-SSA-LSTM),用于实现高精度的辐照度-温度驱动短期光伏功率预测。该模型首先通过GWO优化VMD的关键参数(模态数K和惩罚因子α),实现对非平稳光伏功率序列的高效多尺度分解,降低信号复杂性;随后利用SSA对LSTM的超参数(如隐藏层单元数、Dropout率、学习率等)进行智能寻优,提升网络训练效率与泛化能力;最后通过独立LSTM网络对各VMD模态进行预测并重构输出。模型结合了信号分解、智能优化与深度学习优势,显著提升了在多云、阴雨等复杂天气下的预测精度与稳定性,并通过MAE、RMSE、MAPE和R²等指标验证了有效性。文中还提供了完整的MATLAB实现流程,涵盖数据清洗、归一化、滑动窗口构建、参数优化与结果可视化等环节。; 适合人群:具备一定机器学习与时间序列预测基础,从事新能源发电预测、智能电网调度、电力系统自动化等相关领域的科研人员、工程师及研究生。; 使用场景及目标:①解决光伏功率受辐照度、温度等因素影响而导致的强随机性、非平稳性预测难题;②提升短期(15分钟至1小时)光伏出力预测精度,支撑电网经济调度、储能协同控制与电力现货市场申报;③为多源气象数据驱动的可再生能源预测提供可扩展的技术框架。; 阅读建议:建议结合提供的MATLAB代码逐步实践,重点关注GWO与SSA的优化机制、VMD分解效果及其与LSTM的集成方式,注意避免时间泄漏问题,推荐在实际数据上复现实验以深入理解模型性能。

ISE/Vivado调试过程中经常遇到的几种warning,以及解决办法,一些verilog使用技巧

由于最近已经也刚刚接手项目,遇到特别多的问题,所以把遇到的问题记录一下,自己学习的同时,把过程分享出来,希望对大家有一定的帮助,共同进步。下面就是ISE调试过程中经常遇到的几种warning,以及解决办法,自己整理成文档,以供参考: 1、Redeclaration of ansi port XX is not allowed “不允许重新声明ansi端口XX”。 出现的原因是在程序中声明了两......

weiyunguan8611的博客 9万+

T12_eVTOL低空经济低空公共航线网络规划设计方案_可研报告版.pptx

T12_eVTOL低空经济低空公共航线网络规划设计方案_可研报告版.pptx

【CEEMDAN-VMD-GRU】完备集合经验模态分解-变分模态分解-门控循环单元预测研究(Python代码实现)

内容概要:本文提出了一种融合完备集合经验模态分解(CEEMDAN)、变分模态分解(VMD)与门控循环单元(GRU)的复合预测模型,旨在提升对复杂非平稳时间序列的预测精度。首先利用CEEMDAN对原始信号进行自适应分解,有效缓解模态混叠问题,获得一组本征模态函数(IMFs);随后引入VMD对关键模态分量作进一步优化分解,增强特征分离能力,降低噪声干扰;最后将处理后的平滑分量分别输入GRU神经网络进行独立训练与预测,并通过加权集成策略融合各分量输出,得到最终高精度预测结果。该方法结合了多尺度信号分解与深度学习的优势,显著提升了模型在处理强噪声、非线性、非平稳序列(如电力负荷、风速、光伏发电功率等)时的泛化能力和预测稳定性,文中提供了完整的Python代码实现,便于复现与应用。; 适合人群:具备一定Python编程基础和机器学习理论知识,从事科研或工程应用的硕士、博士研究生及科研人员,尤其适用于能源系统预测、智能电网调度、可再生能源管理、信号处理与时间序列分析等相关领域的研究人员和技术开发者。; 使用场景及目标:①解决传统单一预测模型在面对复杂非平稳时间序列时存在的过拟合、收敛慢、精度低等问题;②通过多阶段信号分解降低原始序列的复杂度与不确定性,提升GRU模型的训练效率与预测鲁棒性;③为电力系统负荷预测、新能源出力预报、设备运行状态预警及市场交易策略制定等关键任务提供高精度数据支撑。; 阅读建议:建议读者结合提供的Python代码逐模块理解算法实现流程,重点关注CEEMDAN与VMD的参数配置对信号分解效果的影响,尝试在不同数据集上进行实验对比,并通过可视化手段分析各模态分量的物理意义,进一步掌握模型调优与性能评估的方法。

MCGS的Http协议驱动,支持http协议

硬件配置 参数设置 设备属性 设备命令 通讯状态 附录 #HTTP 1、 设备简介 驱动类型 该驱动为独立设备,直接添加构件即可,不需要挂载到父设备下 通讯协议 采用标准HTTP1.1协议 通讯方式 驱动构件为客户端。 2、 硬件连接 本软件在通讯之前,请保证通讯连接正确。通讯连接方式:采用标准的RJ45接口的以太网接线(对调网线或使用HUB交换)。 3、 设备通讯参数 本设备为独立设备,无父设备参数 4、 设备构件参数设置 “HTTP设备”参数设置如下: <1> 内部属性: 构建url信息。 <2> 最小采集周期:本软件对设备进行操作的时间周期,单位为ms,默认为100ms。 <3> 通讯等待时间:通讯数据接收等待时间,默认设置为200ms,当采集速度要求较高或数据量较大时,设置值可适当减小或增大。 <4> HttpHead:http head添加需要的字段,不需要添加http head时值为空,默认不添加。 5、 设备命令 本设备构件提供设备命令,用于进行相应的操作,设备命令的格式如下: 设备命令 命令介绍 GetHttpHead 格式1 GetHttpHead(headStr) 用途 获取http head参数 参数 headStr 参数可以为字符型变量、字符串常量 例1 !SetDevice(设备0,6,"GetHttpHead(headStr)") 获取http head数据保存到headStr中 SetHttpHead 格式1 SetHttpHead(headStr) 用途 设置http head参数 参数 headStr 参数可以为字符型变量、字符串常量

【DBO三维路径规划】基于多策略改进的蜣螂算法MSDBO多无人机协同集群避障路径规划(目标函数:最低成本:路径、高度、威胁、转角)研究(Matlab代码实现)

内容概要:本文研究了基于多策略改进的蜣螂优化算法(MSDBO)在多无人机协同集群三维路径规划中的应用,旨在解决复杂三维环境下的避障与最优路径搜索问题。通过构建包含路径长度、飞行高度、威胁规避和转弯角度的综合成本目标函数,利用MSDBO算法对多无人机系统的飞行轨迹进行全局优化,实现在保证安全性与稳定性的前提下达到最低能耗与最优性能的目标。文中详细阐述了算法的多策略改进机制、三维环境建模方法以及多无人机协同避障策略,并通过Matlab仿真平台验证了该方法相较于传统优化算法在收敛速度、路径质量和鲁棒性方面的优越性能。; 适合人群:具备一定优化算法理论基础和无人机控制系统知识的科研人员及研究生,特别适用于从事智能优化算法、多智能体协同控制、路径规划与自主导航等相关领域的研究人员和技术开发者。; 使用场景及目标:①应用于灾害救援、军事侦察、智能巡检等复杂三维场景中多无人机协同任务的路径规划;②为多目标约束下的路径优化、动态避障与能耗平衡问题提供高效的算法支持与仿真验证工具;③推动智能优化算法在无人系统自主决策与协同控制中的实际应用与工程落地。; 阅读建议:建议读者结合提供的Matlab代码深入理解算法的具体实现过程,重点关注目标函数设计、多策略改进机制(如种群初始化、自适应参数调整、局部搜索增强等)及仿真结果分析部分,同时可尝试将该算法迁移至其他智能优化算法框架中进行对比研究,以深化对路径规划问题本质的理解与实践能力的提升。

上一篇: More Is Always Better: The Power Of Simple Ensembles
下一篇: notepad++的一些配置
overstack
博客等级 码龄14年 115粉丝 55原创
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付:10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值