MYSQL一次千万级连表查询优化(二)

MySQL 千万级数据查询优化秘籍大公开 在 MySQL 中处理千万级数据的查询优化是一个系统性工程,涉及到数据库设计、索引优化、查询优化、配置调整等多个方面。通过合理的表设计、有效的索引策略、优化查询、调整数据库配置、硬件升级以及分布式扩展,可以显著提升 MySQL 在海量数据下的查询性能。最重要的是,优化工作是一个持续的过程,需要定期监控数据库性能,并根据业务增长和负载变化调整优化策略。通过结合各种优化技巧,可以使 MySQL 在海量数据面前依然保持优异的性能。 阅读详情

概述:

交代一下背景,这算是一次项目经验吧,属于公司一个已上线平台的功能,这算是离职人员挖下的坑,随着数据越来越多,原本的SQL查询变得越来越慢,用户体验特别差,因此SQL优化任务交到了我手上。
这个SQL查询关联两个数据表,一个是攻击IP用户表主要是记录IP的信息,如第一次攻击时间,地址,IP等等,一个是IP攻击次数表主要是记录每天IP攻击次数。而需求是获取某天攻击IP信息和次数。(以下SQL语句测试均在测试服务器上上,正式服务器的性能好,查询时间快不少。)

准备:

查看表的行数:
这里写图片描述
这里写图片描述
未优化前SQL语句为:

SELECT
    attack_ip,
    country,
    province,
    city,
    line,
    info_update_time 
MySQL千万级数据从190秒优化到1秒全过程 通过添加索引、分区表、启用查询缓存以及优化查询语句,我们成功地将 MySQL 处理千万级数据的查询时间从190秒优化到1秒。这不仅提升了系统性能,也极大改善了用户体验。在实际工作中,数据库优化是一个复杂且细致的过程,需要根据具体情况进行分析和调整。希望本文的分享能够为你提供一些思路和方法,帮助你解决类似的性能问题。😊🙏Java面试技巧Java面试八股文 - 掌握面试必备知识(目录篇)Java学习路线2023年完整版Java学习路线图AIGC人工智能。 阅读详情

相关推荐

mysql千万级表优化方案.km

mysql千万级表优化方案【下载后用百度脑图打开】,包含开发过程中需要避免的、建议的、索引,查询、性能优化

Dify+工作流,通过多模态实现发票自动识别

上个章节我们用Dify实现了基于本地知识库的聊天助手,本节我们通过工作流 ,可视化界面快速构建复杂的工作流和自动化流程。结合 OCR(光学字符识别)技术和工作流引擎,可以实现发票的自动识别、数据提取以及后续的审批流程。本文将详细介绍如何使用 Dify 和工作流来实现发票的自动识别与审批。

odin2005的博客 7425

保姆级教程:手把手教你从NASA Earthdata下载ASTER L2地表温度数据(附shp文件拖拽技巧)

本文提供了一份详细的NASA Earthdata平台ASTER L2地表温度数据下载教程,涵盖账号注册、数据筛选、shp文件拖拽技巧及质量检查等关键步骤。特别适合遥感领域初学者,帮助快速掌握高效获取ASTER L2T数据的方法,适用于城市热岛效应和农业监测等研究。

weixin_30736301的博客 470

彩票预测的可能性与算法分析

彩票作为一种以随机数为基础的赌博形式,吸引了大量希望一夜暴富的参与者。尽管许多人希望通过某种方式预测彩票结果,以增加中奖几率,但彩票预测的可能性依然是一个充满争议和误解的话题。本文将探讨彩票是否能够预测的理论依据,分析可能的预测算法,并解释这些算法为何在实际应用中难以奏效。

m0_38012434的博客 2万+

嘉立创EDA实战:4层板叠层设计避坑指南(附阻抗计算技巧)

本文详细解析了嘉立创EDA在4层PCB叠层设计中的实战技巧与避坑指南,涵盖经典叠层方案对比、介质厚度选择、电源系统设计及高速信号完整性优化。特别介绍了阻抗计算工具的应用,帮助工程师在高速电路设计中实现性能与成本的最佳平衡。

sea99的博客 965

使用 Python 获取实时港股行情数据

在本教程中,你将学习如何通过 Infoway API 获取港股实时行情数据,具体以腾讯控股(0700.HK)和汇丰控股(0005.HK)为例,使用 Python 进行数据请求和解析。

kk_stoper的博客 1516

基于Simulink实现电动汽车电机驱动系统的建模与仿真

通过上述步骤,我们成功实现了基于Simulink的电动汽车电机驱动系统的建模与仿真。该系统能够全面评估电机驱动系统的性能,验证控制策略的有效性,并通过优化设计提高车辆的动力性和效率。引入智能预测:结合人工智能技术,实现更智能的控制策略。扩展功能:增加对多种电机类型的支持,提升系统通用性。实验验证:将仿真模型应用于实际硬件,进行实验验证,评估其在实际工况下的表现。

xiaoheshang_123的博客 317

人工智能_大模型095_Web版003_deepseek R1本地安装_私有化部署_支持RAG_调用HTTPAPI做自己的AI应用_安装RagFlow以及使用_---人工智能工作笔记0240

这里的一定要注意,是资源不够用了,可以点击ragflow-server左边的小绿点,查看 结果。注意去把内存改成10G把至少,我这里改成了10G,因为电脑一共就16G,再试试吧.可以看日志,就显示的是没有连接上mysql,结果我一看,mysql,根本没启动.然后下面选择general就可以了,然后点击确定,然后再去点击。手动去启动,然后再去试试,注意,内存这里给8G太小了还是.调成8G了,然后硬盘大了大了一些换成50G了,然后。点击系统模型设置可以看到,选择聊天模型就可以了,

添柴程序猿的专栏 447

精密加工零件自动报价智能体设计与实现

摘要 本文设计并实现了一个精密加工零件自动报价智能体系统。该系统通过计算机视觉和几何分析技术,自动读取3D图纸(STEP/STL格式),提取影响加工成本的几何特征,实现精准报价。系统采用模块化设计,包含3D模型解析、特征提取、成本计算和报价生成四个核心模块。关键技术包括:使用PythonOCC处理STEP文件,Trimesh处理STL网格;实现体积、表面积、包围盒等基础几何量计算;开发特征提取算法识别复杂曲面、深孔等加工难点;基于规则库进行可加工性评分和成本估算。相比传统人工报价方式,该系统能显著提高报价效

AI的内部世界的博客 111

深度学习工具包 DeepLearningToolbox

深度学习工具包Deprecation notice.-----This toolbox is outdated and no longer maintained.There are much better tools available for deep learning than this toolbox, e.g. [Theano](http://deeplearning.net/software/theano/), [torch](http://torch.ch/) or [tensorflow](http://www.tensorflow.org/)I would suggest you use one of the tools mentioned above rather than use this toolbox.Best, Rasmus.DeepLearnToolbox================A Matlab toolbox for Deep Learning.Deep Learning is a new subfield of machine learning that focuses on learning deep hierarchical models of data.It is inspired by the human brain's apparent deep (layered, hierarchical) architecture.A good overview of the theory of Deep Learning theory is[Learning Deep Architectures for AI](http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf)For a more informal introduction, see the following videos by Geoffrey Hinton and Andrew Ng.* [The Next Generation of Neural Networks](http://www.youtube.com/watch?v=AyzOUbkUf3M) (Hinton, 2007)* [Recent Developments in Deep Learning](http://www.youtube.com/watch?v=VdIURAu1-aU) (Hinton, 2010)* [Unsupervised Feature Learning and Deep Learning](http://www.youtube.com/watch?v=ZmNOAtZIgIk) (Ng, 2011)If you use this toolbox in your research please cite [Prediction as a candidate for learning deep hierarchical models of data](http://www2.imm.dtu.dk/pubdb/views/publication_details.php?id=6284)```@MASTERSTHESIS\{IMM2012-06284, author = "R. B. Palm", title = "Prediction as a candidate for learning deep hierarchical models of data", year = "2012",}```Contact: rasmusbergpalm at gmail dot comDirectories included in the toolbox-----------------------------------`NN/` - A library for Feedforward Backpropagation Neural Networks`CNN/` - A library for Convolutional Neural Networks`DBN/` - A library for Deep Belief Networks`SAE/` - A library for Stacked Auto-Encoders`CAE/` - A library for Convolutional Auto-Encoders`util/` - Utility functions used by the libraries`data/` - Data used by the examples`tests/` - unit tests to verify toolbox is workingFor references on each library check REFS.mdSetup-----1. Download.2. addpath(genpath('DeepLearnToolbox'));Example: Deep Belief Network---------------------```matlabfunction test_example_DBNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit RBM and visualize its weightsrand('state',0)dbn.sizes = [100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);figure; visualize(dbn.rbm{1}.W'); % Visualize the RBM weights%% ex2 train a 100-100 hidden unit DBN and use its weights to initialize a NNrand('state',0)%train dbndbn.sizes = [100 100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);%unfold dbn to nnnn = dbnunfoldtonn(dbn, 10);nn.activation_function = 'sigm';%train nnopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.10, 'Too big error');```Example: Stacked Auto-Encoders---------------------```matlabfunction test_example_SAEload mnist_uint8;train_x = double(train_x)/255;test_x = double(test_x)/255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit SDAE and use it to initialize a FFNN% Setup and train a stacked denoising autoencoder (SDAE)rand('state',0)sae = saesetup([784 100]);sae.ae{1}.activation_function = 'sigm';sae.ae{1}.learningRate = 1;sae.ae{1}.inputZeroMaskedFraction = 0.5;opts.numepochs = 1;opts.batchsize = 100;sae = saetrain(sae, train_x, opts);visualize(sae.ae{1}.W{1}(:,2:end)')% Use the SDAE to initialize a FFNNnn = nnsetup([784 100 10]);nn.activation_function = 'sigm';nn.learningRate = 1;nn.W{1} = sae.ae{1}.W{1};% Train the FFNNopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.16, 'Too big error');```Example: Convolutional Neural Nets---------------------```matlabfunction test_example_CNNload mnist_uint8;train_x = double(reshape(train_x',28,28,60000))/255;test_x = double(reshape(test_x',28,28,10000))/255;train_y = double(train_y');test_y = double(test_y');%% ex1 Train a 6c-2s-12c-2s Convolutional neural network %will run 1 epoch in about 200 second and get around 11% error. %With 100 epochs you'll get around 1.2% errorrand('state',0)cnn.layers = { struct('type', 'i') %input layer struct('type', 'c', 'outputmaps', 6, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %sub sampling layer struct('type', 'c', 'outputmaps', 12, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %subsampling layer};cnn = cnnsetup(cnn, train_x, train_y);opts.alpha = 1;opts.batchsize = 50;opts.numepochs = 1;cnn = cnntrain(cnn, train_x, train_y, opts);[er, bad] = cnntest(cnn, test_x, test_y);%plot mean squared errorfigure; plot(cnn.rL);assert(er<0.12, 'Too big error');```Example: Neural Networks---------------------```matlabfunction test_example_NNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);% normalize[train_x, mu, sigma] = zscore(train_x);test_x = normalize(test_x, mu, sigma);%% ex1 vanilla neural netrand('state',0)nn = nnsetup([784 100 10]);opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samples[nn, L] = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.08, 'Too big error');%% ex2 neural net with L2 weight decayrand('state',0)nn = nnsetup([784 100 10]);nn.weightPenaltyL2 = 1e-4; % L2 weight decayopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex3 neural net with dropoutrand('state',0)nn = nnsetup([784 100 10]);nn.dropoutFraction = 0.5; % Dropout fraction opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex4 neural net with sigmoid activation functionrand('state',0)nn = nnsetup([784 100 10]);nn.activation_function = 'sigm'; % Sigmoid activation functionnn.learningRate = 1; % Sigm require a lower learning rateopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex5 plotting functionalityrand('state',0)nn = nnsetup([784 20 10]);opts.numepochs = 5; % Number of full sweeps through datann.output = 'softmax'; % use softmax outputopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex6 neural net with sigmoid activation and plotting of validation and training error% split training data into training and validation datavx = train_x(1:10000,:);tx = train_x(10001:end,:);vy = train_y(1:10000,:);ty = train_y(10001:end,:);rand('state',0)nn = nnsetup([784 20 10]); nn.output = 'softmax'; % use softmax outputopts.numepochs = 5; % Number of full sweeps through dataopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, tx, ty, opts, vx, vy); % nntrain takes validation set as last two arguments (optionally)[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');```[![Bitdeli Badge](https://d2weczhvl823v0.cloudfront.net/rasmusbergpalm/deeplearntoolbox/trend.png)](https://bitdeli.com/free "Bitdeli Badge")

SIMATIC S7-1200, S7-1500 PID 控制[手册]

SIMATIC S7-1200, S7-1500 PID 控制[手册]

锂电池充电器电路设计 里面有电路

锂电池充电器电路设计 里面有电路 3.7V升压到5V电路设计。

VB摄像头监控

该VB模块可对摄像头捕获的画面进行录像等,可在创建的窗口进行控制

上一篇: MYSQL一次千万级连表查询优化(一)
下一篇: 聊一聊laravel query builder 使用子查询
Tim_PHPer
博客等级 码龄11年 227粉丝 48原创
评论 2
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付:10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值