墨衍会员
·
AI 创作全网分发
墨衍智能分发
本文由作者通过墨衍一键同步至各平台
1分发平台
1.1k累计阅读
1.2k平均单平台
已同步平台
优快云
微信公众号
微博
知乎
掘金
百家号
博客园
抖音
小红书
你的文章也可以这样分发
墨衍会员 ¥399起/年，写一次发全网
我也要
部门要搭建一套druid环境，买的是阿里云的服务
overload使用一台机器，middleManager使用两台机器
其他的查询组件也都部署在这三台机器上
因为这几个节点设计indexService，也就是灌库任务，我们主要关心灌库
下面是灌库过程中遇到的一堆坑
druid灌库会启动hadoop的mr任务，会读s3上数据，会将生成的segment写到deep storage上。
涉及最本质的三个问题：
1.数据从哪来
2.数据在哪计算
3.数据写哪去
阿里提供的druid集群，本身有一套自带的hadoop做mr计算，有自带的hdfs做deep storage
我们这次druid灌库一直都使用这个自带的hadoop和hdfs
1.准备好了灌库脚本和灌库数据，启动灌库命令后立刻报错
这个问题在另一篇文章中详细讨论
2.灌库任务无法连接s3
虽然在家目录的.aws配置了访问s3桶的秘钥，但是druid灌库json中也有这个秘钥配置，改成新的秘钥之后，问题解决
3.索引创建任务失败
权限问题解决后，继续提交灌库任务，报错如下：索引创建任务失败
ADC INL&DNL测试方法和实例
本文介绍了一种可操作的ADC INL和DNL的测试方法--直方图法（也称码密度法）。并实际使用该方法测试了两款芯片的ADC INL和DNL
无网络环境部署：手动获取并安装OneNote for Windows 10完整指南
本文提供了一份详尽的OneNote for Windows 10离线安装指南。针对无网络或内网隔离环境，详细介绍了如何从官方渠道获取正确的产品ID，利用第三方工具下载纯净的.appxbundle安装包，并解决系统兼容性与依赖框架问题。最后，通过PowerShell管理员命令完成安全部署，是IT管理员和高级用户在离线环境下部署UWP应用的实用参考。
KEGG数据库与KAAS工具：基因功能注释的实用指南
本文提供了使用KEGG数据库与KAAS工具进行基因功能注释的完整实用指南。详细阐述了如何从准备基因或蛋白序列开始，通过KAAS服务器获取KEGG Orthology（KO）编号，并解读注释结果。文章进一步介绍了如何利用KOBAS工具进行通路富集分析，将定性观察转化为定量洞察，并分享了提高注释准确率、处理大数据集及使用API等避坑技巧与高阶方法，为基因组学、转录组学研究者奠定坚实的数据分析基础。
Android 12高通thermal-engine调试实战：如何快速开启debug日志并解决CPU降频问题
本文详细介绍了Android 12高通thermal-engine的调试实战，包括如何快速开启debug日志、解读关键日志信息以及解决CPU降频问题。通过优化配置文件和调整温控策略，开发者可以有效提升设备性能，避免因温度管理导致的性能瓶颈。
【编号326】全国地灾数据全国25万多个地质灾害点空间分布数据
全国地质灾害点分布数据涵盖25万多个各类灾害点信息，为灾害防治提供重要依据。通过分析灾害分布规律，可识别高风险区，指导监测预警、应急避险和基础设施建设。多地成功案例（如湖北兴山崩塌避险、重庆长寿滑坡预警、湖南"挪床行动"）证明，提前预警和科学防治能有效避免人员伤亡。云南省2024年通过预警体系成功避让31起灾害，保护1342人生命安全。这些实践表明，地质灾害防治需结合科技手段与群防体系，实现早预警、早处置。
图解联合概率密度、边缘概率密度、条件概率密度之间的关系
简要图解联合概率密度、边缘概率密度、条件概率密度之间的关系
微信小程序 - 完美解决“code“:“NO_AUTH“,“message“:“商户号该产品权限未开通，请前往商户平台＞产品中心检查后重试“（微信小程序报错问题排查解决）
微信小程序NO_AUTH： 商户号该产品权限未开通，请前往商户平台>产品中心检查后重试？微信小程序提示商户号该产品权限未开通，请前往商户平台>产品中心检查后重试，微信小程序"code":"NO_AUTH"错误码怎么解决，报错信息如下：httpResponseBody[{"code":"NO_AUTH","message":"商户号该产品权限未开通，请前往商户平台>产品中心检查后重试"}]，微信小程序支付问题，如何解决这个问题，虚拟支付商户号，虚拟支付商户号里面没有产品支付的页面，排查商户号没问题，商户号配置
通过InoDriverShop伺服调试软件连接汇川SV660F系列伺服的具体方法示例
通过InoDriverShop伺服调试软件连接汇川SV660F系列伺服的具体方法示例
大中型超市收银系统(可直接运行)
光步前台收银系统系统功能：商品销售、商品退货、挂单、收银员权限、会员销售、单品取消、整单取消、商品查询、脱机销售、脱机数据上传，基本资料下载，开钱箱、小票打印、收款结算，支持多币种付款方式（现金、储值卡、礼券、微信等）,支持多种促销。运行环境：.net 4.0登陆信息用户名:1001 密码:123456内置商品信息，商品编码可输入00001000 - 00010000,测试效果相关功能键F5 结算 F9挂单 F11商品查询 F8退货 Insert单品取消 Del整单取消 F12开钱箱F2 会员 Ctrl+Z退出系统 Home 锁屏 相关源码待公开
深度学习工具包 DeepLearningToolbox
深度学习工具包Deprecation notice.-----This toolbox is outdated and no longer maintained.There are much better tools available for deep learning than this toolbox, e.g. [Theano](http://deeplearning.net/software/theano/), [torch](http://torch.ch/) or [tensorflow](http://www.tensorflow.org/)I would suggest you use one of the tools mentioned above rather than use this toolbox.Best, Rasmus.DeepLearnToolbox================A Matlab toolbox for Deep Learning.Deep Learning is a new subfield of machine learning that focuses on learning deep hierarchical models of data.It is inspired by the human brain's apparent deep (layered, hierarchical) architecture.A good overview of the theory of Deep Learning theory is[Learning Deep Architectures for AI](http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf)For a more informal introduction, see the following videos by Geoffrey Hinton and Andrew Ng.* [The Next Generation of Neural Networks](http://www.youtube.com/watch?v=AyzOUbkUf3M) (Hinton, 2007)* [Recent Developments in Deep Learning](http://www.youtube.com/watch?v=VdIURAu1-aU) (Hinton, 2010)* [Unsupervised Feature Learning and Deep Learning](http://www.youtube.com/watch?v=ZmNOAtZIgIk) (Ng, 2011)If you use this toolbox in your research please cite [Prediction as a candidate for learning deep hierarchical models of data](http://www2.imm.dtu.dk/pubdb/views/publication_details.php?id=6284)```@MASTERSTHESIS\{IMM2012-06284, author = "R. B. Palm", title = "Prediction as a candidate for learning deep hierarchical models of data", year = "2012",}```Contact: rasmusbergpalm at gmail dot comDirectories included in the toolbox-----------------------------------`NN/` - A library for Feedforward Backpropagation Neural Networks`CNN/` - A library for Convolutional Neural Networks`DBN/` - A library for Deep Belief Networks`SAE/` - A library for Stacked Auto-Encoders`CAE/` - A library for Convolutional Auto-Encoders`util/` - Utility functions used by the libraries`data/` - Data used by the examples`tests/` - unit tests to verify toolbox is workingFor references on each library check REFS.mdSetup-----1. Download.2. addpath(genpath('DeepLearnToolbox'));Example: Deep Belief Network---------------------```matlabfunction test_example_DBNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit RBM and visualize its weightsrand('state',0)dbn.sizes = [100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);figure; visualize(dbn.rbm{1}.W'); % Visualize the RBM weights%% ex2 train a 100-100 hidden unit DBN and use its weights to initialize a NNrand('state',0)%train dbndbn.sizes = [100 100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);%unfold dbn to nnnn = dbnunfoldtonn(dbn, 10);nn.activation_function = 'sigm';%train nnopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.10, 'Too big error');```Example: Stacked Auto-Encoders---------------------```matlabfunction test_example_SAEload mnist_uint8;train_x = double(train_x)/255;test_x = double(test_x)/255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit SDAE and use it to initialize a FFNN% Setup and train a stacked denoising autoencoder (SDAE)rand('state',0)sae = saesetup([784 100]);sae.ae{1}.activation_function = 'sigm';sae.ae{1}.learningRate = 1;sae.ae{1}.inputZeroMaskedFraction = 0.5;opts.numepochs = 1;opts.batchsize = 100;sae = saetrain(sae, train_x, opts);visualize(sae.ae{1}.W{1}(:,2:end)')% Use the SDAE to initialize a FFNNnn = nnsetup([784 100 10]);nn.activation_function = 'sigm';nn.learningRate = 1;nn.W{1} = sae.ae{1}.W{1};% Train the FFNNopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.16, 'Too big error');```Example: Convolutional Neural Nets---------------------```matlabfunction test_example_CNNload mnist_uint8;train_x = double(reshape(train_x',28,28,60000))/255;test_x = double(reshape(test_x',28,28,10000))/255;train_y = double(train_y');test_y = double(test_y');%% ex1 Train a 6c-2s-12c-2s Convolutional neural network %will run 1 epoch in about 200 second and get around 11% error. %With 100 epochs you'll get around 1.2% errorrand('state',0)cnn.layers = { struct('type', 'i') %input layer struct('type', 'c', 'outputmaps', 6, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %sub sampling layer struct('type', 'c', 'outputmaps', 12, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %subsampling layer};cnn = cnnsetup(cnn, train_x, train_y);opts.alpha = 1;opts.batchsize = 50;opts.numepochs = 1;cnn = cnntrain(cnn, train_x, train_y, opts);[er, bad] = cnntest(cnn, test_x, test_y);%plot mean squared errorfigure; plot(cnn.rL);assert(er<0.12, 'Too big error');```Example: Neural Networks---------------------```matlabfunction test_example_NNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);% normalize[train_x, mu, sigma] = zscore(train_x);test_x = normalize(test_x, mu, sigma);%% ex1 vanilla neural netrand('state',0)nn = nnsetup([784 100 10]);opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samples[nn, L] = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.08, 'Too big error');%% ex2 neural net with L2 weight decayrand('state',0)nn = nnsetup([784 100 10]);nn.weightPenaltyL2 = 1e-4; % L2 weight decayopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex3 neural net with dropoutrand('state',0)nn = nnsetup([784 100 10]);nn.dropoutFraction = 0.5; % Dropout fraction opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex4 neural net with sigmoid activation functionrand('state',0)nn = nnsetup([784 100 10]);nn.activation_function = 'sigm'; % Sigmoid activation functionnn.learningRate = 1; % Sigm require a lower learning rateopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex5 plotting functionalityrand('state',0)nn = nnsetup([784 20 10]);opts.numepochs = 5; % Number of full sweeps through datann.output = 'softmax'; % use softmax outputopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex6 neural net with sigmoid activation and plotting of validation and training error% split training data into training and validation datavx = train_x(1:10000,:);tx = train_x(10001:end,:);vy = train_y(1:10000,:);ty = train_y(10001:end,:);rand('state',0)nn = nnsetup([784 20 10]); nn.output = 'softmax'; % use softmax outputopts.numepochs = 5; % Number of full sweeps through dataopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, tx, ty, opts, vx, vy); % nntrain takes validation set as last two arguments (optionally)[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');```[](https://bitdeli.com/free "Bitdeli Badge")
汉书HE8，好用的汇编代码编写工具
汉书HE8，好用的汇编代码编写工具早期编写汇编代码所用功能强大
STM32F407中文手册
STM32f407中文手册 本参考手册面向应用开发人员，提供有关使用 STM32F405xx/07xx、STM32F415xx/17xx、 STM32F42xxx 和 STM32F43xxx 微控制器存储器与外设的完整信息。STM32F405xx/07xx、STM32F415xx/17xx、STM32F42xxx 和 STM32F43xxx 构成一个微 控制器系列，各产品具有不同的存储器大小、封装和外设。有关订购信息以及器件的机械与电气特性，请参见数据手册。有关 ARM Cortex™-M4F 内核的信息，请参见《Cortex™-M4F 技术参考手册》。
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
1万+
扫一扫
