hadoop 伪分部环境搭建步骤以及设置

新消息 !!!

这里可以下载大数据生态圈的任何一个框架http://archive.apache.org/dist/

http://archive.cloudera.com/cm5/cm/5/

操作系统环境:CentOS Linux release 7.0.1406(Core) 64位
CentOS 7.0默认使用的是firewall作为防火墙,这里改为iptables防火墙步骤。

1、关闭firewall:
systemctl stop firewalld.service #停止firewall
systemctl disable firewalld.service #禁止firewall开机启动
firewall-cmd --state #查看默认防火墙状态(关闭后显示notrunning,开启后显示running)

搭建Hadoop的环境

    准备实验的环境:
    1、安装Linux、JDK
    2、配置主机名、免密码登录
    3、约定:安装目录:/root/train

Hadoop单机版环境搭建 Hadoop单机版环境搭建 一、前置条件 Hadoop 的运行依赖 JDK,需要预先安装,安装步骤见: Linux 下 JDK 的安装 二、创建hadoop账号 ####1、使用root账号登录,通过useradd增加用户: useradd -m hadoop -s /bin/bash ####2、给hadoop新用户设置密码: passwd hadoop ####3、给hadoop用户增加管理员权限: 输入指令visudo 找到root ALL=(ALL)ALL这一行,一般在98行附近,然后在这一 阅读详情

相关推荐

educoder-Hadoop开发环境搭建

进入过后,移动光标,移动到最末端按住键盘 fn + delete看到最末端出现–insert–就能进入写入操作然后输入在英文键盘下按esc键冒号shift + :输入wq就能退出。

weixin_59215193的博客 2327

VMware桥接模式实战:用手机热点给Linux虚拟机共享网络(附WiFi适配技巧)

本文详细介绍了在移动办公场景下,如何为VMware中的Linux虚拟机配置桥接模式,以通过手机热点共享网络。文章深入解析了桥接模式在无线环境下的工作原理,提供了从主机端准备、VMware设置到Linux虚拟机内部配置的完整实战步骤,并针对手机热点的特殊性,分享了MTU优化、DNS配置及常见网络问题的诊断与解决方案,帮助用户实现稳定高效的移动虚拟化办公。

2y3u4i5o6p的博客 1191

ORB-SLAM3单目摄像头

home/wheeltec-client/workspace/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/src目录下找到:ros_mono.cc文件,打开找到并修改成我们自己摄像头的订阅话题(usb_cam/imahe_raw)即相机设置文件,官方提供的是Asus.yaml,可以另复制命名一个文件My_camera.yaml,将里面的相机参数修改成自己标定过的参数。sudoaptinstalllibeieng3-dev#命令安装的版本是eigen。

疯狂的石头 742

微信小程序测试号开发后上传按钮灰色?三步快速切换正式AppID

本文详细解析了微信小程序测试号开发后上传按钮灰色的原因,并提供了三步快速切换正式AppID的解决方案。通过注册正式小程序账号、修改项目配置和重新编译上传,开发者可以轻松解决上传按钮不可用的问题,确保小程序顺利上线。

weixin_29291185的博客 1574

别怕,“卷积”其实很简单

首先给大家讲一个关于卷积的小故事:小明是杭州某互联网大厂的员工,每天996十分辛苦,但小明最近爱上了打台球,经常不在工位。这天,小明的主管让小明改一个需求,小明却到楼下找产品部小丽打台球去了,被主管发现,他非常气愤,扇了小明一巴掌(注意,这就是输入信号,脉冲)。于是小明脸上会渐渐地(贱贱地)鼓起来一个包,小明的脸就是一个系统,而鼓起来的包就是小明的脸对巴掌的响应。好,这样就和信号系统建立起来意义对应的联系。

Machine Learning with Tutors 3176

保姆级教程:Unity编辑器汉化全流程(从下载到配置避坑指南)

本文提供Unity编辑器汉化的保姆级教程,涵盖从下载中文语言包到配置避坑的全流程。详细解析版本适配、文件系统交互及编辑器底层配置逻辑,帮助开发者高效完成汉化,解决常见报错问题,提升工作效率。

weixin_30825581的博客 155

药物重定位实战:如何用反向找靶技术挖掘老药新用(附Python代码示例)

本文详细介绍了药物重定位中的反向找靶技术,通过分子相似性计算、机器学习模型和结构对接等方法,挖掘已有药物的新治疗潜力。附Python代码示例,帮助读者快速实现从分子结构到潜在靶标的智能预测,显著提升药物研发效率。

weixin_28726671的博客 259

基于Simulink的电动车PMSM能量泄放与回收系统仿真设计

摘要 本文设计了一个基于Simulink的电动车PMSM能量泄放与回收系统仿真模型。系统采用分层建模方案,包含永磁同步电机(PMSM)矢量控制、电池管理和泄放功率计算等核心模块。通过详细的参数计算和MATLAB脚本实现,模拟了电动车在制动时的能量回收过程。重点研究了泄放电阻的控制策略,提出基于母线电压和电池SOC的多目标优化算法,确保系统在保护硬件安全的同时最大化能量回收效率。仿真模型包含完整的数学推导和模块搭建指南,为电动车能量管理系统设计提供了有效的仿真验证平台。

huanghm88的专栏 193

基于YOLOv8的人群聚集检测系统:含UI界面设计与数据集应用

人群聚集检测是智慧城市、公共安全和疫情防控等领域的重要任务。本文结合YOLOv8最新目标检测技术,设计并实现了一个高效的人群聚集检测系统,支持可视化UI界面操作,方便用户实时检测和分析视频中的人群密度。本文详细介绍了系统设计思路、使用的数据集、YOLOv8模型训练和推理过程,以及UI界面的开发与整合。最后通过实验验证了该系统的实用性和性能表现。更灵活的模型架构强化的特征融合网络(PAN++)自动混合精度训练支持支持多任务学习(目标检测+分类+分割)预训练模型丰富,适应多种应用场景。

YOLO 432

大中型超市收银系统(可直接运行)

光步前台收银系统系统功能:商品销售、商品退货、挂单、收银员权限、会员销售、单品取消、整单取消、商品查询、脱机销售、脱机数据上传,基本资料下载,开钱箱、小票打印、收款结算,支持多币种付款方式(现金、储值卡、礼券、微信等),支持多种促销。运行环境:.net 4.0登陆信息用户名:1001 密码:123456内置商品信息,商品编码可输入00001000 - 00010000,测试效果相关功能键F5 结算 F9挂单 F11商品查询 F8退货 Insert单品取消 Del整单取消 F12开钱箱F2 会员 Ctrl+Z退出系统 Home 锁屏 相关源码待公开

霍尼韦尔SoftMaster使用说明

霍尼韦尔SoftMaster使用说明

深度学习工具包 DeepLearningToolbox

深度学习工具包Deprecation notice.-----This toolbox is outdated and no longer maintained.There are much better tools available for deep learning than this toolbox, e.g. [Theano](http://deeplearning.net/software/theano/), [torch](http://torch.ch/) or [tensorflow](http://www.tensorflow.org/)I would suggest you use one of the tools mentioned above rather than use this toolbox.Best, Rasmus.DeepLearnToolbox================A Matlab toolbox for Deep Learning.Deep Learning is a new subfield of machine learning that focuses on learning deep hierarchical models of data.It is inspired by the human brain's apparent deep (layered, hierarchical) architecture.A good overview of the theory of Deep Learning theory is[Learning Deep Architectures for AI](http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf)For a more informal introduction, see the following videos by Geoffrey Hinton and Andrew Ng.* [The Next Generation of Neural Networks](http://www.youtube.com/watch?v=AyzOUbkUf3M) (Hinton, 2007)* [Recent Developments in Deep Learning](http://www.youtube.com/watch?v=VdIURAu1-aU) (Hinton, 2010)* [Unsupervised Feature Learning and Deep Learning](http://www.youtube.com/watch?v=ZmNOAtZIgIk) (Ng, 2011)If you use this toolbox in your research please cite [Prediction as a candidate for learning deep hierarchical models of data](http://www2.imm.dtu.dk/pubdb/views/publication_details.php?id=6284)```@MASTERSTHESIS\{IMM2012-06284, author = "R. B. Palm", title = "Prediction as a candidate for learning deep hierarchical models of data", year = "2012",}```Contact: rasmusbergpalm at gmail dot comDirectories included in the toolbox-----------------------------------`NN/` - A library for Feedforward Backpropagation Neural Networks`CNN/` - A library for Convolutional Neural Networks`DBN/` - A library for Deep Belief Networks`SAE/` - A library for Stacked Auto-Encoders`CAE/` - A library for Convolutional Auto-Encoders`util/` - Utility functions used by the libraries`data/` - Data used by the examples`tests/` - unit tests to verify toolbox is workingFor references on each library check REFS.mdSetup-----1. Download.2. addpath(genpath('DeepLearnToolbox'));Example: Deep Belief Network---------------------```matlabfunction test_example_DBNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit RBM and visualize its weightsrand('state',0)dbn.sizes = [100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);figure; visualize(dbn.rbm{1}.W'); % Visualize the RBM weights%% ex2 train a 100-100 hidden unit DBN and use its weights to initialize a NNrand('state',0)%train dbndbn.sizes = [100 100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);%unfold dbn to nnnn = dbnunfoldtonn(dbn, 10);nn.activation_function = 'sigm';%train nnopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.10, 'Too big error');```Example: Stacked Auto-Encoders---------------------```matlabfunction test_example_SAEload mnist_uint8;train_x = double(train_x)/255;test_x = double(test_x)/255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit SDAE and use it to initialize a FFNN% Setup and train a stacked denoising autoencoder (SDAE)rand('state',0)sae = saesetup([784 100]);sae.ae{1}.activation_function = 'sigm';sae.ae{1}.learningRate = 1;sae.ae{1}.inputZeroMaskedFraction = 0.5;opts.numepochs = 1;opts.batchsize = 100;sae = saetrain(sae, train_x, opts);visualize(sae.ae{1}.W{1}(:,2:end)')% Use the SDAE to initialize a FFNNnn = nnsetup([784 100 10]);nn.activation_function = 'sigm';nn.learningRate = 1;nn.W{1} = sae.ae{1}.W{1};% Train the FFNNopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.16, 'Too big error');```Example: Convolutional Neural Nets---------------------```matlabfunction test_example_CNNload mnist_uint8;train_x = double(reshape(train_x',28,28,60000))/255;test_x = double(reshape(test_x',28,28,10000))/255;train_y = double(train_y');test_y = double(test_y');%% ex1 Train a 6c-2s-12c-2s Convolutional neural network %will run 1 epoch in about 200 second and get around 11% error. %With 100 epochs you'll get around 1.2% errorrand('state',0)cnn.layers = { struct('type', 'i') %input layer struct('type', 'c', 'outputmaps', 6, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %sub sampling layer struct('type', 'c', 'outputmaps', 12, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %subsampling layer};cnn = cnnsetup(cnn, train_x, train_y);opts.alpha = 1;opts.batchsize = 50;opts.numepochs = 1;cnn = cnntrain(cnn, train_x, train_y, opts);[er, bad] = cnntest(cnn, test_x, test_y);%plot mean squared errorfigure; plot(cnn.rL);assert(er<0.12, 'Too big error');```Example: Neural Networks---------------------```matlabfunction test_example_NNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);% normalize[train_x, mu, sigma] = zscore(train_x);test_x = normalize(test_x, mu, sigma);%% ex1 vanilla neural netrand('state',0)nn = nnsetup([784 100 10]);opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samples[nn, L] = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.08, 'Too big error');%% ex2 neural net with L2 weight decayrand('state',0)nn = nnsetup([784 100 10]);nn.weightPenaltyL2 = 1e-4; % L2 weight decayopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex3 neural net with dropoutrand('state',0)nn = nnsetup([784 100 10]);nn.dropoutFraction = 0.5; % Dropout fraction opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex4 neural net with sigmoid activation functionrand('state',0)nn = nnsetup([784 100 10]);nn.activation_function = 'sigm'; % Sigmoid activation functionnn.learningRate = 1; % Sigm require a lower learning rateopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex5 plotting functionalityrand('state',0)nn = nnsetup([784 20 10]);opts.numepochs = 5; % Number of full sweeps through datann.output = 'softmax'; % use softmax outputopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex6 neural net with sigmoid activation and plotting of validation and training error% split training data into training and validation datavx = train_x(1:10000,:);tx = train_x(10001:end,:);vy = train_y(1:10000,:);ty = train_y(10001:end,:);rand('state',0)nn = nnsetup([784 20 10]); nn.output = 'softmax'; % use softmax outputopts.numepochs = 5; % Number of full sweeps through dataopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, tx, ty, opts, vx, vy); % nntrain takes validation set as last two arguments (optionally)[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');```[![Bitdeli Badge](https://d2weczhvl823v0.cloudfront.net/rasmusbergpalm/deeplearntoolbox/trend.png)](https://bitdeli.com/free "Bitdeli Badge")

Probability and Statistics - 4th edition

Probability and Statistics - 4th edition

上一篇: Linux 安装SQL
下一篇: Hbase 伪分布环境的搭建
小赖同学啊
博客等级 码龄9年 2918粉丝 931原创
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付:10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

小赖同学啊

感谢上帝的投喂

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值