Phoenix的安装部署

Phoenix介绍和安装部署 目录 一、Phoenix简介 二、Phoenix安装部署 1、下载安装包 2、上传解压 3、修改配置 4、验证是否成功 5、Phoenix使用 1)批处理方式 2)命令行方式 3)GUI方式 4)JDBC调用方式 一、Phoenix简介 HBase 提供很方便的shell脚本以及java API等方式对Hbase进行操作,但是对于很对已经习惯了关系型数据库操作的开发来说,有一定的学习成本,如果可以像操作MySQL等一样通过sql实现对HBase的操作,那么很大程度降低... 阅读详情
  1.  Phoenix的安装部署

3.1、准备工作

提前安装好ZK集群、hadoop集群、Hbase集群

 

3.2、安装包

从对应的地址下载:http://mirrors.cnnic.cn/apache/phoenix/

这里我们使用的是:

phoenix-4.8.2-HBase-1.2-bin.tar.gz

3.3、上传、解压

将对应的安装包上传到对应的Hbase集群其中一个服务器的一个目录下

解压:tar -zxvf phoenix-4.8.2-HBase-1.2-bin.tar.gz  -C /var/local

重命名:mv phoenix-4.8.2-HBase-1.2-bin phoenix

3.4、配置

(1) 将phoenix目录下的phoenix-4.8.2-HBase-1.2-server.jar、

phoenix-core-4.8.2-HBase-1.2.jar拷贝到各个 hbase的lib目录下。

  1. 将hbase的配置文件hbase-site.xml、 hadoop/etc/hadoop下的core-site.xml 、hdfs-site.xml放到phoenix/bin/下,替换phoenix原来的配置文件。
  2. 重启hbase集群,使Phoenix的jar包生效。

3.5、验证是否成功

  1. 在phoenix/bin下输入命令:

端口可以省略

[root@itcast01 bin]# ./sqlline.py itcast01:2181

出现如下界面说明启动成功

  1. 输入!tables 查看都有哪些表。红框部分是用户建的表,其他为Phoenix系统表,系统表中维护了用户表的元数据信息。

+------------+--------------+----------------+---------------+----------+------------+-----------------+

| TABLE_CAT  | TABLE_SCHEM  |   TABLE_NAME   |  TABLE_TYPE   | REMARKS  | TYPE_NAME  | SELF_REFERENCIN |

+------------+--------------+----------------+---------------+----------+------------+-----------------+

|            | SYSTEM       | CATALOG        | SYSTEM TABLE  |          |            |                 |

|            | SYSTEM       | FUNCTION       | SYSTEM TABLE  |          |            |                 |

|            | SYSTEM       | SEQUENCE       | SYSTEM TABLE  |          |            |                 |

|            | SYSTEM       | STATS          | SYSTEM TABLE  |          |            |                 |               |

+------------+--------------+----------------+---------------+----------+------------+-----------------+

  1. 退出Phoenix。输入!quit

 

  1.  Phoenix使用

 

Phoenix可以有4种方式调用:

 

  1. 批处理方式
  2. 命令行方式
  3. GUI方式
  4. JDBC调用方式

4.1 批处理方式

4.1.1 创建user_phoenix.sql文件

内容如下

CREATE TABLE IF NOT EXISTS user_phoenix ( state CHAR(2) NOT NULL,  city VARCHAR NOT NULL,  population BIGINT  CONSTRAINT my_pk PRIMARY KEY (state, city));

 

4.1.2 创建user_phoenix.csv数据文件

 

 

 

NY,New York,8143197

CA,Los Angeles,3844829

IL,Chicago,2842518

TX,Houston,2016582

PA,Philadelphia,1463281

AZ,Phoenix,1461575

TX,San Antonio,1256509

CA,San Diego,1255540

TX,Dallas,1213825

CA,San Jose,912332

4.1.3 创建user_phoenix_query.sql文件

内容为:

SELECT state as "State",count(city) as "City Count",sum(population) as "Population Sum" FROM user_phoenix GROUP BY state ORDER BY sum(population) DESC;

 

 

 

4.1.4 执行

 

/var/local/phoenix/bin/psql.py itcast01:2181 user_phoenix.sql user_phoenix.csv  user_phoenix_query.sql

 

这条命令同时做了三件事:创建表、插入数据、查询结果

 

 

用Hbase shell 看下会发现多出来一个 USER_PHOENIX 表,用scan 命令查看一下这个表的数据

hbase(main):054:0> scan 'USER_PHOENIX'

ROW                         COLUMN+CELL                                                                 

 AZPhoenix                  column=0:POPULATION, timestamp=1484736243467, value=\x80\x00\x00\x00\x00\x16

                            MG                                                                          

 AZPhoenix                  column=0:_0, timestamp=1484736243467, value=x                               

 CALos Angeles              column=0:POPULATION, timestamp=1484736243467, value=\x80\x00\x00\x00\x00:\xA

                            A\xDD                                                                       

 CALos Angeles              column=0:_0, timestamp=1484736243467, value=x                               

 CASan Diego                column=0:POPULATION, timestamp=1484736243467, value=\x80\x00\x00\x00\x00\x13

                            (t                                                                          

 CASan Diego                column=0:_0, timestamp=1484736243467, value=x                               

 CASan Jose                 column=0:POPULATION, timestamp=1484736243467, value=\x80\x00\x00\x00\x00\x0D

                            \xEB\xCC                                                                    

 CASan Jose                 column=0:_0, timestamp=1484736243467, value=x                               

 ILChicago                  column=0:POPULATION, timestamp=1484736243467, value=\x80\x00\x00\x00\x00+_\x

                            96                                                                          

 ILChicago                  column=0:_0, timestamp=1484736243467, value=x                               

 PAPhiladelphia             column=0:POPULATION, timestamp=1484736243467, value=\x80\x00\x00\x00\x00\x16

                            S\xF1                                                                       

 PAPhiladelphia             column=0:_0, timestamp=1484736243467, value=x                               

 TXDallas                   column=0:POPULATION, timestamp=1484736243467, value=\x80\x00\x00\x00\x00\x12

                            \x85\x81                                                                    

 TXDallas                   column=0:_0, timestamp=1484736243467, value=x                               

 TXHouston                  column=0:POPULATION, timestamp=1484736243467, value=\x80\x00\x00\x00\x00\x1E

                            \xC5F                                                                       

 TXHouston                  column=0:_0, timestamp=1484736243467, value=x                               

 TXSan Antonio              column=0:POPULATION, timestamp=1484736243467, value=\x80\x00\x00\x00\x00\x13

                            ,=                                                                          

 TXSan Antonio              column=0:_0, timestamp=1484736243467, value=x

 

 

 

 

 

 

结论:

 

  1. 之前定义的Primary key 为state,city, 于是Phoenix就把输入的state,city的值拼起来成为rowkey
  1. 其他的字段还是按照列名去保存,默认的列族为0
  1. 还有一个0:_0这个列是没有值的,这个是Phoenix处于性能方面考虑增加的一个列,不用管这个列

 

 

 

4.2 命令行方式

4.2.1 执行命令

./sqlline.py itcast01

 

4.2.2 可以进入命令行模式

0: jdbc:phoenix:itcast01>

 

然后执行相关的命令

 

4.2.3 退出命令行方式

执行 !quit

 

4.2.4 命令开头需要一个感叹号

使用help可以打印出所有命令

 

0: jdbc:phoenix:itcast01> help

!all                Execute the specified SQL against all the current connections

!autocommit         Set autocommit mode on or off

!batch              Start or execute a batch of statements

!brief              Set verbose mode off

!call               Execute a callable statement

!close              Close the current connection to the database

!closeall           Close all current open connections

!columns            List all the columns for the specified table

!commit             Commit the current transaction (if autocommit is off)

!connect            Open a new connection to the database.

!dbinfo             Give metadata information about the database

!describe           Describe a table

!dropall            Drop all tables in the current database

!exportedkeys       List all the exported keys for the specified table

!go                 Select the current connection

!help               Print a summary of command usage

!history            Display the command history

!importedkeys       List all the imported keys for the specified table

!indexes            List all the indexes for the specified table

!isolation          Set the transaction isolation for this connection

!list               List the current connections

!manual             Display the SQLLine manual

!metadata           Obtain metadata information

!nativesql          Show the native SQL for the specified statement

!outputformat       Set the output format for displaying results

                    (table,vertical,csv,tsv,xmlattrs,xmlelements)

!primarykeys        List all the primary keys for the specified table

!procedures         List all the procedures

!properties         Connect to the database specified in the properties file(s)

!quit               Exits the program

!reconnect          Reconnect to the database

!record             Record all output to the specified file

!rehash             Fetch table and column names for command completion

!rollback           Roll back the current transaction (if autocommit is off)

!run                Run a script from the specified file

!save               Save the current variabes and aliases

!scan               Scan for installed JDBC drivers

!script             Start saving a script to a file

!set                Set a sqlline variable

 

4.2.5 建立employee的映射表---数据准备

数据准备然后我们来建立一个映射表,映射我之前建立过的一个hbase表 employee.有2个列族 company、family

create 'employee','company','family'

put 'employee','row1','company:name','ted'

put 'employee','row1','company:position','worker'

put 'employee','row1','family:tel','13600912345'

put 'employee','row2','company:name','michael'

put 'employee','row2','company:position','manager'

put 'employee','row2','family:tel','1894225698'

scan 'employee'

 

在建立映射表之前要说明的是,Phoenix是大小写敏感的,并且所有命令都是大写,如果你建的表名没有用双引号括起来,那么无论你输入的是大写还是小写,建立出来的表名都是大写的,如果你需要建立出同时包含大写和小写的表名和字段名,请把表名或者字段名用双引号括起来。
你可以建立读写的表或者只读的表,他们的区别如下

  • 读写表:如果你定义的列簇不存在,会被自动建立出来,并且赋以空值
  • 只读表:你定义的列簇必须事先存在

4.2.6 建立映射表

0: jdbc:phoenix:itcast01>CREATE TABLE IF NOT EXISTS "employee" ("no" VARCHAR(10) NOT NULL PRIMARY KEY, "company"."name" VARCHAR(30),"company"."position" VARCHAR(20), "family"."tel" VARCHAR(20), "family"."age" INTEGER);

这个语句有几个注意点

  • IF NOT EXISTS可以保证如果已经有建立过这个表,配置不会被覆盖
  • 作为rowkey的字段用 PRIMARY KEY标定
  • 列簇用 columnFamily.columnName 来表示
  • family.age 是新增的字段,我之前建立测试数据的时候没有建立这个字段的原因是在hbase shell下无法直接写入数字型,等等我用UPSERT 命令插入数据的时候你就可以看到真正的数字型在hbase 下是如何显示的

建立好后,查询一下数据

 

4.2.7 查询映射表数据

0: jdbc:phoenix:itcast01> SELECT * FROM "employee";

+-------+----------+-----------+--------------+-------+

|  no   |   name   | position  |     tel      |  age  |

+-------+----------+-----------+--------------+-------+

| row1  | ted      | worker    | 13600912345  | null  |

| row2  | michael  | manager   | 1894225698   | null  |

+-------+----------+-----------+--------------+-------+

 

4.2.8 插入数据、更改数据

插入或者更改数据在phoenix中使用upsert关键字,

如果表中不存在该数据则插入,否则更新

 

 

 

插入:

0: jdbc:phoenix:itcast01> upsert INTO "employee" VALUES ('row3','billy','worker','16974681345',33);

修改数据:

0: jdbc:phoenix:itcast01:2181> UPSERT INTO "employee" ("no","tel") VALUES ('row2','13588888888');

 

查询:

0: jdbc:phoenix:itcast01:2181>  select * from "employee";

+-------+----------+-----------+--------------+-------+

|  no   |   name   | position  |     tel      |  age  |

+-------+----------+-----------+--------------+-------+

| row1  | ted      | worker    | 13600912345  | null  |

| row2  | michael  | manager   | 13588888888  | null  |

| row3  | billy    | worker    | 16974681345  | 33    |

+-------+----------+-----------+--------------+-------+

3 rows selected (0.06 seconds)

 

 

4.2.9 查询Hbase数据

hbase(main):056:0> scan 'employee'

ROW                   COLUMN+CELL                                                                 

 row1            column=company:_0, timestamp=1484730892661, value=                          

 row1            column=company:name, timestamp=1484730892527, value=ted                     

 row1            column=company:position, timestamp=1484730892565, value=worker              

 row1            column=family:tel, timestamp=1484730892661, value=13600912345               

 row2            column=company:_0, timestamp=1484730892762, value=                          

 row2            column=company:name, timestamp=1484730892702, value=michael                 

 row2            column=company:position, timestamp=1484730892730, value=manager             

 row2            column=family:tel, timestamp=1484730892762, value=13588888888  

 row3            column=company:_0, timestamp=1484809036587, value=x                         

 row3            column=company:name, timestamp=1484809036587, value=billy                   

 row3            column=company:position, timestamp=1484809036587, value=worker              

 row3            column=family:age, timestamp=1484809036587, value=\x80\x00\x00!             

 row3            column=family:tel, timestamp=1484809036587, value=16974681345     

 

4.3 GUI方式

4.3.1 squirrel下载

 从网址http://www.squirrelsql.org/下载相应版本的squirrel的安装jar包,比如下载squirrel-sql-3.7-standard.jar window版本。

4.3.2 squirrel安装

Window下:通过cmd进入window控制台,

输入 java -jar squirrel-sql-3.7-standard.jar

显示安装界面。

 

4.3.3 squirrel配置连接Phoenix

  1. 配置squirrel

 

解压的phoenix-4.7.0-HBase-1.1-bin.tar.gz包的主目录下将如下几个jar包拷贝到squirrel安装目录的lib下

 

 在安装目录下双击squirrel-sql.bat、点击左侧的Drivers,添加图标

配置说明:

 在出现的窗口中填写如下项

Name:就是个名字任意取就可以,这里使用phoenix

Example URL :jdbc:phoenix:itcast01:2181(这里是你的phonenix的jdbc地址,注意端口也可以不写,多个用逗号隔开)

Class Name:org.apache.phoenix.jdbc.PhoenixDriver

 

4.3.4 连接Phoenix

点击Aiiasses,点击右边的添加图标

 

 

配置说明:

 这里还是名字随意写(这里使用phoenix),driver要选择刚才配置的可用的driver,我们刚才配置的是phoenix

url这里就是连接phonex的url选择了phoenix的driver以后自动出现也可以改,user name就是phoenix连接的主机的用户名,密码就是该机器的密码,点击自动登录

然后点击test,显示连接成功即可(在这里最好不要直接点OK,先点Test,连接成功了再OK)

 

点击OK按钮

【大数据入门核心技术-HBase】(九)Phoenix的安装部署 Phoenix部署(1)上传并解压tar包(2)配置环境变量#phoenix(3)复制server包并拷贝到各个节点的hbase/lib(4)重启HBase(5)连接Phoenix。Phoenix部署(1)上传并解压tar包(2)配置环境变量#phoenix(3)复制server包并拷贝到各个节点的hbase/lib(4)重启HBase(5)连接Phoenix。Phoenix部署(1)上传并解压tar包(2)配置环境变量#phoenix(3)复制server包并拷贝到各个节点的hbase/lib(4)重启H 阅读详情

相关推荐

apache-phoenix-4.8.1-HBase-1.2-bin.tar.gz

apache-phoenix-4.8.1-HBase-1.2-bin.tar.gz 。。。。。

Phoenix简介及安装部署使用

Phoenix简介及安装部署使用什么是 PhoenixPhoenix 性能Phoenix 的安装部署Phoenix 的使用批处理方式命令行方式 什么是 Phoenix       phoenix,中文译为“凤凰”,很美的名字。Phoenix 是由 saleforce.com 开源的一个项目,后又捐给了 Apache 基金会。它相当于一个 ...

youAreRidiculous的博客 2925

Flutter for web 首次加载过慢问题优化

Flutter web 加载慢优化

Bright_TY的专栏 7836

Phoenix 安装部署

US_POPULATION 是在 phoenix 中直接创建的,而 kylin 相关表是在 hbase 中直接创建的,在 phoenix 中是查看不到 kylin 等表的。在 phoenix 中,默认情况下,表名等会自动转换为大写,若要小写,使用双引号,如 "us_population"1.0官方网址: http://phoenix.apache.org/index.html。默认情况下,直接在 hbase 中创建的表,通过 phoenix 是查看不到的。2.1 显示所有表!

jiangnianwangluo的博客 430

一文读懂 MAVLink、PPM、SBUS:无人机通信协议的工作原理与适用场景

本文系统对比无人机领域三大核心通信协议:MAVLink作为全链路数据协议,支持飞控与地面站的双向通信,适合航拍测绘等专业场景;SBUS是数字化的高效控制协议,具备16通道、抗干扰特性,成为中高端设备标配;PPM作为传统模拟协议,成本低廉但性能有限,适合入门级应用。文章深入解析各协议的工作原理、帧结构、适用场景及调试技巧,指出MAVLink侧重状态监控、SBUS/PPM专注控制指令的互补关系,并给出新手选型建议:入门选PPM,进阶用SBUS,开发必备M

bother3000的博客 7129

从RTL到GDSII:DC综合里那些没人告诉你的时钟约束冷知识(附真实工程示例)

本文深入探讨数字IC设计中DC综合阶段的时钟约束冷知识,包括无毛刺时钟切换、寄存器分频时钟约束、虚拟时钟应用等关键技巧。通过真实工程示例,揭示常见约束误区及解决方案,帮助工程师提升时序收敛效率。特别针对input_delay和output_delay在多时钟域场景下的设置陷阱提供实用指南。

weixin_29306317的博客 430

Phoenix5.0 安装部署

现有hbase的查询工具有很多如:Hive,Tez,Impala,Shark/Spark,Phoenix等。phoenix是一个在hbase上面实现的基于hadoop的OLTP技术,具有低延迟、事务性、可使用sql、提供jdbc接口的特点。而且phoenix还提供了hbase二级索引的解决方案,丰富了hbase查询的多样性,继承了hbase海量数据快速随机查询的特点。但是在生产环境中,不可以用在OLTP中。

weixin_45131680的博客 877

Unity游戏开发客户端面经——数据结构(初级)

Unity游戏开发客户端面经 之 数据结构篇

Sea3752的博客 1万+

Phoenix整合HBase安装部署、HBase整合Phoenix安装部署

Phoenix整合HBase安装部署前言一、安装HBase、Phoenix二、安装1.HBase安装2.Phoenix安装3.部署1.配置HBase整合Phoenix分发集群文件启动验证总结 、 文章目录前言一、安装HBase、Phoenix二、安装1.HBase安装2.Phoenix安装3.部署1.配置HBase整合Phoenix分发集群文件启动验证总结 前言 相信很多小伙伴在安装phoenix连接hbase的时候被版本和配置问题困扰过,那么这篇文章可以很好的帮助你解决这个问题。 PS:HBase和

qq_40751062的博客 8489

静态时序分析(STA)基本概念

静态时序分析中常见概念(fasle Path,multicycle path等)

m0_61544122的博客 2256

【数据分析】宏基因组alpha和beta多样性分析和差异物种筛选案例

本教程为读者呈现了宏基因组α和β多样性分析的完整案例。通过实际数据和详细步骤,读者可以深入理解这两种分析方法及其在R语言中的实现过程。教程中不仅提供了清晰的代码示例,还对每一步骤进行了详细解释,使读者能够轻松跟随并掌握分析技巧。无论是初学者还是有一定基础的研究者,都能通过本教程快速上手宏基因组多样性分析,为微生物组学研究提供有力支持。

专注生信领域 2221

Phoenix安装部署

目录Phoenix安装部署配置 HBase 支持 Phoenix 创建二级索引 Phoenix安装部署 步骤 1: 下载 Phoenix http://archive.apache.org/dist/phoenix/apache-phoenix-4.14.2-HBase-1.3/ 步骤 2: 解压 jar 包 tar -zxvf apache-phoenix-4.14.2-HBase-1.3-bin.tar.gz -C /export/servers mv apache-phoenix-4.14.2-HBa

Knight 828

华为OD机试:查找接口成功率最优时间段 (C/C++/Py/Java/JS/GO)

华为OD机试双机位C卷 查找接口成功率最优时间段,提供C语言、Java、C++、Python、Go、JavaScript源码实现,并提供完整思路讲解。

qq_45776114的博客 4687

Phoenix概述及其安装部署

文章目录概述安装部署 概述 概念 Phoenix是一种提供SQL查询,将SQL转化为JavaAPI来操作Hbase的工具 功能 专门基于Hbase所涉及的SQL on Hbase 工具 使用Phoenix实现基于SQL操作Hbase 使用Phoenix自动构建二级索引并维护二级索引 原理 上层提供SQL接口 底层全部通过Hbase Java API来实现,通过构建一系列的Scan和Put来实现数据的读写 功能非常丰富 底层封装了大量的内置的协处理器,可以实现各种复杂的处理需求,例如二级索引

大数据跟我学i 331

YOLO11改进|卷积篇|引入多样化分支块DBB

YOLO11中添加多样化分支块DBB

A1983Z的博客 1450

深度学习工具包 DeepLearningToolbox

深度学习工具包Deprecation notice.-----This toolbox is outdated and no longer maintained.There are much better tools available for deep learning than this toolbox, e.g. [Theano](http://deeplearning.net/software/theano/), [torch](http://torch.ch/) or [tensorflow](http://www.tensorflow.org/)I would suggest you use one of the tools mentioned above rather than use this toolbox.Best, Rasmus.DeepLearnToolbox================A Matlab toolbox for Deep Learning.Deep Learning is a new subfield of machine learning that focuses on learning deep hierarchical models of data.It is inspired by the human brain's apparent deep (layered, hierarchical) architecture.A good overview of the theory of Deep Learning theory is[Learning Deep Architectures for AI](http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf)For a more informal introduction, see the following videos by Geoffrey Hinton and Andrew Ng.* [The Next Generation of Neural Networks](http://www.youtube.com/watch?v=AyzOUbkUf3M) (Hinton, 2007)* [Recent Developments in Deep Learning](http://www.youtube.com/watch?v=VdIURAu1-aU) (Hinton, 2010)* [Unsupervised Feature Learning and Deep Learning](http://www.youtube.com/watch?v=ZmNOAtZIgIk) (Ng, 2011)If you use this toolbox in your research please cite [Prediction as a candidate for learning deep hierarchical models of data](http://www2.imm.dtu.dk/pubdb/views/publication_details.php?id=6284)```@MASTERSTHESIS\{IMM2012-06284, author = "R. B. Palm", title = "Prediction as a candidate for learning deep hierarchical models of data", year = "2012",}```Contact: rasmusbergpalm at gmail dot comDirectories included in the toolbox-----------------------------------`NN/` - A library for Feedforward Backpropagation Neural Networks`CNN/` - A library for Convolutional Neural Networks`DBN/` - A library for Deep Belief Networks`SAE/` - A library for Stacked Auto-Encoders`CAE/` - A library for Convolutional Auto-Encoders`util/` - Utility functions used by the libraries`data/` - Data used by the examples`tests/` - unit tests to verify toolbox is workingFor references on each library check REFS.mdSetup-----1. Download.2. addpath(genpath('DeepLearnToolbox'));Example: Deep Belief Network---------------------```matlabfunction test_example_DBNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit RBM and visualize its weightsrand('state',0)dbn.sizes = [100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);figure; visualize(dbn.rbm{1}.W'); % Visualize the RBM weights%% ex2 train a 100-100 hidden unit DBN and use its weights to initialize a NNrand('state',0)%train dbndbn.sizes = [100 100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);%unfold dbn to nnnn = dbnunfoldtonn(dbn, 10);nn.activation_function = 'sigm';%train nnopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.10, 'Too big error');```Example: Stacked Auto-Encoders---------------------```matlabfunction test_example_SAEload mnist_uint8;train_x = double(train_x)/255;test_x = double(test_x)/255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit SDAE and use it to initialize a FFNN% Setup and train a stacked denoising autoencoder (SDAE)rand('state',0)sae = saesetup([784 100]);sae.ae{1}.activation_function = 'sigm';sae.ae{1}.learningRate = 1;sae.ae{1}.inputZeroMaskedFraction = 0.5;opts.numepochs = 1;opts.batchsize = 100;sae = saetrain(sae, train_x, opts);visualize(sae.ae{1}.W{1}(:,2:end)')% Use the SDAE to initialize a FFNNnn = nnsetup([784 100 10]);nn.activation_function = 'sigm';nn.learningRate = 1;nn.W{1} = sae.ae{1}.W{1};% Train the FFNNopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.16, 'Too big error');```Example: Convolutional Neural Nets---------------------```matlabfunction test_example_CNNload mnist_uint8;train_x = double(reshape(train_x',28,28,60000))/255;test_x = double(reshape(test_x',28,28,10000))/255;train_y = double(train_y');test_y = double(test_y');%% ex1 Train a 6c-2s-12c-2s Convolutional neural network %will run 1 epoch in about 200 second and get around 11% error. %With 100 epochs you'll get around 1.2% errorrand('state',0)cnn.layers = { struct('type', 'i') %input layer struct('type', 'c', 'outputmaps', 6, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %sub sampling layer struct('type', 'c', 'outputmaps', 12, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %subsampling layer};cnn = cnnsetup(cnn, train_x, train_y);opts.alpha = 1;opts.batchsize = 50;opts.numepochs = 1;cnn = cnntrain(cnn, train_x, train_y, opts);[er, bad] = cnntest(cnn, test_x, test_y);%plot mean squared errorfigure; plot(cnn.rL);assert(er<0.12, 'Too big error');```Example: Neural Networks---------------------```matlabfunction test_example_NNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);% normalize[train_x, mu, sigma] = zscore(train_x);test_x = normalize(test_x, mu, sigma);%% ex1 vanilla neural netrand('state',0)nn = nnsetup([784 100 10]);opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samples[nn, L] = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.08, 'Too big error');%% ex2 neural net with L2 weight decayrand('state',0)nn = nnsetup([784 100 10]);nn.weightPenaltyL2 = 1e-4; % L2 weight decayopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex3 neural net with dropoutrand('state',0)nn = nnsetup([784 100 10]);nn.dropoutFraction = 0.5; % Dropout fraction opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex4 neural net with sigmoid activation functionrand('state',0)nn = nnsetup([784 100 10]);nn.activation_function = 'sigm'; % Sigmoid activation functionnn.learningRate = 1; % Sigm require a lower learning rateopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex5 plotting functionalityrand('state',0)nn = nnsetup([784 20 10]);opts.numepochs = 5; % Number of full sweeps through datann.output = 'softmax'; % use softmax outputopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex6 neural net with sigmoid activation and plotting of validation and training error% split training data into training and validation datavx = train_x(1:10000,:);tx = train_x(10001:end,:);vy = train_y(1:10000,:);ty = train_y(10001:end,:);rand('state',0)nn = nnsetup([784 20 10]); nn.output = 'softmax'; % use softmax outputopts.numepochs = 5; % Number of full sweeps through dataopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, tx, ty, opts, vx, vy); % nntrain takes validation set as last two arguments (optionally)[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');```[![Bitdeli Badge](https://d2weczhvl823v0.cloudfront.net/rasmusbergpalm/deeplearntoolbox/trend.png)](https://bitdeli.com/free "Bitdeli Badge")

hbase-06 Phoenix安装部署&构建二级索引

对于HBase而言,如果想精确地定位到某行记录,唯一的办法是通过rowkey来查询。如果不通过rowkey来查找数据,就必须逐行地比较每一列的值,即全表扫瞄。 对于较大的表,全表扫描的代价是不可接受的。

我的博客 890

IMSL CNL 7.0.0 (x86-64)Crack

IMSL CNL 7.0.0 (x86-64)Crack intel的算法库破解文件,这是C++版本的破解文件

Hbase Phoenix简介及其安装部署

1. 什么是Phoenix? phoenix,中文译为“凤凰”,很美的名字。Phoenix是由saleforce.com开源的一个项目,后又捐给了Apache基金会。它相当于一个Java中间件,提供jdbc连接,操作hbase数据表。Phoenix是一个HBase的开源SQL引擎。你可以使用标准的JDBC API代替HBase客户端API来创建表,插入数据,查询你的HBase数据。Phoenix...

Transkai 321

OV9734 DATASHEET,完整规格书

OV9734完整规格书,包括电气参数,尺寸规格,寄存器配置表等

Outlook Express 6 (OE6) for Win7/Win8/Win10

英文版 Outlook Express6.0, 兼容Win7/Win10/WinXP,解压就可用。

Phoenix的简介以及安装部署

1.什么是Phoenix? phoenix,中文译为“凤凰”,很美的名字。Phoenix是由saleforce.com开源的一个项目,后又捐给了Apache基金会。它相当于一个Java中间件,提供jdbc连接,操作hbase数据表。Phoenix是一个HBase的开源SQL引擎。你可以使用标准的JDBC API代替HBase客户端API来创建表,插入数据,查询你的HBase数据。Phoenix的团...

布里啾啾迪布利多的博客 3707

hbase学习笔记-phoenix安装部署

phoenix安装部署 准备 需要先安装好hbase集群,phoenix只是一个工具,只需要在一台机器上安装就可以了 1、下载安装包 从对应的地址下载:http://mirrors.cnnic.cn/apache/phoenix/ 这里我们使用的是 phoenix-4.8.2-HBase-1.2-bin.tar.gz 2、上传解压 tar -zxvf phoenix-4.8.2-HB...

陈同学 303

2022-09-06 Phoenix安装部署

Phoenix安装部署

D___H的博客 1048
上一篇: [一起学Hive]之五-Hive的视图和分区
下一篇: Ambari 监控信息的提取
王树民
博客等级 码龄10年 912粉丝 560原创
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付:10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值