hbase client api 例子-2

更新hbase client api 2.0.1记录 今天更新hbase-client版本发现原本使用的utils类提示方法废弃,所以查询新的api文档发现确实做了变更,而且以前1.x用的方法将会在以后3.x版本删除; 更新的版本maven:   <dependency> <groupId>org.apache.hbase</groupId> <artifactId>hbase-... 阅读详情

发布一个k8s部署视频:https://edu.youkuaiyun.com/course/detail/26967

课程内容:各种k8s部署方式。包括minikube部署,kubeadm部署,kubeasz部署,rancher部署,k3s部署。包括开发测试环境部署k8s,和生产环境部署k8s。

腾讯课堂连接地址https://ke.qq.com/course/478827?taid=4373109931462251&tuin=ba64518

第二个视频发布  https://edu.youkuaiyun.com/course/detail/27109

腾讯课堂连接地址https://ke.qq.com/course/484107?tuin=ba64518

介绍主要的k8s资源的使用配置和命令。包括configmap,pod,service,replicaset,namespace,deployment,daemonset,ingress,pv,pvc,sc,role,rolebinding,clusterrole,clusterrolebinding,secret,serviceaccount,statefulset,job,cronjob,podDisruptionbudget,podSecurityPolicy,networkPolicy,resourceQuota,limitrange,endpoint,event,conponentstatus,node,apiservice,controllerRevision等。

第三个视频发布:https://edu.youkuaiyun.com/course/detail/27574

详细介绍helm命令,学习helm chart语法,编写helm chart。深入分析各项目源码,学习编写helm插件
————————————————------------------------------------------------------------------------------------------------------------------

 

package example;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.HColumnDescriptor;
import org.apache.hadoop.hbase.HTableDescriptor;
import org.apache.hadoop.hbase.client.HBaseAdmin;
import org.apache.hadoop.hbase.client.HTable;
import org.apache.hadoop.hbase.util.Bytes;
import org.apache.hadoop.hbase.util.Pair;

//Administrator
public class AdvancedCreateTableExample {
	public static void printRegion(String tableName)throws Exception{
		System.out.println("tableName:"+tableName);
		Configuration conf=HBaseConfiguration.create();
		HTable table=new HTable(conf,tableName);
		Pair<byte[][],byte[][]> pair=table.getStartEndKeys();
		byte[][]firsts=pair.getFirst();
		for(int i=0;i<firsts.length;i++){
			if(firsts[i].length==8){
				System.out.println(Bytes.toLong(firsts[i])+"--------"+Bytes.toLong(pair.getSecond()[i]));
			}else{
				System.out.println(Bytes.toStringBinary(firsts[i])+"--------"+Bytes.toStringBinary(pair.getSecond()[i]));
			}
			
		}
	}
	/**
	 * @param args
	 */
	public static void main(String[] args) throws Exception{
		Configuration conf=HBaseConfiguration.create();
		HBaseAdmin admin=new HBaseAdmin(conf);
		HTableDescriptor tabDesc=new HTableDescriptor(Bytes.toBytes("test13"));
		HColumnDescriptor colDesc=new HColumnDescriptor("fam1");
		tabDesc.addFamily(colDesc);
		admin.createTable(tabDesc,Bytes.toBytes(1L),Bytes.toBytes(100L),10);
		printRegion("test13");
		HTableDescriptor tabDesc2=new HTableDescriptor(Bytes.toBytes("test14"));
		HColumnDescriptor colDesc2=new HColumnDescriptor("fam1");
		tabDesc2.addFamily(colDesc2);
		byte [][]spilit=new byte[][]{
				Bytes.toBytes("A"),
				Bytes.toBytes("B"),
				Bytes.toBytes("C"),
				Bytes.toBytes("D")
		};
		admin.createTable(tabDesc2, spilit);
		printRegion("test14");

	}

}
package example;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.HColumnDescriptor;
import org.apache.hadoop.hbase.HTableDescriptor;
import org.apache.hadoop.hbase.client.HBaseAdmin;

//Administrator
public class CreateTableExample {

	/**
	 * @param args
	 */
	public static void main(String[] args) throws Exception{
		Configuration conf=HBaseConfiguration.create();
		HBaseAdmin admin=new HBaseAdmin(conf);
		HTableDescriptor tableDesc=new HTableDescriptor("test2");
		HColumnDescriptor colDesc=new HColumnDescriptor("col1");
		tableDesc.addFamily(colDesc);
		admin.createTable(tableDesc);
		System.out.println(admin.isTableAvailable("test2"));
	}

}

 

package example;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.client.HBaseAdmin;

//Administrator
public class DDEExample {

	/**
	 * @param args
	 */
	public static void main(String[] args) throws Exception{
		Configuration conf=HBaseConfiguration.create();
		HBaseAdmin admin=new HBaseAdmin(conf);
		System.out.println("available:"+admin.isTableAvailable("test9"));
		System.out.println("enalbe:"+admin.isTableEnabled("test9"));
		System.out.println("disable:"+admin.isTableDisabled("test9"));
		System.out.println("------------");
		admin.disableTable("test9");
		System.out.println("available:"+admin.isTableAvailable("test9"));
		System.out.println("enalbe:"+admin.isTableEnabled("test9"));
		System.out.println("disable:"+admin.isTableDisabled("test9"));
		System.out.println("------------");
		admin.deleteTable("test9");
		System.out.println("available:"+admin.isTableAvailable("test9"));
		System.out.println("------------");
		System.out.println("exist:"+admin.tableExists("test9"));

	}

}

 

package example;
import java.util.ArrayList;
import java.util.List;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.KeyValue;
import org.apache.hadoop.hbase.client.HTable;
import org.apache.hadoop.hbase.client.Result;
import org.apache.hadoop.hbase.client.ResultScanner;
import org.apache.hadoop.hbase.client.Scan;
import org.apache.hadoop.hbase.filter.BinaryComparator;
import org.apache.hadoop.hbase.filter.CompareFilter.CompareOp;
import org.apache.hadoop.hbase.filter.Filter;
import org.apache.hadoop.hbase.filter.FilterList;
import org.apache.hadoop.hbase.filter.FilterList.Operator;
import org.apache.hadoop.hbase.filter.QualifierFilter;
import org.apache.hadoop.hbase.filter.RowFilter;
import org.apache.hadoop.hbase.filter.SubstringComparator;
import org.apache.hadoop.hbase.filter.ValueFilter;
import org.apache.hadoop.hbase.util.Bytes;

//Administrator
public class FilterListExample {

	/**
	 * @param args
	 */
	public static void main(String[] args) throws Exception{
		Configuration conf=HBaseConfiguration.create();
		HTable table=new HTable(conf,"testtable");
		List<Filter> filterList=new ArrayList<Filter>();
		Filter rowFilter=new RowFilter(CompareOp.EQUAL, new SubstringComparator("6"));
		filterList.add(rowFilter);
		Filter columnFilter=new QualifierFilter(CompareOp.NOT_EQUAL,  new SubstringComparator("6"));
		filterList.add(columnFilter);
		Filter valueFilter=new ValueFilter(CompareOp.LESS, new BinaryComparator(Bytes.toBytes("value8")));
		filterList.add(valueFilter);
		FilterList filters=new FilterList(filterList);
		Scan scan=new Scan();
		scan.setFilter(filters);
		ResultScanner resultScanner=table.getScanner(scan);
		for(Result result:resultScanner){
			for(KeyValue kv:result.raw()){
				System.out.println(kv+"--------"+Bytes.toString(kv.getValue()));
			}
		}
		resultScanner.close();
		System.out.println("------------");
		filters=new FilterList(Operator.MUST_PASS_ONE,filterList);
		scan.setFilter(filters);
		 resultScanner=table.getScanner(scan);
		for(Result result:resultScanner){
			for(KeyValue kv:result.raw()){
				System.out.println(kv+"--------"+Bytes.toString(kv.getValue()));
			}
		}
		resultScanner.close();

	}

}

 

package example;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.client.HTable;
import org.apache.hadoop.hbase.util.Bytes;

//Administrator
public class IncrementExample {

	/**
	 * @param args
	 */
	public static void main(String[] args) throws Exception{
		Configuration conf=HBaseConfiguration.create();
		HTable table=new HTable(conf,"testtable");
		long l=table.incrementColumnValue(Bytes.toBytes("row9"), Bytes.toBytes("family1"), Bytes.toBytes("inc"), 1);
		System.out.println(l);
		 l=table.incrementColumnValue(Bytes.toBytes("row9"), Bytes.toBytes("family1"), Bytes.toBytes("inc"), 0);
		System.out.println(l);
		l=table.incrementColumnValue(Bytes.toBytes("row9"), Bytes.toBytes("family1"), Bytes.toBytes("inc"), -1);
		System.out.println(l);

	}

}

 

package example;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.HTableDescriptor;
import org.apache.hadoop.hbase.client.HBaseAdmin;
import org.apache.hadoop.hbase.util.Bytes;

//Administrator
public class ListTableExample {

	/**
	 * @param args
	 */
	public static void main(String[] args)throws Exception {
		Configuration conf=HBaseConfiguration.create();
		HBaseAdmin admin=new HBaseAdmin(conf);
		HTableDescriptor[] descList=admin.listTables();
		for(HTableDescriptor desc:descList){
			System.out.println(Bytes.toString(desc.getName()));
		}
		HTableDescriptor desc=admin.getTableDescriptor(Bytes.toBytes("testtable"));
		System.out.println(Bytes.toString(desc.getName()));
	}

}

 

package example;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.HColumnDescriptor;
import org.apache.hadoop.hbase.HTableDescriptor;
import org.apache.hadoop.hbase.client.HBaseAdmin;
import org.apache.hadoop.hbase.util.Bytes;

//Administrator
public class ModifyTableExample {

	/**
	 * @param args
	 */
	public static void main(String[] args)throws Exception {
		Configuration conf=HBaseConfiguration.create();
		HBaseAdmin admin=new HBaseAdmin(conf);
		HTableDescriptor desc=new HTableDescriptor("modify");
		HColumnDescriptor colDesc=new HColumnDescriptor("fam1");
		desc.addFamily(colDesc);
		admin.createTable(desc);
		HTableDescriptor desc2=admin.getTableDescriptor(Bytes.toBytes("modify"));
		HColumnDescriptor colDesc2=new HColumnDescriptor("fami2");
		desc2.addFamily(colDesc2);
		admin.disableTable("modify");
		admin.modifyTable("modify", desc2);
		admin.enableTable("modify");
		HTableDescriptor desc3=admin.getTableDescriptor(Bytes.toBytes("modify"));
		System.out.println("is equal:"+desc2.equals(desc3));
		System.out.println(desc3);
	}

}

 

package example;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.HColumnDescriptor;
import org.apache.hadoop.hbase.client.HBaseAdmin;
import org.apache.hadoop.hbase.util.Bytes;

//Administrator
public class ModifyTableExample2 {

	/**
	 * @param args
	 */
	public static void main(String[] args)throws Exception {
		Configuration conf=HBaseConfiguration.create();
		HBaseAdmin admin=new HBaseAdmin(conf);
		admin.enableTable("test8");
		admin.disableTable("test8");
		admin.deleteColumn("test8", "fam1");
		admin.enableTable("test8");
		System.out.println(admin.getTableDescriptor(Bytes.toBytes("test8")));
		admin.disableTable("test8");
		admin.addColumn("test8", new HColumnDescriptor("fam1"));
		System.out.println(admin.getTableDescriptor(Bytes.toBytes("test8")));

	}

}

 

package example;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.KeyValue;
import org.apache.hadoop.hbase.client.HTable;
import org.apache.hadoop.hbase.client.Increment;
import org.apache.hadoop.hbase.client.Result;
import org.apache.hadoop.hbase.util.Bytes;

//Administrator
public class MultiIncrementExample {

	/**
	 * @param args
	 */
	public static void main(String[] args)throws Exception {
		Configuration conf=HBaseConfiguration.create();
		HTable table=new HTable(conf,"testtable");
		Increment increment=new Increment(Bytes.toBytes("row9"));
		increment.addColumn(Bytes.toBytes("family1"), Bytes.toBytes("inc"), 0);
		increment.addColumn(Bytes.toBytes("family1"), Bytes.toBytes("inc1"), -1);
		increment.addColumn(Bytes.toBytes("family1"), Bytes.toBytes("inc2"), 2);
		
		Result result=table.increment(increment);
		for(KeyValue kv:result.raw()){
			System.out.println(kv+"-----"+Bytes.toLong(kv.getValue()));
		}

	}

}

 

package example;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.client.HTableInterface;
import org.apache.hadoop.hbase.client.HTablePool;

//Administrator
public class TablePoolExample {

	/**
	 * @param args
	 */
	public static void main(String[] args) throws Exception{
		Configuration conf=HBaseConfiguration.create();
		HTablePool pool=new HTablePool(conf,5);
		HTableInterface [] inter=new HTableInterface[10];
		for(int i=0;i<inter.length;i++){
			inter[i]=pool.getTable("testtable");
		}
		for(int i=0;i<inter.length;i++){
			pool.putTable(inter[i]);
		}
		
		pool.closeTablePool("testtable");

	}

}


 

HBase之Java API 1.Configuration 在使用Java API时,Client端需要知道HBase的配置环境,如存储地址,zookeeper等信息。这些信息通过Configuration对象来封装,可通过如下代码构建该对象         Configuration config=HBaseConfiguration.create(); 在调用HBaseConfiguration.create( 阅读详情

相关推荐

手把手教你用MATLAB和PSIM搞定Buck电路控制器:从传递函数到运放电路实战

本文详细介绍了如何使用MATLAB和PSIM协同设计Buck电路控制器,从传递函数推导到运放电路实现的完整流程。通过小信号建模、波特图分析和双平台仿真验证,帮助电力电子工程师解决理论到实践的转化难题,提升Buck电路控制器设计的效率和可靠性。

weixin_29271053的博客 269

HBase client API 实例

创建Maven  Demo项目,在pom.xml中引入HBase Client jar包: org.apache.hbase hbase-client 1.1.2 commons-logging

随便写点的专栏 1901

从信号到振镜:STM32F103 + XY2-100协议 + AM26LS31芯片的激光打标/雕刻系统信号链搭建指南

本文详细介绍了基于STM32F103微控制器和XY2-100协议的激光打标/雕刻系统信号链搭建方法。从数字振镜控制协议解析、AM26LS31差分驱动电路设计到信号完整性优化,提供了一套完整的解决方案,帮助工程师实现高精度激光控制系统的开发与调试。

weixin_42536058的博客 237

HBase 创建表时的预分区

如果知道hbase数据表的key的分布情况,就可以在建表的时候对hbase进行region的预分区。这样做的好处是防止大数据量插入的热点问题,提高数据插入的效率。 背景:HBase默认建表时有一个region,这个region的rowkey是没有边界的,即没有startkey和endkey,在数据写入时,所有数据都会写入这个默认的region,随着数据量的不断 增加,此region已经不能承受不...

u012957549的博客 1万+

【全志V821_FoxPi】6-3 GC2083 MIPI摄像头适配

本文介绍了在全志V821芯片上适配GC2083 MIPI摄像头的方案。GC2083是一款支持1080P@30fps的高性能CMOS图像传感器,采用MIPI CSI-2接口,具有低功耗特性。文章详细列出了传感器的技术参数和工作电压要求,并提供了设备树配置示例,包括CSI接口、ISP处理单元和传感器节点的具体设置。通过快速配置命令one_gc2083_sensor或手动修改设备树即可启用该摄像头模组,适用于IPC等对图像质量有要求的嵌入式应用场景。

第四维度的博客 1930

零基础实战:用 Docker 和 vLLM 本地部署 bge-reranker-v2-m3 重排序模型

本文介绍了如何利用Docker容器和vLLM推理框架在本地部署BAAI开发的bge-reranker-v2-m3文本排序模型。

举世誉之而不加劝,举世非之而不加沮,定乎内外之分,辩乎荣辱之境,斯已矣。 5972

源码解读--(3)hbase-examples MultiThreadedClientExample

地址:http://aperise.iteye.com/blog/2372534源码解读--(1)hbase客户端源代码http://aperise.iteye.com/blog/2372350源码解读--(2)hbase-examples BufferedMutator Examplehttp://aperise.iteye.com/blog/2372505源码解读--(3)hbase-exam...

路漫漫其修远兮 622

openmediavault(OMV) (1)安装部署

EXT3(Third Extended File System): EXT3 是 Linux 中广泛使用的日志文件系统,是 EXT2 文件系统的升级版本。这里可以根据自己的需求进行设置,目前只设置了自动登出为1天,为方便后续配置使用,其他设置可根据自己的具体需求进行设置。通过cd 命令进入到我们的raid存储中,通过ls发现里面没有任何的数据,因为我们的raid存储中还未创建任何数据。我的系统要安装在100G的磁盘上,故选择100G的磁盘,实际可以根据具体情况选择。

Monster✺◟(∗❛ัᴗ❛ั∗)◞✺的博客 2168

Hbase-client客户端操作模板

2019独角兽企业重金招聘Python工程师标准>>> ...

weixin_33949359的博客 1632

Linux(31)Rockchip RK3568 Ubuntu22.04上部署 Docker: 问题与解决方案

本博客在总结这些问题及其解决方案,为遇到类似问题的兄弟提供参考,如果我后续有碰到其他Docker 问题也会在此更新补充。

我其实什么都不会 1646

ESP32 DW1000 UWB 室内定位系统

详细介绍ESP32 DW1000 UWB 室内定位系统的设计演示demo。

DT程序员的博客 1662

hbase java client 简介_HBase Client JAVA API

旧 的 HBase 接口逻辑与传统 JDBC 方式很不相同,新的接口与传统 JDBC 的逻辑更加相像,具有更加清晰的 Connection 管理方式。同时,在旧的接口中,客户端何时将 Put 写到服务端也需要设置,一个 Put 马上写到服务端,还是攒到一批写到服务端,新用户往往对此不太清楚。在新的接口中,引入了 BufferedMutator,可以提供更加高效清晰的写操作。HBase 0.98 与...

weixin_39574869的博客 325

数据处理和分析之数据聚类:层次聚类(Hierarchical Clustering):单链接层次聚类方法

单链接层次聚类(Single-Linkage Hierarchical Clustering)是凝聚型层次聚类的一种,它基于数据点之间的最小距离来定义聚类间的相似性。在单链接层次聚类中,两个聚类之间的距离定义为这两个聚类中任意两个数据点之间的最短距离。这种聚类方法倾向于形成链状或线性的聚类,因为聚类的边界是由最远的数据点决定的,而不是聚类的平均或中心位置。单链接层次聚类通过计算簇间最近点的距离来决定簇的合并,这种方法简单直观,但可能导致簇的形状不规则,尤其是在数据点分布不均匀的情况下。

2401_87715305的博客 1443

Hbase入门之三HBase Client API使用入门1

Hbase的基本介绍   今天公司进行了一样关于Hbase的培训,让我对Hbase有了基础的了解。和大家分享一样。自己的理解不是很深,如果有说的不对地方,希望大家斧正。   什么是Hbase?   HBase是一个分布式的、面向列的开源数据库。   分布式:Hbase的分布式是依托于Hadoop的,那Hadoop又是什么?Hadoop 是一个能够对大量数据进行分布式处理的软

云智禅师的专栏 2953

OV9734 DATASHEET,完整规格书

OV9734完整规格书,包括电气参数,尺寸规格,寄存器配置表等

数学建模模型总结

这是本人在两年内准备参加数学建模竞赛搜集到的模型资料,已经经过了本人的筛选和整理,希望能够对准备参加数学建模竞赛的学弟学妹们提供一些帮助!

HBase Java API: hbase-client

一:简介 hbase-client 是hbase的api 二:API package org.apache.hadoop.hbase.client; public class ConnectionFactory { /* * 创建连接 * @param conf hbase属性配置 */ public static Connection createConnection(Configu...

人不风流枉少年 1万+

深度学习工具包 DeepLearningToolbox

深度学习工具包Deprecation notice.-----This toolbox is outdated and no longer maintained.There are much better tools available for deep learning than this toolbox, e.g. [Theano](http://deeplearning.net/software/theano/), [torch](http://torch.ch/) or [tensorflow](http://www.tensorflow.org/)I would suggest you use one of the tools mentioned above rather than use this toolbox.Best, Rasmus.DeepLearnToolbox================A Matlab toolbox for Deep Learning.Deep Learning is a new subfield of machine learning that focuses on learning deep hierarchical models of data.It is inspired by the human brain's apparent deep (layered, hierarchical) architecture.A good overview of the theory of Deep Learning theory is[Learning Deep Architectures for AI](http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf)For a more informal introduction, see the following videos by Geoffrey Hinton and Andrew Ng.* [The Next Generation of Neural Networks](http://www.youtube.com/watch?v=AyzOUbkUf3M) (Hinton, 2007)* [Recent Developments in Deep Learning](http://www.youtube.com/watch?v=VdIURAu1-aU) (Hinton, 2010)* [Unsupervised Feature Learning and Deep Learning](http://www.youtube.com/watch?v=ZmNOAtZIgIk) (Ng, 2011)If you use this toolbox in your research please cite [Prediction as a candidate for learning deep hierarchical models of data](http://www2.imm.dtu.dk/pubdb/views/publication_details.php?id=6284)```@MASTERSTHESIS\{IMM2012-06284, author = "R. B. Palm", title = "Prediction as a candidate for learning deep hierarchical models of data", year = "2012",}```Contact: rasmusbergpalm at gmail dot comDirectories included in the toolbox-----------------------------------`NN/` - A library for Feedforward Backpropagation Neural Networks`CNN/` - A library for Convolutional Neural Networks`DBN/` - A library for Deep Belief Networks`SAE/` - A library for Stacked Auto-Encoders`CAE/` - A library for Convolutional Auto-Encoders`util/` - Utility functions used by the libraries`data/` - Data used by the examples`tests/` - unit tests to verify toolbox is workingFor references on each library check REFS.mdSetup-----1. Download.2. addpath(genpath('DeepLearnToolbox'));Example: Deep Belief Network---------------------```matlabfunction test_example_DBNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit RBM and visualize its weightsrand('state',0)dbn.sizes = [100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);figure; visualize(dbn.rbm{1}.W'); % Visualize the RBM weights%% ex2 train a 100-100 hidden unit DBN and use its weights to initialize a NNrand('state',0)%train dbndbn.sizes = [100 100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);%unfold dbn to nnnn = dbnunfoldtonn(dbn, 10);nn.activation_function = 'sigm';%train nnopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.10, 'Too big error');```Example: Stacked Auto-Encoders---------------------```matlabfunction test_example_SAEload mnist_uint8;train_x = double(train_x)/255;test_x = double(test_x)/255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit SDAE and use it to initialize a FFNN% Setup and train a stacked denoising autoencoder (SDAE)rand('state',0)sae = saesetup([784 100]);sae.ae{1}.activation_function = 'sigm';sae.ae{1}.learningRate = 1;sae.ae{1}.inputZeroMaskedFraction = 0.5;opts.numepochs = 1;opts.batchsize = 100;sae = saetrain(sae, train_x, opts);visualize(sae.ae{1}.W{1}(:,2:end)')% Use the SDAE to initialize a FFNNnn = nnsetup([784 100 10]);nn.activation_function = 'sigm';nn.learningRate = 1;nn.W{1} = sae.ae{1}.W{1};% Train the FFNNopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.16, 'Too big error');```Example: Convolutional Neural Nets---------------------```matlabfunction test_example_CNNload mnist_uint8;train_x = double(reshape(train_x',28,28,60000))/255;test_x = double(reshape(test_x',28,28,10000))/255;train_y = double(train_y');test_y = double(test_y');%% ex1 Train a 6c-2s-12c-2s Convolutional neural network %will run 1 epoch in about 200 second and get around 11% error. %With 100 epochs you'll get around 1.2% errorrand('state',0)cnn.layers = { struct('type', 'i') %input layer struct('type', 'c', 'outputmaps', 6, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %sub sampling layer struct('type', 'c', 'outputmaps', 12, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %subsampling layer};cnn = cnnsetup(cnn, train_x, train_y);opts.alpha = 1;opts.batchsize = 50;opts.numepochs = 1;cnn = cnntrain(cnn, train_x, train_y, opts);[er, bad] = cnntest(cnn, test_x, test_y);%plot mean squared errorfigure; plot(cnn.rL);assert(er<0.12, 'Too big error');```Example: Neural Networks---------------------```matlabfunction test_example_NNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);% normalize[train_x, mu, sigma] = zscore(train_x);test_x = normalize(test_x, mu, sigma);%% ex1 vanilla neural netrand('state',0)nn = nnsetup([784 100 10]);opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samples[nn, L] = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.08, 'Too big error');%% ex2 neural net with L2 weight decayrand('state',0)nn = nnsetup([784 100 10]);nn.weightPenaltyL2 = 1e-4; % L2 weight decayopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex3 neural net with dropoutrand('state',0)nn = nnsetup([784 100 10]);nn.dropoutFraction = 0.5; % Dropout fraction opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex4 neural net with sigmoid activation functionrand('state',0)nn = nnsetup([784 100 10]);nn.activation_function = 'sigm'; % Sigmoid activation functionnn.learningRate = 1; % Sigm require a lower learning rateopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex5 plotting functionalityrand('state',0)nn = nnsetup([784 20 10]);opts.numepochs = 5; % Number of full sweeps through datann.output = 'softmax'; % use softmax outputopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex6 neural net with sigmoid activation and plotting of validation and training error% split training data into training and validation datavx = train_x(1:10000,:);tx = train_x(10001:end,:);vy = train_y(1:10000,:);ty = train_y(10001:end,:);rand('state',0)nn = nnsetup([784 20 10]); nn.output = 'softmax'; % use softmax outputopts.numepochs = 5; % Number of full sweeps through dataopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, tx, ty, opts, vx, vy); % nntrain takes validation set as last two arguments (optionally)[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');```[![Bitdeli Badge](https://d2weczhvl823v0.cloudfront.net/rasmusbergpalm/deeplearntoolbox/trend.png)](https://bitdeli.com/free "Bitdeli Badge")

Java API操作 HBase

导入数据问题 Rowkey问题 Java API操作HBase Ref: HBase Java API之TableDescriptorBuilder,ColumnDescriptorBuilder创建表、列族_追梦*小生的博客-优快云博客 JAVA连接HBase客户端及HBase写入数据和读取数据原理解析_菜鸟周星星的博客-优快云博客 浅谈HBase的数据分布 - 知乎 Hbase API以及java如何操作Hbase的说明_Uncle_Mo的博客-优快云博客_hbase j

noobiee的博客 4605

霍尼韦尔SoftMaster使用说明

霍尼韦尔SoftMaster使用说明

Hbase-APi Client的简单操作

官网是最好的学习材料,请大家移步官网HbaseApi import java.io.IOException; import org.apache.hadoop.hbase.HBaseConfiguration; import org.apache.hadoop.hbase.TableName; import org.apache.hadoop.hbase.client.Connection; import org.apache.hadoop.hbase.client.ConnectionFactory;

weixin_41794129的博客 272

推荐使用Golang HBase Client:纯Go语言编写的高效HBase接口

推荐使用Golang HBase Client:纯Go语言编写的高效HBase接口 在大数据处理领域中,Apache HBase作为一款强大的NoSQL数据库,以其分布式存储和实时查询的特性赢得了广泛的应用。对于Golang开发者来说,我们很高兴向您推荐一个强大的HBase客户端库——Golang HBase client。这个开源项目由(tsuna/gohbase)[https://github...

gitblog_00875的博客 742

HBase常用API操作

package test; import java.io.IOException; import java.text.DecimalFormat; import java.util.ArrayList; import java.util.List; import javax.print.attribute.ResolutionSyntax; import org.apache.hadoop....

weixin_43695091的博客 5300

Hbase 最新版的 Client DDL API 示例

package HbaseDemo; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.hbase.HBaseConfiguration; import org.apache.hadoop.hbase.TableName; import org.apache.hadoop.hbase.client.*; i...

李昊轩的博客 1078

JAVA api 远程hbase出错 org.apache.hadoop.hbase.client.RetriesExhaustedException

JAVA api 远程hbase出错 org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=36, exceptions: Tue Jul 19 16:36:05 CST 2016, null, java.net.SocketTimeoutException: callTimeout=60000, callDuration=79721: row 'testtable,,' on table 'h

chen798213337的博客 6万+

hbase的api操作

hbase的api操作TestHbaseConnectionTestHbaseDDLTestHbaseDMLTestHbaseFilter总结 TestHbaseConnection 两大对象: HbaseConfiguration: hbase的加载配置文件的对象,用于加载默认配置文件 hbase-dafult.xml Connection: hbase的连接对象 package hba...

u013111855的博客 2492

HBase Client JAVA API

旧 的 HBase 接口逻辑与传统 JDBC 方式很不相同,新的接口与传统 JDBC 的逻辑更加相像,具有更加清晰的 Connection 管理方式。 同时,在旧的接口中,客户端何时将 Put 写到服务端也需要设置,一个 Put 马上写到服务端,还是攒到一批写到服务端,新用户往往对此不太清楚。 在新的接口中,引入了 BufferedMutator,可以提供更加高效清晰的写操作。...

weixin_30565199的博客 159

hbase-client java API 操作

spring boot集成hbase-client 参考上文使用spring-boot-starter-hbase和RowMapper. @Autowired private HbaseTemplate hbaseTemplate; 创建表 /** * 创建表 * @return * @throws IOException */ public String createTable() thr...

Happywzy~的博客 544

HBase client api例子集合-1

import java.util.ArrayList; import java.util.List; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.hbase.HBaseConfiguration; import org.apache.hadoop.hbase.client.Delete; import

mark's technic world 3005

HBase Client API简单

package leap.dao.hbase.impl; import leap.dao.hbase.HbaseDao; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.hbase.HBaseConfiguration; import org.apache.hadoop.hbase.HColumnDesc

^_^TaeYang-Jade^_^ 530
上一篇: HBase client api例子集合-1
下一篇: 编译hadoop eclipse 2.x插件
hxpjava1
博客等级 码龄19年 1365粉丝 511原创
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付:10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

hxpjava1

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值