基于管道的即时通讯(java nio)

本文深入解析了一种基于管道的通信机制,该机制使服务层能够与底层socket解耦,通过数据池实现数据的自动传递。文章详细介绍了客户端和服务端的实现方式,包括消息的发送、接收、处理流程,以及过滤器的作用。

此项目实现原理

sevice只需往管道中(数据池)中发送数据,等到池中有数据了,它自动会找你。你不必要关心数据怎么发送与接收,只需要关注你业务的处理。
如下图


优点:

基于管道的实现是消息的发送或接受只需要发送到管道或者从管道读取,而不用关注如何通过Channer发送,这样则实现了service层与socket的解耦。
依赖于广播而不依赖于回调函数,与nio的异步非阻塞,真正实现线程的零等待。

缺点:

发送的数据很难通过会掉函数实现(或者根本不能),只能通过广播实现。

相关类介绍

ClientMessagePool,ServiceMessagePool管道(数据池)
内部实现原理是一个链表队列,数据的增加读取对应队列中的压入队列,读取队列头元素

Sevice 
业务逻辑处理类,必须实现IMessageSevice接口,并向MessageObserver注册
MessageObserver
内部有一个IMessageSevice的链表,保存各个实现IMessageSevice接口的Service,与Sevice 构成观察者模式,
会有一个线程专门监测MessagePool,一旦有数据,就交给MessageObserver。MessageObserver根据特定消息类推送给制定的service.
SocketChannel
实现了一个SockenChannel的类,相当于一个客户端。从管道中(ClientMessagePool)中读取数据,一旦有数据,则将数据写入管道
Selector
接收管道的注册,并根纷发条件,向指定的SocketChannel推动数据。也会根据过滤条件过滤数据。

代码实现

管道代码实现
package com.pool;

import java.util.Queue;
import java.util.concurrent.LinkedBlockingQueue;

public class MessagePool {
	public static Queue<String> clintmessageQueue = new LinkedBlockingQueue<String>(); 
	public static Queue<String> serverMessageQueue = new LinkedBlockingQueue<String>(); 
	
}
接口
package com.pool;

public interface IMessagePool {

	public void addMessage(String message);

	public String pollMessage();
	
	public boolean isEmpty();

}


实现类
package com.pool.impl;

import com.pool.IMessagePool;
import com.pool.MessagePool;

public class ClientMessagePool implements IMessagePool {
	@Override
	public  void addMessage(String message) {
		MessagePool.clintmessageQueue.add(message);
	}
	@Override
	public  String pollMessage() {
		return MessagePool.clintmessageQueue.poll();
	}
	
	@Override
	public boolean isEmpty() {
		if(MessagePool.clintmessageQueue.size()>0)
			return true;
		else
			return false;
	}
}

客户端
package com.socket;

import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.SocketChannel;
import java.util.Iterator;

import org.apache.commons.lang.ArrayUtils;

import com.pool.IMessagePool;
import com.pool.impl.ClientMessagePool;
import com.util.PackageUtil;

public class MySocket {
	private SocketChannel mSocketChannel;

	private SelectionKey key;

	public static String CHARSET = "utf-8";

	public static String ADDRESS = "127.0.0.1";

	public static int HOST = 34521;

	protected Selector mSelector;

	protected IMessagePool messagePool = new ClientMessagePool();;

	ByteBuffer buffer;

	public MySocket() {
		try {
			mSelector = Selector.open();
			initSocketChannel();
			initBassiness();
		} catch (Exception e) {
			e.printStackTrace();
		} finally {
			try {
				key.channel().close();
			} catch (IOException e) {
				e.printStackTrace();
			}
		}

	}

	/**
	 * 业务逻辑
	 * 
	 * @throws Exception
	 */
	private void initBassiness() throws Exception {
		while (true) {
			checkWriteable();
			// 瞬时检测
			if (mSelector.select(100) > 0) {
				Iterator<SelectionKey> keys = mSelector.selectedKeys()
						.iterator();
				while (keys.hasNext()) {
					SelectionKey key = keys.next();
					if (key.isReadable()) {
						dispose4Readable(key);
					}
					if (key.isValid() && key.isWritable()) {
						dispose4Writable(key);
					}
					keys.remove();
				}
			}
		}
	}

	/**
	 * 可读请求
	 * 
	 * @param key
	 * @throws Exception
	 */
	protected void dispose4Readable(SelectionKey key) throws Exception {
		SocketChannel mSocketChannel = ((SocketChannel) key.channel());
		buffer = ByteBuffer.allocate(1024);
		mSocketChannel.read(buffer);
		buffer.flip();
		this.unPacket(buffer.array(), key);
	}

	/**
	 * 可写请求
	 * 
	 * @param key
	 * @throws Exception
	 */
	protected void dispose4Writable(SelectionKey key) throws Exception {

		SocketChannel mSocketChannel = ((SocketChannel) key.channel());
		int value = 0;
		do{
			value = mSocketChannel.write(buffer);
		}while(value!=0);
		key.interestOps(key.interestOps() & ~SelectionKey.OP_WRITE);
		key.interestOps(SelectionKey.OP_READ);
	}

	/**
	 * 解包
	 * 
	 * @param buf
	 * @return
	 */
	public byte[] unPacket(byte[] buf, SelectionKey key) {

		int len = buf.length;// 37
		int i;
		for (i = 0; i < len; i++) {
			if (len < i + PackageUtil.PACKAGEHEADERLENGTH
					+ PackageUtil.PACKAGESAVEDATALENGTH) {
				break;
			}
			String tmp = new String(ArrayUtils.subarray(buf, i, i
					+ PackageUtil.PACKAGEHEADERLENGTH));
			if (tmp.equals(PackageUtil.PACKAGEHEADER)) {
				int messageLength = PackageUtil.byte2Int(ArrayUtils.subarray(
						buf, i + PackageUtil.PACKAGEHEADERLENGTH, i
								+ PackageUtil.PACKAGEHEADERLENGTH
								+ PackageUtil.PACKAGESAVEDATALENGTH));

				if (len < i + PackageUtil.PACKAGEHEADERLENGTH
						+ PackageUtil.PACKAGESAVEDATALENGTH + messageLength) {
					break;
				}
				byte[] data = ArrayUtils.subarray(buf, i
						+ PackageUtil.PACKAGEHEADERLENGTH
						+ PackageUtil.PACKAGESAVEDATALENGTH, i
						+ PackageUtil.PACKAGEHEADERLENGTH
						+ PackageUtil.PACKAGESAVEDATALENGTH + messageLength);
				String message = new String(data);
				System.out.println(message);
//				Filter.filterRead(message, key, messagePool);
				i += PackageUtil.PACKAGEHEADERLENGTH
						+ PackageUtil.PACKAGESAVEDATALENGTH + messageLength - 1;
			}
		}
		if (i == len) {
			return new byte[0];
		}
		return ArrayUtils.subarray(buf, i, buf.length);
	}

	void initSocketChannel() throws Exception {
		mSocketChannel = SocketChannel.open();
		mSocketChannel.connect(new InetSocketAddress(ADDRESS, HOST));
		mSocketChannel.configureBlocking(false);
		key = mSocketChannel.register(mSelector, SelectionKey.OP_CONNECT|SelectionKey.OP_READ);
	}

	void checkWriteable() {
		if (messagePool.isEmpty()) {
			String values = messagePool.pollMessage();
			System.out.println("                                   "+values);
			buffer = ByteBuffer.wrap(PackageUtil.packet(values.getBytes()));
			key.interestOps(SelectionKey.OP_WRITE);
		}
	}
}

服务器
package com.socket;

import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.ServerSocketChannel;
import java.nio.channels.SocketChannel;
import java.util.Iterator;
import java.util.Set;

import org.apache.commons.lang.ArrayUtils;

import com.filter.Filter;
import com.pool.IMessagePool;
import com.pool.impl.ServerMessagePoll;
import com.util.PackageUtil;

public class MyServerSocket {

	private ServerSocketChannel mServerSocketChannel;

	private static MyServerSocket serverSocket;

	public static String CHARSET = "utf-8";

	public static String ADDRESS = "127.0.0.1";

	public static int HOST = 34521;

	protected Selector mSelector;

	protected IMessagePool messagePool = new ServerMessagePoll();;

	ByteBuffer buffer;

	private MyServerSocket() throws Exception {
		try {
			mSelector = Selector.open();
			initSocketChannel();
			initBassiness();
		} catch (Exception e) {
			e.printStackTrace();
		} finally {
			Set<SelectionKey> keys = mSelector.keys();
			{
				for (SelectionKey key : keys) {
					try {
						key.channel().close();
					} catch (IOException e) {
						e.printStackTrace();
						continue;
					}
				}
			}
		}

	}

	/**
	 * 业务逻辑
	 * 
	 * @throws Exception
	 */
	private void initBassiness() throws Exception {
		while (true) {
			checkWriteable();
			// 瞬时检测
			if (mSelector.select() > 0) {
				Iterator<SelectionKey> keys = mSelector.selectedKeys()
						.iterator();
				while (keys.hasNext()) {
					SelectionKey key = keys.next();
					if (key.isAcceptable()) {
						dispose4Acceptable(key);
					}
					if (key.isReadable()) {
						dispose4Readable(key);
					}
					if (key.isValid() && key.isWritable()) {
						dispose4Writable(key);
					}

					keys.remove();
				}
			}
		}
	}
	/**
	 * 响应读
	 * @param key
	 * @throws Exception
	 */
	protected void dispose4Readable(SelectionKey key) throws Exception {
		SocketChannel mSocketChannel = ((SocketChannel) key.channel());
		buffer = ByteBuffer.allocate(1024);
		mSocketChannel.read(buffer);
		buffer.flip();
		this.unPacket(buffer.array(), key);
	}

	/**
	 * 可写请求
	 * 
	 * @param key
	 * @throws Exception
	 */
	protected void dispose4Writable(SelectionKey key) throws Exception {

		SocketChannel mSocketChannel = ((SocketChannel) key.channel());
		if(mSocketChannel.write(buffer)!=-1){
			buffer.clear();
		}
		key.interestOps(key.interestOps() & ~SelectionKey.OP_WRITE);
//		key.interestOps(SelectionKey.OP_READ);
	}
	/**
	 * 解包
	 * 
	 * @param buf
	 * @return
	 */
	private byte[] unPacket(byte[] buf, SelectionKey key) {

		int len = buf.length;// 37
		int i;
		for (i = 0; i < len; i++) {
			if (len < i + PackageUtil.PACKAGEHEADERLENGTH
					+ PackageUtil.PACKAGESAVEDATALENGTH) {
				break;
			}
			String tmp = new String(ArrayUtils.subarray(buf, i, i
					+ PackageUtil.PACKAGEHEADERLENGTH));
			if (tmp.equals(PackageUtil.PACKAGEHEADER)) {
				int messageLength = PackageUtil.byte2Int(ArrayUtils.subarray(
						buf, i + PackageUtil.PACKAGEHEADERLENGTH, i
								+ PackageUtil.PACKAGEHEADERLENGTH
								+ PackageUtil.PACKAGESAVEDATALENGTH));

				if (len < i + PackageUtil.PACKAGEHEADERLENGTH
						+ PackageUtil.PACKAGESAVEDATALENGTH + messageLength) {
					break;
				}
				byte[] data = ArrayUtils.subarray(buf, i
						+ PackageUtil.PACKAGEHEADERLENGTH
						+ PackageUtil.PACKAGESAVEDATALENGTH, i
						+ PackageUtil.PACKAGEHEADERLENGTH
						+ PackageUtil.PACKAGESAVEDATALENGTH + messageLength);
				String message = new String(data);
				System.out.println("server read message" + message);
				Filter.filterRead(message, key, messagePool);
				i += PackageUtil.PACKAGEHEADERLENGTH
						+ PackageUtil.PACKAGESAVEDATALENGTH + messageLength - 1;
			}
		}
		if (i == len) {
			return new byte[0];
		}
		return ArrayUtils.subarray(buf, i, buf.length);
	}

	public static MyServerSocket newInstence() throws Exception {

		if (serverSocket == null) {
			return new MyServerSocket();
		}
		return serverSocket;
	}
	/**
	 * SocketChannel初始化
	 * @throws Exception
	 */
	void initSocketChannel() throws Exception {
		mServerSocketChannel = ServerSocketChannel.open();
		mServerSocketChannel.configureBlocking(false);
		mServerSocketChannel.bind(new InetSocketAddress(ADDRESS, HOST));
		mServerSocketChannel.register(mSelector, SelectionKey.OP_ACCEPT);
	}

	void dispose4Acceptable(SelectionKey key) throws Exception {
		SocketChannel mSocketChannel = ((ServerSocketChannel) key.channel())
				.accept();
		mSocketChannel.configureBlocking(false);
		mSocketChannel.register(mSelector, SelectionKey.OP_READ);
	}
	void checkWriteable() {
		if (messagePool.isEmpty()) {
			String value = messagePool.pollMessage();
			String result = Filter.filterWrite(value, mSelector);
			if (result != null) {
				System.out.println("server:" + result);
				buffer = ByteBuffer.wrap(PackageUtil.packet(result.getBytes()));
			}
		}
	}

}

过滤器
package com.filter;

import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.util.Set;

import com.model.BaseModule;
import com.model.Chat;
import com.model.User;
import com.pool.IMessagePool;
import com.util.StringUtil;

public class Filter {
	private static final String LOGIN = "login";
	private static BaseModule modul = new BaseModule();
	
	private static SelectionKey selectionKey=null;
	
	private static Selector selector = null;
	
	/**
	 * TODO 线程启动
	 * 
	 * @param message
	 * @return
	 */
	public static void filterRead(String message, SelectionKey key,
			IMessagePool messagePool) {
		selectionKey = key;
		try {
			
			BaseModule filterModul = (BaseModule) StringUtil.string2Bean(modul,
					message);
			if(filterType(filterModul.getType())){
				if(filterValue(filterModul.getMessage())){
//					messagePool.addMessage(message);
				}else{
					
				}
			}else{
				messagePool.addMessage(message);
			}
		
		} catch (Exception e) {
			return;
		}
	}
	
	public static String filterWrite(String message,Selector mSelector){
		selector = mSelector;
		return filter(message);
		
	}
	private static String filter(String message){
		BaseModule filterModul = (BaseModule) StringUtil.string2Bean(modul,
				message);
		Chat chat = (Chat) StringUtil.string2Bean(new Chat(),
				filterModul.getMessage());
		Set<SelectionKey> keys=selector.keys();
		for(SelectionKey key:keys){
			String  markString=key.attachment()!=null?key.attachment().toString():null;
			if(markString!=null && markString.equals(chat.getTo())){
				key.interestOps(SelectionKey.OP_WRITE);
				return chat.getMessage();
			}
		}
		return null;
	}
	/**
	 * 过滤类型
	 * @param value
	 * @return
	 */
	private static boolean filterType(String value) {
		if (LOGIN.equals(value)) {
			return true;
		}
		return false;
	}
	/**
	 * 过滤内容
	 * @param value
	 * @return
	 */
	private static boolean filterValue(String value) {
		return filterLogin(value);
	}

	private static boolean filterLogin(String value) {
		User user = (User) StringUtil.string2Bean(new User(), value);
		if (user.getUserName() != null) {
			selectionKey.attach(user.getUserName());
			return true;
		}
		return false;
	}

}


service接口
package com.service;

public interface IMessageService {
	
	void doMessage(String message);

}
util
package com.util;


import java.io.UnsupportedEncodingException;

import org.apache.commons.lang.ArrayUtils;

public class PackageUtil {
	
	public static final String PACKAGEHEADER = "↨-↨";//消息长度
	public static final int PACKAGEHEADERLENGTH = 7;  //数据头长�?
	public static final int PACKAGESAVEDATALENGTH = 4; //数据长度站的位数
	
	/**
	 * 打包
	 * @param pkg 要打包的字节数组
	 * @return
	 */
	public static byte[] packet(byte[] pkg) {

		int intValue = pkg.length;
		byte[] b = new byte[4];
		for (int i = 0; i < 4; i++) {
			b[i] = (byte) (intValue >> 8 * (3 - i) & 0xFF);
			// System.out.print(Integer.toBinaryString(b[i])+" ");
			//System.out.println((b[i] & 0xFF) + " ");
		}
		try {
			byte[] newPkg = ArrayUtils.addAll(PackageUtil.PACKAGEHEADER.getBytes("utf-8"), b);
			newPkg = ArrayUtils.addAll(newPkg, pkg);
			return newPkg;
		} catch (UnsupportedEncodingException e) {
			e.printStackTrace();
		}
		
		return null;
	}
	
	/**
	 * 字节数组转整形
	 * @param b
	 * @return
	 */
	public static int byte2Int(byte[] b) {
		int intValue = 0;
		for (int i = 0; i < b.length; i++) {
			intValue += (b[i] & 0xFF) << (8 * (3 - i));
			// System.out.print(Integer.toBinaryString(intValue)+" ");
		}
		return intValue;
	}
	
	/**
	 * @param args
	 */
	public static void main(String[] args) {
		// TODO Auto-generated method stub
	}
}


StringUtil
package com.util;

import com.google.gson.Gson;

public class StringUtil {
	
	private static Gson json =new Gson();
	/**
	 * 将字符串专为json
	 * @param clazz
	 * @param message
	 * @return
	 */
	public static Object string2Bean(Object clazz,String message){
		return json.fromJson(message, clazz.getClass());
	}
	/**
	 * 将json专为字符串
	 * @param clazz
	 * @return
	 */
	public static String bean2Json(Object clazz){
		return json.toJson(clazz);
	}

}

module
package com.model;
/**
 * 默认包装
 * @author Administrator
 *
 */
public class BaseModule {
	
	String type ;
	
	String message;

	public String getType() {
		return type;
	}

	public void setType(String type) {
		this.type = type;
	}

	public String getMessage() {
		return message;
	}

	public void setMessage(String message) {
		this.message = message;
	}

}

这个为原型。后面会对具体细节进行实现以及原理进行讲解
<think>我们之前已经详细讨论了Selector(多路复用器)的原理、使用方法和应用场景。现在用户明确表示想获取Java NIO的介绍、使用方法及相关技术,因此我们需要全面介绍Java NIO的核心组件:Buffer、Channel、Selector等,并结合之前讨论的Selector部分,形成一个完整的Java NIO知识体系。 根据用户提供的引用,我们可以知道: [^1]提到Java NIO提供了一套强大而灵活的API,可用于构建高效、可扩展的I/O应用程序。 [^2]提到Java I/O包已经以NIO为基础重新实现,可以利用NIO的特性。 [^3]提到Java NIO通过缓冲区、通道和选择器的组合,实现了高性能、非阻塞的I/O操作。 因此,我们的回答将围绕以下结构展开: 1. Java NIO概述:与传统I/O的区别,核心组件介绍,优势。 2. 核心组件详解: - Buffer(缓冲区) - Channel(通道) - Selector(选择器,复用之前内容但做精简) 3. 使用方法:结合代码示例 4. 高级特性:内存映射文件、文件锁等 5. 应用场景 6. 相关问题生成 注意:之前关于Selector的详细内容已经讲过,这里可以适当精简,重点突出整体框架。 另外,注意数学表达式的格式要求(行内用$...$,独立公式用$$...$$),以及引用标注。 下面开始组织内容: </think>### Java NIO 全面解析:核心组件与高效 I/O 实践 --- #### **一、Java NIO 核心架构** 相较于传统 I/O 的**流式阻塞模型**,NIO 采用**通道-缓冲区-选择器**三层架构实现非阻塞 I/O: ```mermaid graph LR A[应用程序] --> B[Selector 多路复用器] B --> C[SocketChannel] B --> D[FileChannel] C --> E[ByteBuffer] D --> E ``` 1. **Buffer(缓冲区)** - 核心类:`ByteBuffer`, `CharBuffer`, `IntBuffer` - **环形缓冲区机制**:通过指针实现高效读写 $$ \text{position}_\text{new} = (\text{position}_\text{old} + n) \mod \text{capacity} $$ - 关键操作: ```java ByteBuffer buffer = ByteBuffer.allocate(1024); // 创建堆内缓冲区 buffer.put(data); // 写入数据 buffer.flip(); // 切换读模式 (position=0, limit=写入位置) while(buffer.hasRemaining()) { System.out.print((char)buffer.get()); // 逐字节读取 } buffer.clear(); // 重置缓冲区 ``` 2. **Channel(通道)** | 通道类型 | 特性 | 使用场景 | |------------------|-------------------------------|--------------------------| | `SocketChannel` | TCP 网络通信 | Web服务器/即时通讯 | | `FileChannel` | 文件读写 | 大文件传输 | | `DatagramChannel`| UDP 数据传输 | 实时音视频 | ```java try (FileChannel inChannel = FileChannel.open(Paths.get("input.txt")); FileChannel outChannel = FileChannel.open(Paths.get("output.txt"), CREATE, WRITE)) { inChannel.transferTo(0, inChannel.size(), outChannel); // 零拷贝文件传输 } ``` 3. **Selector(多路复用器)** 单线程管理数千连接的核心组件([详见前一回答](#))[^3]: ```java Selector selector = Selector.open(); channel.configureBlocking(false); channel.register(selector, SelectionKey.OP_READ); // 注册读事件 ``` --- #### **二、关键技术特性** 1. **非阻塞 I/O 模型** - 通道设置 `configureBlocking(false)` 后,`read()`/`write()` 立即返回 - 通过 `SelectionKey` 事件驱动处理就绪操作 2. **零拷贝优化** - `FileChannel.transferTo()` 直接在内核空间传输数据 - 对比传统 I/O 减少 2 次上下文切换和内存拷贝: ```mermaid graph LR A[传统I/O] --> B[用户态->内核态] --> C[内核缓冲区->Socket缓冲区] --> D[网卡] E[零拷贝] --> F[内核缓冲区直接到网卡] ``` 3. **内存映射文件** ```java FileChannel channel = FileChannel.open(path); MappedByteBuffer buffer = channel.map(MapMode.READ_WRITE, 0, 1024); buffer.put(0, (byte)'A'); // 直接修改磁盘文件 ``` --- #### **三、实战应用场景** 1. **高并发网络服务器** - Tomcat NIO 连接器处理 10K+ 并发连接 - Netty 框架底层基于 NIO 实现异步事件驱动 2. **大文件高效传输** - 结合 `FileChannel` + 内存映射实现 GB 级文件秒传 - 断点续传通过 `position()` 精确定位 3. **实时数据管道** ```java Pipe pipe = Pipe.open(); Pipe.SinkChannel sink = pipe.sink(); // 写入端 Pipe.SourceChannel source = pipe.source(); // 读取端 ``` --- #### **四、与传统 I/O 的对比** | **特性** | `java.io` | `java.nio` | |------------------|-------------------------|--------------------------| | 数据模型 | 流(Stream) | 块(Buffer) | | 阻塞行为 | 阻塞式 | 非阻塞+选择器 | | 内存使用 | JVM 堆内存 | 堆外内存(可选) | | 大文件处理 | 逐块复制 | 内存映射+零拷贝 | | 并发连接管理 | 1线程/连接 | 1线程管理数千连接 [^1][^3] | > **性能公式**: > 吞吐量提升比 $\eta \approx \frac{T_\text{io}}{T_\text{nio}} \propto \frac{\text{连接数}}{\log(\text{连接数})}$ ---
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值