netty Unpooled字节buf分配器

netty 字节buf定义:[url]http://donald-draper.iteye.com/blog/2393813[/url]
netty 资源泄漏探测器:[url]http://donald-draper.iteye.com/blog/2393940[/url]
netty 抽象字节buf解析:[url]http://donald-draper.iteye.com/blog/2394078[/url]
netty 抽象字节buf引用计数器:[url]http://donald-draper.iteye.com/blog/2394109[/url]
netty 复合buf概念:[url]http://donald-draper.iteye.com/blog/2394408[/url]
netty 抽象字节buf分配器:[url]http://donald-draper.iteye.com/blog/2394419[/url]
[size=medium][b]引言[/b][/size]
上一篇我们看了抽象字节buf分配器,先来回顾一下:
创建字节buf主要根据字节buf分配器的directByDefault属性,来决定分配buf是否为direct类型还是heap类型;创建direct和heap buf实际通过newDirectBuffer和newHeapBuffer方法,待子类扩展。看出ioBuffer方法创建的字节buf,优先为direct类型,当系统平台不支持Unsafe时,才为heap类型;创建复合buf主要根据字节buf分配器的directByDefault属性,来决定分配buf是否为direct类型还是heap类型;创建复合buf时,如果资源泄漏探测功能开启,则追踪复合buf内存泄漏情况。

今天我们来看抽象字节buf分配器一个具体实现非池类字节buf分配器UnpooledByteBufAllocator
/**
* Simplistic {@link ByteBufAllocator} implementation that does not pool anything.
*/
public final class UnpooledByteBufAllocator extends AbstractByteBufAllocator implements ByteBufAllocatorMetricProvider {
//字节buf分配器Metric
private final UnpooledByteBufAllocatorMetric metric = new UnpooledByteBufAllocatorMetric();
private final boolean disableLeakDetector;//是否关闭资源泄漏探测

/**
* Default instance which uses leak-detection for direct buffers.
默认非池类字节buf分配器
*/
public static final UnpooledByteBufAllocator DEFAULT =
new UnpooledByteBufAllocator(PlatformDependent.directBufferPreferred());

/**
* Create a new instance which uses leak-detection for direct buffers.
*
* @param preferDirect {@code true} if {@link #buffer(int)} should try to allocate a direct buffer rather than
* a heap buffer
*/
public UnpooledByteBufAllocator(boolean preferDirect) {
this(preferDirect, false);
}

/**
* Create a new instance
*
* @param preferDirect {@code true} if {@link #buffer(int)} should try to allocate a direct buffer rather than
* a heap buffer
* @param disableLeakDetector {@code true} if the leak-detection should be disabled completely for this
* allocator. This can be useful if the user just want to depend on the GC to handle
* direct buffers when not explicit released.
*/
public UnpooledByteBufAllocator(boolean preferDirect, boolean disableLeakDetector) {
super(preferDirect);
this.disableLeakDetector = disableLeakDetector;
}
}


来看一下度量器
private static final class UnpooledByteBufAllocatorMetric implements ByteBufAllocatorMetric {
final LongCounter directCounter = PlatformDependent.newLongCounter();//direct内存使用计数器
final LongCounter heapCounter = PlatformDependent.newLongCounter();//Heap内存使用计数器
@Override
public long usedHeapMemory() {
return heapCounter.value();
}
@Override
public long usedDirectMemory() {
return directCounter.value();
}
@Override
public String toString() {
return StringUtil.simpleClassName(this) +
"(usedHeapMemory: " + usedHeapMemory() + "; usedDirectMemory: " + usedDirectMemory() + ')';
}
}


//LongCounter Long计数器
/**
* Counter for long.
*/
public interface LongCounter {
void add(long delta);
void increment();
void decrement();
long value();
}

来看创建堆buf
@Override
protected ByteBuf newHeapBuffer(int initialCapacity, int maxCapacity) {
return PlatformDependent.hasUnsafe() ?
new InstrumentedUnpooledUnsafeHeapByteBuf(this, initialCapacity, maxCapacity) :
new InstrumentedUnpooledHeapByteBuf(this, initialCapacity, maxCapacity);
}

堆buf有两种,我们分别来看
先看InstrumentedUnpooledUnsafeHeapByteBuf
1.
//InstrumentedUnpooledUnsafeHeapByteBuf
private static final class InstrumentedUnpooledUnsafeHeapByteBuf extends UnpooledUnsafeHeapByteBuf {
InstrumentedUnpooledUnsafeHeapByteBuf(UnpooledByteBufAllocator alloc, int initialCapacity, int maxCapacity) {
super(alloc, initialCapacity, maxCapacity);
}

@Override
byte[] allocateArray(int initialCapacity) {
byte[] bytes = super.allocateArray(initialCapacity);
//更新buf分配器Heap内存使用量
((UnpooledByteBufAllocator) alloc()).incrementHeap(bytes.length);
return bytes;
}

@Override
void freeArray(byte[] array) {
int length = array.length;
super.freeArray(array);
//释放buf分配器Heap内存使用量
((UnpooledByteBufAllocator) alloc()).decrementHeap(length);
}
}


//UnpooledUnsafeHeapByteBuf
class UnpooledUnsafeHeapByteBuf extends UnpooledHeapByteBuf {

/**
* Creates a new heap buffer with a newly allocated byte array.
*
* @param initialCapacity the initial capacity of the underlying byte array
* @param maxCapacity the max capacity of the underlying byte array
*/
UnpooledUnsafeHeapByteBuf(ByteBufAllocator alloc, int initialCapacity, int maxCapacity) {
super(alloc, initialCapacity, maxCapacity);
}
...
}

2.
再来看另一种堆buf InstrumentedUnpooledHeapByteBuf
//InstrumentedUnpooledHeapByteBuf
private static final class InstrumentedUnpooledHeapByteBuf extends UnpooledHeapByteBuf {
InstrumentedUnpooledHeapByteBuf(UnpooledByteBufAllocator alloc, int initialCapacity, int maxCapacity) {
super(alloc, initialCapacity, maxCapacity);
}

@Override
byte[] allocateArray(int initialCapacity) {
byte[] bytes = super.allocateArray(initialCapacity);
//更新buf分配器Heap内存使用量
((UnpooledByteBufAllocator) alloc()).incrementHeap(bytes.length);
return bytes;
}

@Override
void freeArray(byte[] array) {
int length = array.length;
super.freeArray(array);
//释放buf分配器Heap内存使用量
((UnpooledByteBufAllocator) alloc()).decrementHeap(length);
}
}

我们来看一下Unpooled堆字节buf:
/**
* Big endian Java heap buffer implementation.
*/
public class UnpooledHeapByteBuf extends AbstractReferenceCountedByteBuf {

private final ByteBufAllocator alloc;
byte[] array;//存储数据的字节数组
private ByteBuffer tmpNioBuf;//临时nio 字节部分

/**
* Creates a new heap buffer with a newly allocated byte array.
*
* @param initialCapacity the initial capacity of the underlying byte array
* @param maxCapacity the max capacity of the underlying byte array
*/
protected UnpooledHeapByteBuf(ByteBufAllocator alloc, int initialCapacity, int maxCapacity) {
super(maxCapacity);

checkNotNull(alloc, "alloc");

if (initialCapacity > maxCapacity) {
throw new IllegalArgumentException(String.format(
"initialCapacity(%d) > maxCapacity(%d)", initialCapacity, maxCapacity));
}

this.alloc = alloc;
setArray(allocateArray(initialCapacity));
setIndex(0, 0);
}

/**
* Creates a new heap buffer with an existing byte array.
*
* @param initialArray the initial underlying byte array
* @param maxCapacity the max capacity of the underlying byte array
*/
protected UnpooledHeapByteBuf(ByteBufAllocator alloc, byte[] initialArray, int maxCapacity) {
super(maxCapacity);

checkNotNull(alloc, "alloc");
checkNotNull(initialArray, "initialArray");

if (initialArray.length > maxCapacity) {
throw new IllegalArgumentException(String.format(
"initialCapacity(%d) > maxCapacity(%d)", initialArray.length, maxCapacity));
}

this.alloc = alloc;
setArray(initialArray);
setIndex(0, initialArray.length);
}
//分配initialCapacity容量的字节数组
byte[] allocateArray(int initialCapacity) {
return new byte[initialCapacity];
}
...
@Override
public ByteBuf getBytes(int index, byte[] dst, int dstIndex, int length) {
checkDstIndex(index, length, dstIndex, dst.length);
System.arraycopy(array, index, dst, dstIndex, length);
return this;
}
....
@Override
public ByteBuf setBytes(int index, ByteBuf src, int srcIndex, int length) {
checkSrcIndex(index, length, srcIndex, src.capacity());
if (src.hasMemoryAddress()) {
PlatformDependent.copyMemory(src.memoryAddress() + srcIndex, array, index, length);
} else if (src.hasArray()) {
setBytes(index, src.array(), src.arrayOffset() + srcIndex, length);
} else {
src.getBytes(srcIndex, array, index, length);
}
return this;
}

@Override
public ByteBuf setBytes(int index, byte[] src, int srcIndex, int length) {
checkSrcIndex(index, length, srcIndex, src.length);
System.arraycopy(src, srcIndex, array, index, length);
return this;
}
...

@Override
public ByteBuffer nioBuffer(int index, int length) {
ensureAccessible();
return ByteBuffer.wrap(array, index, length).slice();
}

@Override
public ByteBuffer[] nioBuffers(int index, int length) {
return new ByteBuffer[] { nioBuffer(index, length) };
}

@Override
public ByteBuffer internalNioBuffer(int index, int length) {
checkIndex(index, length);
return (ByteBuffer) internalNioBuffer().clear().position(index).limit(index + length);
}
...
private ByteBuffer internalNioBuffer() {
ByteBuffer tmpNioBuf = this.tmpNioBuf;
if (tmpNioBuf == null) {
this.tmpNioBuf = tmpNioBuf = ByteBuffer.wrap(array);
}
return tmpNioBuf;
}
...
@Override
protected void deallocate() {
freeArray(array);
array = null;
}
//待子类扩展
void freeArray(byte[] array) {
// NOOP
}
}

从上面可以看出,非池类堆字节buf,实际为一个字节数组。

//堆内存使用更新与释放,委托给内部度量器
void incrementHeap(int amount) {
metric.heapCounter.add(amount);
}
void decrementHeap(int amount) {
metric.heapCounter.add(-amount);
}


再来看分配direct buf:
@Override
protected ByteBuf newDirectBuffer(int initialCapacity, int maxCapacity) {
final ByteBuf buf;
if (PlatformDependent.hasUnsafe()) {
buf = PlatformDependent.useDirectBufferNoCleaner() ?
new InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf(this, initialCapacity, maxCapacity) :
new InstrumentedUnpooledUnsafeDirectByteBuf(this, initialCapacity, maxCapacity);
} else {
buf = new InstrumentedUnpooledDirectByteBuf(this, initialCapacity, maxCapacity);
}
return disableLeakDetector ? buf : toLeakAwareBuffer(buf);
}

从上面来看direct buf有三种,分别为InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf,
InstrumentedUnpooledUnsafeDirectByteBuf,InstrumentedUnpooledDirectByteBuf

我们分别来看这三种:
1.
private static final class InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf
extends UnpooledUnsafeNoCleanerDirectByteBuf {
InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf(
UnpooledByteBufAllocator alloc, int initialCapacity, int maxCapacity) {
super(alloc, initialCapacity, maxCapacity);
}

@Override
protected ByteBuffer allocateDirect(int initialCapacity) {
ByteBuffer buffer = super.allocateDirect(initialCapacity);
//更新buf分配器direct内存使用量
((UnpooledByteBufAllocator) alloc()).incrementDirect(buffer.capacity());
return buffer;
}

@Override
ByteBuffer reallocateDirect(ByteBuffer oldBuffer, int initialCapacity) {
int capacity = oldBuffer.capacity();
ByteBuffer buffer = super.reallocateDirect(oldBuffer, initialCapacity);
//更新buf分配器direct内存使用量
((UnpooledByteBufAllocator) alloc()).incrementDirect(buffer.capacity() - capacity);
return buffer;
}

@Override
protected void freeDirect(ByteBuffer buffer) {
int capacity = buffer.capacity();
super.freeDirect(buffer);
//释放buf分配器direct内存使用量
((UnpooledByteBufAllocator) alloc()).decrementDirect(capacity);
}
}

//UnpooledUnsafeNoCleanerDirectByteBuf
class UnpooledUnsafeNoCleanerDirectByteBuf extends UnpooledUnsafeDirectByteBuf {

UnpooledUnsafeNoCleanerDirectByteBuf(ByteBufAllocator alloc, int initialCapacity, int maxCapacity) {
super(alloc, initialCapacity, maxCapacity);
}
...
}

再来看第二种:
2.
private static final class InstrumentedUnpooledUnsafeDirectByteBuf extends UnpooledUnsafeDirectByteBuf {
InstrumentedUnpooledUnsafeDirectByteBuf(
UnpooledByteBufAllocator alloc, int initialCapacity, int maxCapacity) {
super(alloc, initialCapacity, maxCapacity);
}

@Override
protected ByteBuffer allocateDirect(int initialCapacity) {
ByteBuffer buffer = super.allocateDirect(initialCapacity);
//更新buf分配器direct内存使用量
((UnpooledByteBufAllocator) alloc()).incrementDirect(buffer.capacity());
return buffer;
}

@Override
protected void freeDirect(ByteBuffer buffer) {
int capacity = buffer.capacity();
super.freeDirect(buffer);
//释放buf分配器direct内存使用量
((UnpooledByteBufAllocator) alloc()).decrementDirect(capacity);
}
}

//UnpooledUnsafeDirectByteBuf
/**
* A NIO {@link ByteBuffer} based buffer. It is recommended to use {@link Unpooled#directBuffer(int)}
* and {@link Unpooled#wrappedBuffer(ByteBuffer)} instead of calling the
* constructor explicitly.
*/
public class UnpooledUnsafeDirectByteBuf extends AbstractReferenceCountedByteBuf {

private final ByteBufAllocator alloc;
private ByteBuffer tmpNioBuf;
private int capacity;//容量
private boolean doNotFree;//是否需要释放内存
ByteBuffer buffer;//内部nio 字节buf
long memoryAddress;
/**
* Creates a new direct buffer.
*
* @param initialCapacity the initial capacity of the underlying direct buffer
* @param maxCapacity the maximum capacity of the underlying direct buffer
*/
protected UnpooledUnsafeDirectByteBuf(ByteBufAllocator alloc, int initialCapacity, int maxCapacity) {
super(maxCapacity);
if (alloc == null) {
throw new NullPointerException("alloc");
}
if (initialCapacity < 0) {
throw new IllegalArgumentException("initialCapacity: " + initialCapacity);
}
if (maxCapacity < 0) {
throw new IllegalArgumentException("maxCapacity: " + maxCapacity);
}
if (initialCapacity > maxCapacity) {
throw new IllegalArgumentException(String.format(
"initialCapacity(%d) > maxCapacity(%d)", initialCapacity, maxCapacity));
}

this.alloc = alloc;
setByteBuffer(allocateDirect(initialCapacity), false);
}

/**
* Creates a new direct buffer by wrapping the specified initial buffer.
*
* @param maxCapacity the maximum capacity of the underlying direct buffer
*/
protected UnpooledUnsafeDirectByteBuf(ByteBufAllocator alloc, ByteBuffer initialBuffer, int maxCapacity) {
// We never try to free the buffer if it was provided by the end-user as we not know if this is an duplicate or
// an slice. This is done to prevent an IllegalArgumentException when using Java9 as Unsafe.invokeCleaner(...)
// will check if the given buffer is either an duplicate or slice and in this case throw an
// IllegalArgumentException.
//
// See http://hg.openjdk.java.net/jdk9/hs-demo/jdk/file/0d2ab72ba600/src/jdk.unsupported/share/classes/
// sun/misc/Unsafe.java#l1250
//
// We also call slice() explicitly here to preserve behaviour with previous netty releases.
this(alloc, initialBuffer.slice(), maxCapacity, false);
}

UnpooledUnsafeDirectByteBuf(ByteBufAllocator alloc, ByteBuffer initialBuffer, int maxCapacity, boolean doFree) {
super(maxCapacity);
if (alloc == null) {
throw new NullPointerException("alloc");
}
if (initialBuffer == null) {
throw new NullPointerException("initialBuffer");
}
if (!initialBuffer.isDirect()) {
throw new IllegalArgumentException("initialBuffer is not a direct buffer.");
}
if (initialBuffer.isReadOnly()) {
throw new IllegalArgumentException("initialBuffer is a read-only buffer.");
}

int initialCapacity = initialBuffer.remaining();
if (initialCapacity > maxCapacity) {
throw new IllegalArgumentException(String.format(
"initialCapacity(%d) > maxCapacity(%d)", initialCapacity, maxCapacity));
}

this.alloc = alloc;
doNotFree = !doFree;
setByteBuffer(initialBuffer.order(ByteOrder.BIG_ENDIAN), false);
writerIndex(initialCapacity);
}

/**
* Allocate a new direct {@link ByteBuffer} with the given initialCapacity.
分配initialCapacity容量的direct buf,实际委托给内部的nio 字节buf
*/
protected ByteBuffer allocateDirect(int initialCapacity) {
return ByteBuffer.allocateDirect(initialCapacity);
}
...
}

从上面我们可以看出,非池类Direct buf,实际为一个nio 字节buf。
再来看第三种:
3.
private static final class InstrumentedUnpooledDirectByteBuf extends UnpooledDirectByteBuf {
InstrumentedUnpooledDirectByteBuf(
UnpooledByteBufAllocator alloc, int initialCapacity, int maxCapacity) {
super(alloc, initialCapacity, maxCapacity);
}

@Override
protected ByteBuffer allocateDirect(int initialCapacity) {
ByteBuffer buffer = super.allocateDirect(initialCapacity);
//更新buf分配器direct内存使用量
((UnpooledByteBufAllocator) alloc()).incrementDirect(buffer.capacity());
return buffer;
}

@Override
protected void freeDirect(ByteBuffer buffer) {
int capacity = buffer.capacity();
super.freeDirect(buffer);
//释放buf分配器direct内存使用量
((UnpooledByteBufAllocator) alloc()).decrementDirect(capacity);
}
}



//UnpooledDirectByteBuf
/**
* A NIO {@link ByteBuffer} based buffer. It is recommended to use {@link Unpooled#directBuffer(int)}
* and {@link Unpooled#wrappedBuffer(ByteBuffer)} instead of calling the
* constructor explicitly.
*/
public class UnpooledDirectByteBuf extends AbstractReferenceCountedByteBuf {

private final ByteBufAllocator alloc;

private ByteBuffer buffer;//内部字节buf
private ByteBuffer tmpNioBuf;
private int capacity;//容量
private boolean doNotFree;//是否需要释放内存

/**
* Creates a new direct buffer.
*
* @param initialCapacity the initial capacity of the underlying direct buffer
* @param maxCapacity the maximum capacity of the underlying direct buffer
*/
protected UnpooledDirectByteBuf(ByteBufAllocator alloc, int initialCapacity, int maxCapacity) {
super(maxCapacity);
if (alloc == null) {
throw new NullPointerException("alloc");
}
if (initialCapacity < 0) {
throw new IllegalArgumentException("initialCapacity: " + initialCapacity);
}
if (maxCapacity < 0) {
throw new IllegalArgumentException("maxCapacity: " + maxCapacity);
}
if (initialCapacity > maxCapacity) {
throw new IllegalArgumentException(String.format(
"initialCapacity(%d) > maxCapacity(%d)", initialCapacity, maxCapacity));
}

this.alloc = alloc;
setByteBuffer(ByteBuffer.allocateDirect(initialCapacity));
}

/**
* Creates a new direct buffer by wrapping the specified initial buffer.
*
* @param maxCapacity the maximum capacity of the underlying direct buffer
*/
protected UnpooledDirectByteBuf(ByteBufAllocator alloc, ByteBuffer initialBuffer, int maxCapacity) {
super(maxCapacity);
if (alloc == null) {
throw new NullPointerException("alloc");
}
if (initialBuffer == null) {
throw new NullPointerException("initialBuffer");
}
if (!initialBuffer.isDirect()) {
throw new IllegalArgumentException("initialBuffer is not a direct buffer.");
}
if (initialBuffer.isReadOnly()) {
throw new IllegalArgumentException("initialBuffer is a read-only buffer.");
}

int initialCapacity = initialBuffer.remaining();
if (initialCapacity > maxCapacity) {
throw new IllegalArgumentException(String.format(
"initialCapacity(%d) > maxCapacity(%d)", initialCapacity, maxCapacity));
}

this.alloc = alloc;
doNotFree = true;
setByteBuffer(initialBuffer.slice().order(ByteOrder.BIG_ENDIAN));
writerIndex(initialCapacity);
}

/**
* Allocate a new direct {@link ByteBuffer} with the given initialCapacity.
分配initialCapacity容量的direct buf,实际委托给内部的nio 字节buf
*/
protected ByteBuffer allocateDirect(int initialCapacity) {
return ByteBuffer.allocateDirect(initialCapacity);
}
...
}


再来看更新direct 内存使用量,更新操作,直接委托给内部度量器:
void incrementDirect(int amount) {
metric.directCounter.add(amount);
}

void decrementDirect(int amount) {
metric.directCounter.add(-amount);
}

再来看复合buf

@Override
public CompositeByteBuf compositeHeapBuffer(int maxNumComponents) {
CompositeByteBuf buf = new CompositeByteBuf(this, false, maxNumComponents);
return disableLeakDetector ? buf : toLeakAwareBuffer(buf);
}

@Override
public CompositeByteBuf compositeDirectBuffer(int maxNumComponents) {
CompositeByteBuf buf = new CompositeByteBuf(this, true, maxNumComponents);
return disableLeakDetector ? buf : toLeakAwareBuffer(buf);
}



[size=medium][b]
总结:[/b][/size]
[color=blue]非池类堆字节buf,实际为一个字节数组,直接在Java虚拟机堆内存中,分配字节缓存;非池类Direct buf,实际为一个nio 字节buf,从操作系统实际物理内存中,分配字节缓存。Unpooled创建字节buf,实际委托给内部字节分配器UnpooledByteBufAllocator。[/color]

附:
我们再来看一下Unpooled:
/**
* Creates a new {@link ByteBuf} by allocating new space or by wrapping
* or copying existing byte arrays, byte buffers and a string.
*Unpooled用于创建一个新的字节buf,包装和拷贝一个已经存在的字节数组,字节buf或String
* <h3>Use static import</h3>
* This classes is intended to be used with Java 5 static import statement:
*
* <pre>
* import static io.netty.buffer.{@link Unpooled}.*;
*
* {@link ByteBuf} heapBuffer = buffer(128);
* {@link ByteBuf} directBuffer = directBuffer(256);
* {@link ByteBuf} wrappedBuffer = wrappedBuffer(new byte[128], new byte[256]);
* {@link ByteBuf} copiedBuffer = copiedBuffer({@link ByteBuffer}.allocate(128));
* </pre>
*
* <h3>Allocating a new buffer</h3>
*
* Three buffer types are provided out of the box.
*
* [list]
* [*]{@link #buffer(int)} allocates a new fixed-capacity heap buffer.

* [*]{@link #directBuffer(int)} allocates a new fixed-capacity direct buffer.

* [/list]
*
* <h3>Creating a wrapped buffer</h3>
*包装buf,为底层buf的视图,所有底层buf的改变,包装buf都可以看到
* Wrapped buffer is a buffer which is a view of one or more existing
* byte arrays and byte buffers. Any changes in the content of the original
* array or buffer will be visible in the wrapped buffer. Various wrapper
* methods are provided and their name is all {@code wrappedBuffer()}.
* You might want to take a look at the methods that accept varargs closely if
* you want to create a buffer which is composed of more than one array to
* reduce the number of memory copy.
*
* <h3>Creating a copied buffer</h3>
*拷贝buf,并不会共享数据,底层buf数据的改变,拷贝buf看不到
* Copied buffer is a deep copy of one or more existing byte arrays, byte
* buffers or a string. Unlike a wrapped buffer, there's no shared data
* between the original data and the copied buffer. Various copy methods are
* provided and their name is all {@code copiedBuffer()}. It is also convenient
* to use this operation to merge multiple buffers into one buffer.
*/
public final class Unpooled {
//字节buf分配器,默认为UnpooledByteBufAllocator.DEFAULT
private static final ByteBufAllocator ALLOC = UnpooledByteBufAllocator.DEFAULT;

/**
* Big endian byte order.
*/
public static final ByteOrder BIG_ENDIAN = ByteOrder.BIG_ENDIAN;

/**
* Little endian byte order.
*/
public static final ByteOrder LITTLE_ENDIAN = ByteOrder.LITTLE_ENDIAN;

/**
* A buffer whose capacity is {@code 0}.
*/
public static final ByteBuf EMPTY_BUFFER = ALLOC.buffer(0, 0);

static {
assert EMPTY_BUFFER instanceof EmptyByteBuf: "EMPTY_BUFFER must be an EmptyByteBuf.";
}
}

再来看分配buf:

/**
* Creates a new big-endian Java heap buffer with reasonably small initial capacity, which
* expands its capacity boundlessly on demand.
*/
public static ByteBuf buffer() {
return ALLOC.heapBuffer();
}

/**
* Creates a new big-endian direct buffer with reasonably small initial capacity, which
* expands its capacity boundlessly on demand.
*/
public static ByteBuf directBuffer() {
return ALLOC.directBuffer();
}

/**
* Creates a new big-endian Java heap buffer with the specified {@code capacity}, which
* expands its capacity boundlessly on demand. The new buffer's {@code readerIndex} and
* {@code writerIndex} are {@code 0}.
*/
public static ByteBuf buffer(int initialCapacity) {
return ALLOC.heapBuffer(initialCapacity);
}

/**
* Creates a new big-endian direct buffer with the specified {@code capacity}, which
* expands its capacity boundlessly on demand. The new buffer's {@code readerIndex} and
* {@code writerIndex} are {@code 0}.
*/
public static ByteBuf directBuffer(int initialCapacity) {
return ALLOC.directBuffer(initialCapacity);
}

/**
* Returns a new big-endian composite buffer with no components.
*/
public static CompositeByteBuf compositeBuffer() {
return compositeBuffer(AbstractByteBufAllocator.DEFAULT_MAX_COMPONENTS);
}

/**
* Returns a new big-endian composite buffer with no components.
*/
public static CompositeByteBuf compositeBuffer(int maxNumComponents) {
return new CompositeByteBuf(ALLOC, false, maxNumComponents);
}

从上面来看Unpooled创建字节buf,实际委托给内部字节分配器UnpooledByteBufAllocator。

包装buf相关的方法:
/**
* Creates a new big-endian buffer which wraps the specified {@code array}.
* A modification on the specified array's content will be visible to the
* returned buffer.
*/
public static ByteBuf wrappedBuffer(byte[] array) {
if (array.length == 0) {
return EMPTY_BUFFER;
}
return new UnpooledHeapByteBuf(ALLOC, array, array.length);
}


/**
* Creates a new buffer which wraps the specified buffer's readable bytes.
* A modification on the specified buffer's content will be visible to the
* returned buffer.
* @param buffer The buffer to wrap. Reference count ownership of this variable is transfered to this method.
* @return The readable portion of the {@code buffer}, or an empty buffer if there is no readable portion.
* The caller is responsible for releasing this buffer.
*/
public static ByteBuf wrappedBuffer(ByteBuf buffer) {
if (buffer.isReadable()) {
return buffer.slice();
} else {
buffer.release();
return EMPTY_BUFFER;
}
}

/**
* Creates a new big-endian composite buffer which wraps the readable bytes of the
* specified buffers without copying them. A modification on the content
* of the specified buffers will be visible to the returned buffer.
* @param maxNumComponents Advisement as to how many independent buffers are allowed to exist before
* consolidation occurs.
* @param buffers The buffers to wrap. Reference count ownership of all variables is transfered to this method.
* @return The readable portion of the {@code buffers}. The caller is responsible for releasing this buffer.
*/
public static ByteBuf wrappedBuffer(int maxNumComponents, ByteBuf... buffers) {
switch (buffers.length) {
case 0:
break;
case 1:
ByteBuf buffer = buffers[0];
if (buffer.isReadable()) {
return wrappedBuffer(buffer.order(BIG_ENDIAN));
} else {
buffer.release();
}
break;
default:
for (int i = 0; i < buffers.length; i++) {
ByteBuf buf = buffers[i];
if (buf.isReadable()) {
return new CompositeByteBuf(ALLOC, false, maxNumComponents, buffers, i, buffers.length);
}
buf.release();
}
break;
}
return EMPTY_BUFFER;
}

...

拷贝buf相关的方法:

**
* Creates a new big-endian buffer whose content is a copy of the
* specified {@code array}. The new buffer's {@code readerIndex} and
* {@code writerIndex} are {@code 0} and {@code array.length} respectively.
*/
public static ByteBuf copiedBuffer(byte[] array) {
if (array.length == 0) {
return EMPTY_BUFFER;
}
return wrappedBuffer(array.clone());
}

/**
* Creates a new buffer whose content is a copy of the specified
* {@code buffer}'s readable bytes. The new buffer's {@code readerIndex}
* and {@code writerIndex} are {@code 0} and {@code buffer.readableBytes}
* respectively.
*/
public static ByteBuf copiedBuffer(ByteBuf buffer) {
int readable = buffer.readableBytes();
if (readable > 0) {
ByteBuf copy = buffer(readable);
copy.writeBytes(buffer, buffer.readerIndex(), readable);
return copy;
} else {
return EMPTY_BUFFER;
}
}


/**
* Return a unreleasable view on the given {@link ByteBuf} which will just ignore release and retain calls.
返回一个不可释放的buf
*/
public static ByteBuf unreleasableBuffer(ByteBuf buf) {
return new UnreleasableByteBuf(buf);
}


//UnreleasableByteBuf
final class UnreleasableByteBuf extends WrappedByteBuf {

private SwappedByteBuf swappedBuf;
...
@Override
public ByteBuf retain(int increment) {
return this;
}

@Override
public ByteBuf retain() {
return this;
}

@Override
public ByteBuf touch() {
return this;
}

@Override
public ByteBuf touch(Object hint) {
return this;
}

@Override
public boolean release() {
return false;
}

@Override
public boolean release(int decrement) {
return false;
}
}

//WrappedByteBuf,理解为字节静态代理
class WrappedByteBuf extends ByteBuf {

protected final ByteBuf buf;
@Override
public final int readerIndex() {
return buf.readerIndex();
}

@Override
public final ByteBuf readerIndex(int readerIndex) {
buf.readerIndex(readerIndex);
return this;
}

@Override
public final int writerIndex() {
return buf.writerIndex();
}

@Override
public final ByteBuf writerIndex(int writerIndex) {
buf.writerIndex(writerIndex);
return this;
}
...
}


/**
* Wrap the given {@link ByteBuf}s in an unmodifiable {@link ByteBuf}. Be aware the returned {@link ByteBuf} will
* not try to slice the given {@link ByteBuf}s to reduce GC-Pressure.
包装buf为不可修改字节buf
*/
public static ByteBuf unmodifiableBuffer(ByteBuf... buffers) {
return new FixedCompositeByteBuf(ALLOC, buffers);
}

//FixedCompositeByteBuf
/**
* {@link ByteBuf} implementation which allows to wrap an array of {@link ByteBuf} in a read-only mode.
* This is useful to write an array of {@link ByteBuf}s.
*/
final class FixedCompositeByteBuf extends AbstractReferenceCountedByteBuf {
private static final ByteBuf[] EMPTY = { Unpooled.EMPTY_BUFFER };
private final int nioBufferCount;
private final int capacity;
private final ByteBufAllocator allocator;
private final ByteOrder order;
private final Object[] buffers;
private final boolean direct;
...
}



//Unpooled

[img]http://dl2.iteye.com/upload/attachment/0127/1505/5bd6b453-0590-3f22-9c45-fe0f92898e82.png[/img]

[img]http://dl2.iteye.com/upload/attachment/0127/1507/0f86a28b-d327-3f49-b15f-9b502a2dd9a5.png[/img]
Netty核心原理与生产实践:从线程模型到内存管理的面试与实战指南 在构建高性能网络应用时,异步事件驱动架构是解决高并发通信挑战的核心范式。其原理基于Reactor模式,通过事件循环机制实现非阻塞I/O处理,从而提升系统吞吐量和资源利用率。这一技术的核心价值在于能够有效管理海量连接,降低线程上下文切换开销,并简化复杂网络编程。在实际应用场景中,从微服务通信到实时消息推送,高性能网络框架都扮演着关键角色。本文聚焦Netty这一Java领域主流实现,深入解析其线程模型设计,特别是主从Reactor多线程架构如何优化连接处理;同时探讨ByteBuf内存管理机制,包括池化技术与直接 阅读详情

相关推荐

Netty_03_ByteBuf和网络中拆包粘包问题及其解决

Netty

毛毛的专栏 1646

Netty:用Unpooled获取包裹的ByteBuf

Netty:用Unpooled获取包裹的ByteBuf

听海边涛声 514

第二章Netty ByteBuf 内存模式完全指南:池化 vs 非池化、堆内存 vs 直接内存详解与实战

本文深入解析Netty核心组件ByteBuf的内存管理模式,重点对比堆内存/直接内存、池化/非池化、Safe/Unsafe三种关键机制。通过代码示例展示不同模式的特点:池化内存通过预分配复用提升性能,非池化内存则适合简单场景;堆内存受GC影响但开发友好,直接内存减少复制开销但管理复杂。文章提供决策流程图和快速参考表,建议高并发场景优先选择池化直接内存,同时强调正确释放内存的重要性,为开发者优化网络应用性能提供实用指南。

ywl470812087的博客 12

深入理解 Netty 中的 Unpooled 使用方法

Unpooled在实现自定义通信协议时,我们往往需要对数据进行分块、组合,Unpooled可以通过轻松实现不同数据块的拼接。这种组合方式避免了多次内存拷贝,提升了通信效率。Unpooled是 Netty 中用于创建未池化ByteBuf的重要工具类。它提供了多种创建和操作ByteBuf的方法,帮助开发者更高效地处理网络通信中的数据。在实际开发中,Unpooled尤其适合那些对内存池要求不高、或者数据生命周期较短的场景。常用的方法如buffer()等可以极大简化缓冲区操作,提升程序的性能和可维护性。

龚小帅的博客 1955

Simulink Test Sequence自动化测试实战:从Excel到模型验证

本文详细介绍了如何利用Excel和MATLAB脚本实现Simulink Test Sequence的自动化测试。通过设计结构化Excel模板管理测试数据,并调用Simulink Test API批量创建与执行测试用例,该方法能显著提升复杂模型单元测试的效率,便于集成到CI/CD流程,实现从数据到验证的全自动化。

weixin_29261711的博客 144

Bresenham画圆算法在嵌入式屏幕(如ILI9806G)上的实战:从C代码到单片机显示

本文详细介绍了Bresenham画圆算法在嵌入式屏幕(如ILI9806G)上的高效实现与优化方法。通过整数化改造和针对STM32等微控制器的特殊优化,显著提升了图形显示性能,适用于资源受限的嵌入式系统开发。文章还分享了实心圆与空心圆的实现策略及实战经验。

weixin_33712987的博客 391

Netty学习十五:内存管理之零拷贝

Netty学习系列

weixin_41402069的博客 3459

001-基础入门-Web演示源码&资源&工具箱等

小迪安全2023笔记

wushangyu32335的博客 4265

Netty核心:Unpooled类全解析

Unpooled是Netty的核心Buffer工厂类,提供非池化ByteBuf的创建能力。作为final工具类,它采用多种设计模式实现高性能Buffer管理:1)工厂模式统一创建接口,支持堆/直接内存Buffer;2)零拷贝包装(wrappedBuffer)和深拷贝(copiedBuffer)双机制;3)智能实现选择策略,根据平台特性自动选用Unsafe优化版本;4)组合Buffer处理时采用惰性创建和空数据跳过优化。类结构包含分配器层、实现层和工具层,通过ByteWrapper泛型接口统一处理不同数据源。

lifallen的博客 930

Centos7.6操作系统安装+Bond配置(600GB磁盘版本)

1.安装前准备工作 准备centos7.6操作系统镜像 CentOS-7-x86_64-DVD-1810.iso 服务器安装一般都是在BMC界面操作,所以安装前需要在BMC界面挂载好操作系统镜像 重启服务器后自动进入安装界面。 华为大数据集群安装操作系统配置

小时光 6010

移远EC800M CAT1模块HTTP POST实战:从AT指令到数据上报的完整避坑指南

本文详细介绍了移远EC800M CAT1模块通过AT指令实现HTTP POST数据上报的完整开发流程,包括模块初始化、PDP上下文配置、HTTP POST实现及优化策略。针对实际开发中的常见问题如网络异常、低功耗需求等提供了经过验证的解决方案,帮助开发者快速实现稳定可靠的物联网数据传输功能。

weixin_42556197的博客 481

Netty原理-Bytebuffer

Netty,ByteBuf

csywxr的博客 432

零基础DSP实战TMS320F28035:第九节,ADC比较器&EPWM

ADC比较器框图如下所示:-电压-----------------------------输出-电压A>电压B------------------ 1-电压A

sinat_30943509的博客 2535

Netty框架:ByteBuf空间不够时自动扩充空间

Netty框架:ByteBuf空间不够时自动扩充空间

听海边涛声 540

PVE虚拟机运行不流畅,出现卡顿。

🏆本文收录于《优快云问答解惑-专业版》专栏,主要记录项目实战过程中的Bug之前因后果及提供真实有效的解决方案,希望能够助你一臂之力,帮你早日登顶实现财富自由🚀;同时,欢迎大家关注&&收藏&&订阅!持续更新中,up!up!up!!

**My Coding Family** 3064

【C++】ROS:ROS1环境配置与基础示例

ROS1环境配置与基础示例

DevFrank的博客 2104

【Netty4】netty ByteBuf (一)如何创建ByteBuf对象

有一点我们需要知道的是,ByteBuf的jar包,是可以单独使用的。比如某个项目中有一个场景,需要处理某个自定义的协议,那么我们在解析协议时,就可以将接收到的将字节内容写入一个ByteBuf,然后从ByteBuf中慢慢的将内容读取出来。下面让我们用一个例子简单的了解下ByteBuf的使用。 ByteBuf的创建 要想使用ByteBuf,首先肯定是要创建一个ByteBuf,更确切的说法就是要申请一块...

m0_45406092的博客 6124

SAP B1中文培訓教材_SDK

公司内部培训资料,对开发工具 ,DIAPI, UIAPI,Add-ON有详细解说

【Netty源码解读和权威指南】第13篇:Netty ByteBuf全解析(下)—— 内存池技术:Netty高性能的秘密武器

Netty内存池技术解析与性能优化实践 本文深入剖析Netty内存池技术(PooledByteBuf)的核心原理与实现,通过一个真实的高并发性能危机案例,揭示非池化内存管理的痛点:内存分配开销大、GC压力高、内存碎片等问题。文章详细解读了Netty内存池的三层架构设计(分配器PooledByteBufAllocator、内存区域PoolArena、内存块PoolChunk)及其高效的内存管理算法,包括jemalloc启发式的内存分配策略和线程本地缓存机制。 通过对比测试表明,使用内存池后性能提升300%,G

xyghehehehe的专栏 344

jre9下载 jdk9

(Java SE Runtime Environment 9.0) 9.0u175 官方最新版 【x86|x64】

TMS_WebGMaps_1.9.8.0_Full_Source

TMS_WebGMaps_1.9.8.0_Full_Source 全源码 delphi cbuilder

Netty ByteBuf 与 Java NIO ByteBuffer 对比:3个维度解析性能差异与零拷贝实现

本文深入对比了Netty ByteBuf与Java NIO ByteBuffer的性能差异与零拷贝实现,从基础架构、内存管理机制和零拷贝技术三个维度进行解析。Netty通过双指针设计、动态扩容和内存池化等优化策略,显著提升了网络应用的吞吐量和延迟表现,特别适合高性能网络编程场景。

weixin_30314813的博客 401

假人有限元模型

假人模型,车辆碰撞假人有限元模型,适合做乘员安全的研究者利用

Netty高性能网络编程实战:从核心原理到生产环境避坑指南

高性能网络编程是构建高并发、低延迟分布式系统的核心技术基础。其核心原理在于采用异步事件驱动模型,通过Reactor模式将连接管理与事件处理解耦,从而用少量线程支撑海量连接,大幅提升系统吞吐量。这一技术价值在微服务通信、游戏服务器、即时通讯等对实时性要求极高的应用场景中尤为凸显。Netty作为这一领域的标杆框架,通过其精妙的Channel、EventLoop、Pipeline与ByteBuf等组件,将高性能网络编程范式工程化、标准化。本文聚焦Netty的核心架构与实战,深入剖析其事件驱动模型与零拷贝机制,并针

weixin_34082854的博客 412

Netty核心架构与高性能网络编程实战指南

在Java网络编程领域,从传统的BIO(阻塞IO)到NIO(非阻塞IO)的演进,是解决高并发连接瓶颈的关键一步。NIO通过Selector和Channel实现了单线程管理多连接,大幅提升了资源利用率。然而,直接使用原生NIO API需要处理复杂的缓冲区管理、事件分发和线程安全等问题,开发效率低下。Netty作为基于Java NIO的高性能异步事件驱动框架,通过Reactor线程模型和责任链模式,封装了底层复杂性,提供了优雅的编程模型与强大的内存管理能力,其核心价值在于让开发者能专注于业务逻辑,快速构建高可靠

weixin_30402343的博客 334

Java基础之《netty(17)—Unpooled类》

写的时候writerIndex先加,readerIndex在后面追赶,只能读到writerIndex的位置。通过给定的数据和字符编码返回一个ByteBuf对象(类似于NIO中的ByteBuffer但有区别)。1、netty提供一个专门用来操作缓冲区(即netty的数据容器)的工具类。不指定索引,用readByte()方法读,readerIndex会增加。Unpooled类获取netty的数据容器ByteBuf的基本使用。4、readerIndex属性和writerIndex属性。

csj50的专栏 1408

从零学Netty(六)Netty实现Http服务器

编写HttpServlet规范类 Servlet抽象类 /** * Servlet * * @author LionLi */ public abstract class Servlet { public abstract void doGet(Request request,Response response); public abstract void doPost(Request request,Response response); } Request相关功能.

李虹柏的博客 1731

Netty4.0源码解析:字节容器UnpooledHeapByteBuf

一、引言 Java NIO提供了ByteBuffer作为字节容器,供Channel读入和写入数据。但ByteBuffer使用过于繁琐,灵活性不够强。Netty实现了ByteBuf来替代JDK的ByteBuffer。 ByteBuf有以下几大优点: 1、它可以被用户自定义的缓冲区类型扩展 2、通过内置的复合缓冲区类型实现了透明的零拷贝 3、容量可以按需增长 4、读写切换无需调用ByteBuffer的...

APlus 2357

8.基于netty实现群聊,心跳检测

1.本文总结自B站《netty-尚硅谷》,很不错;2.本文示例代码包括。

PacosonSWJTU的博客 1005

【JAVA 网络编程系列】Netty -- 简介

【JAVA 网络编程系列】Netty -- 简介 【1】Netty 整体框架 【2】Netty 模块 【3】Netty 中的核心组件 【3.1】Bootstrap 与 ServerBootstrap Bootstrap(用于客户端引导) 与 ServerBootstrap(用于服务端引导) 是 Netty 程序的引导类,主要用于配置各种参数,并启动整个 Netty 服务; 【3.2】EventLoopGroup EventLoopGroup 可以理解为一个线程池,对于服务端程序,一

CopperSun 技术博客 824
上一篇: netty 抽象字节buf分配器
下一篇: netty Pooled字节buf分配器
Donald_Draper
博客等级 码龄14年 9粉丝 106原创
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符  | 博主筛选后可见
 
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付:10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值