原文链接：点击打开链接
避免原文打不开，记录如下：
背景：
当然，以上测试程序均在PC java端运行。真正要移植到Android上进行内存映射，还需要：
ByteBuffer littleEndian = ByteBuffer.allocateDirect(11160 * 4).order(ByteOrder.LITTLE_ENDIAN);
i = 0;
while (i < 11160) {
littleEndian.putFloat(cubeTextureCoordinateData[i++]);
}
littleEndian.flip();
try {
FileOutputStream fos = new FileOutputStream("memorymap_texture_little_endian");
FileChannel fc = fos.getChannel();
fc.write(littleEndian);
fc.close();
fos.close();
} catch (IOException e) {
e.printStackTrace();
}
Power of Java MemoryMapped File
In JDK 1.4 an interesting feature of Memory mapped file was added to Java, which allows to map any file to OS memory for efficient reading. A memory mapped file can be used to develop an IPC type of solution. This article is an experiment with memory mapped file to create IPC.
Some details about Memory Mapped File, definition from WIKI
A memory-mapped file is a segment of virtual memory which has been assigned a direct byte-for-byte correlation with some portion of a file or file-like resource. This resource is typically a file that is physically present on-disk, but can also be a device, shared memory object, or other resource that the operating system can reference through a file descriptor. Once present, this correlation between the file and the memory space permits applications to treat the mapped portion as if it were primary memory.
Sample Program
Below we have two Java programs, one is a writer and the other is a reader. The writer is the producer and tries to write to Memory Mapped file, the reader is the consumer and it reads messages from the memory mapped file. This is just a sample program to show you the idea, it does’t handle many edge cases but it is good enough to build something on top of a memory mapped file.
MemoryMapWriter
MemoryMapReader
Observation
Using a memory mapped file can be a very good option for developing Inter Process communication, throughput is also reasonably well for both produce & consumer. Performance stats by run producer and consumer together:
Each message is one long number
Produce – 10 Million message – 16(s)
Consumer – 10 Million message 0.6(s)
A very simple message is used to show you the idea, but it can be any type of complex message, but when there is complex data structure then serialization can add to overhead. There are many techniques to get over that overhead. More in next blog.
Java NIO - Channels and Memory mapping
Channel Classes
- Channel- A channel represents a connection to a file, socket or any other component that can perform IO operations. A channel reads or writes data to a byte buffer. Channels are generally thread safe. A channel may be specified to be read only or both readable and writable.
- ReadableByteChannel - A ReadableByteChannel permits read operation on a buffer, allowing only one thread to read at a time.
- WritableByteChannel - A WritableByteChannel permits write operation on a buffer, allowing only one thread to read at a time.
- ByteChannel - A ByteChannel extends both ReadableByteChannel and WritableByteChannel and allows both read and write.
- SeekableByteChannel - A SeekableByteChannel extends ByteChannel and allows to maintain and modify the current position on the underlying entity to which it is connected. It has methods to get the size of the underlying entity or truncate it to a given size, if permitted.
- GatheringByteChannel-A GatheringByteChannel is used to write data from multiple buffers into a single channel.
- ScatteringByteChannel-A ScatteringByteChannel is used to read data from a channel into multiple buffers.
File Channel
A FileChannel is used to read and write data to a file. It implements seekableByteChannel, ScatteringByteChannel and GatheringByteChannel. It is possible to map a region of file directly into memory. Data can be transferred to another channel or from another channel using the transferTo(..) and transferFrom(..) methods. These methods use the underlying optimization of the operating system. File locking can be applied to manage access between multiple processes. The methods are thread safe and a thread that wishes to modify the position may be blocked until another thread is acting upon the file.
Scatter Read using File Channel
Data from a File Channel can be read into multiple buffers. This is known as a scatter read. Here's and example demonstrating a scatter read.
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
|
// create a temp file
Path tempFile =
Files.createTempFile(
"fileChannelExample"
,
"txt"
,
new
FileAttribute<?>[
0
]);
// write some data to this file
PrintWriter writer =
new
PrintWriter(tempFile.toFile());
writer.println(
"First Line"
);
writer.println(
"Second Line"
);
writer.println(
"Third Line"
);
writer.close();
// get an input stream for this file.
FileInputStream in =
new
FileInputStream(tempFile.toFile());
// get the fileChannel for this input stream
FileChannel fileChannel = in.getChannel();
// get the position
System.out.println(fileChannel.position());
//print 0
// create a char buffer
ByteBuffer buffer = ByteBuffer.allocate(
11
);
// read the fist line - 10 characters into the byte buffer
fileChannel.read(buffer);
// get the position of the file
System.out.println(fileChannel.position());
// prints 11
// we have read the first 10 chars into the buffer now. find out the
// total size of the file
System.out.println(fileChannel.size());
// prints 37
// we read the next 27 characters in two buffers using a scattering read
ByteBuffer buffer1 = ByteBuffer.allocate(
14
);
ByteBuffer buffer2 = ByteBuffer.allocate(
12
);
fileChannel.read(
new
ByteBuffer[] { buffer1, buffer2 });
// lets see the contents of the buffers
buffer1.flip();
buffer2.flip();
System.out.print(Charset.defaultCharset().decode(buffer1));
// prints "Second Line"
System.out.print(Charset.defaultCharset().decode(buffer2));
// prints "Third Line"
fileChannel.close();
|
Scatter Write using File Channel
we can write the bytes back to the file using a scattering write. The filechannel was created on the inputstream so the channel is only readable but not writable. we create a filechannel from an output stream
|
1
2
3
4
5
6
7
8
9
10
11
|
FileOutputStream out =
new
FileOutputStream(tempFile.toFile());
FileChannel fileOutputChannel = out.getChannel();
ByteBuffer buffer3 = Charset.defaultCharset().encode(
CharBuffer.wrap(
"Line1\n"
.toCharArray()));
ByteBuffer buffer4 = Charset.defaultCharset().encode(
CharBuffer.wrap(
"Line2"
.toCharArray()));
fileOutputChannel.write(
new
ByteBuffer[] { buffer3, buffer4 });
// we force the update to the file
fileOutputChannel.force(
true
);
// lets look at the position in the file
System.out.println(fileOutputChannel.position());
|
Reading Huge Files using Memory Mapped Buffer
Java NIO allows reading huge files directly from the memory. i.e. The file need not be loaded into the JVM. All reads and writes on the byte buffer happen direclty on the file.
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
|
import
java.io.File;
import
java.io.IOException;
import
java.io.RandomAccessFile;
import
java.nio.MappedByteBuffer;
import
java.nio.channels.FileChannel;
public
class
ReadingHugeFilesUsingMemoryMappedBuffer {
/**
* use a MappedByteBuffer to wrap a huge file. Using a MappedByteBuffer does
* not load the file in JVM but reads it directly off the file system
* memory. The file can be opened in read, write or private mode.
*/
// to test you can use any video movie file if you dont have any other large
// file for testing.
private
static
String hugeFile =
"A Huge File"
;
public
static
void
main(String[] args)
throws
IOException {
File file =
new
File(hugeFile);
FileChannel fileChannel =
new
RandomAccessFile(file,
"r"
).getChannel();
MappedByteBuffer buffer = fileChannel.map(
FileChannel.MapMode.READ_ONLY,
0
, fileChannel.size());
// the buffer now reads the file as if it were loaded in memory. note
// that for smaller files it would be faster
// to just load the file in memory
// lets see if this buffer is loaded fully into memory
System.out.println(buffer.isLoaded());
// the mappedbytebuffer can be used as a normal buffer to do read and/or
// write operations
// read the size
System.out.println(buffer.capacity());
}
}
|
扫一扫
3562
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
