MappedByteBuffer 读取超过2G文件

最新推荐文章于 2025-05-28 23:55:09 发布

zoQ

最新推荐文章于 2025-05-28 23:55:09 发布

阅读量5.1k

点赞数

CC 4.0 BY-SA版权

分类专栏： nio java Java粗浅认识

本文链接：https://blog.youkuaiyun.com/bpz31456/article/details/82255985

java 同时被 3 个专栏收录

18 篇文章

订阅专栏

Java粗浅认识

16 篇文章

订阅专栏

nio

4 篇文章

订阅专栏

介绍了使用MappedByteBuffer处理大文件的方法，因一次只能读2G内容到内存，需循环读取处理。还提到要注意移位运算，使用java.nio.channels.FileChannel#map方法，其size最大为Integer.MAX_VALUE（2G）。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

MappedByteBuffer处理大文件，一次只能读2G内容到内存中，为了读取大文件，需要循环读取处理。

protected void head() {
        //512m
        long length = 1L << 29;
        //4g
        long _4G = 1L << 32;
        long cur = 0L;
        try {
            MappedByteBuffer mappedByteBuffer;
            Random random = new Random();
            while (cur < _4G) {
                mappedByteBuffer = new RandomAccessFile("D:\\test\\bigfile.txt", "rw").getChannel()
                        .map(FileChannel.MapMode.READ_WRITE, cur, length);
                IntBuffer intBuffer = mappedByteBuffer.asIntBuffer();
                while (intBuffer.position() < intBuffer.capacity()) {
                    intBuffer.put(random.nextInt());
                }
                cur += length;
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

需要注意的地方：移位运算(<<)

//这里的lL一定要带上L，不然是已int来计算的，结果会越界
long _4G = 1L << 32;

    @Override
    protected void head() {
        long _4g = 1<<32;
        //结果：1
        System.out.println(_4g);
        long _4G = 1L<<32;
        //结果：4294967296
        System.out.println(_4G);
    }

方法：java.nio.channels.FileChannel#map，这里的siez最大 = Integer.MAX_VALUE(2G)

    /*
     * @param  mode
     *         One of the constants {@link MapMode#READ_ONLY READ_ONLY}, {@link
     *         MapMode#READ_WRITE READ_WRITE}, or {@link MapMode#PRIVATE
     *         PRIVATE} defined in the {@link MapMode} class, according to
     *         whether the file is to be mapped read-only, read/write, or
     *         privately (copy-on-write), respectively
     *
     * @param  position
     *         The position within the file at which the mapped region
     *         is to start; must be non-negative
     *
     * @param  size
     *         The size of the region to be mapped; must be non-negative and
     *         no greater than {@link java.lang.Integer#MAX_VALUE}
     *
     * @return  The mapped byte buffer
     *
     * @throws NonReadableChannelException
     *         If the <tt>mode</tt> is {@link MapMode#READ_ONLY READ_ONLY} but
     *         this channel was not opened for reading
     *
     * @throws NonWritableChannelException
     *         If the <tt>mode</tt> is {@link MapMode#READ_WRITE READ_WRITE} or
     *         {@link MapMode#PRIVATE PRIVATE} but this channel was not opened
     *         for both reading and writing
     *
     * @throws IllegalArgumentException
     *         If the preconditions on the parameters do not hold
     *
     * @throws IOException
     *         If some other I/O error occurs
     *
     * @see java.nio.channels.FileChannel.MapMode
     * @see java.nio.MappedByteBuffer
     */
    public abstract MappedByteBuffer map(MapMode mode,
                                         long position, long size)
        throws IOException;