亚马逊的S3分块下载

最新推荐文章于 2024-07-14 12:04:38 发布

原创最新推荐文章于 2024-07-14 12:04:38 发布 · 2.5k 阅读

5 ·

CC 4.0 BY-SA版权

文章标签：

#S3 #亚马逊 #下载 #分块 #分段

Java 专栏收录该内容

5 篇文章

订阅专栏

博客提供了网易云—S3对象存储帮助文档的参考链接，涉及S3对象存储相关内容，可能对S3存储的下载、分块等操作有指导作用。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

 @Test
    public void testS3() throws Exception{
        String mainBuckt = awsS3Component.getMainBuckt();
        String s3Key = 9c8bd_2019-05-20.tar.gz";
        AmazonS3 s3Client = awsS3Component.getInstance();
        long length = awsS3Component.length(mainBuckt, s3Key);
        long a = length%100L;
        System.out.println("总长度："+length);
        //因为是压缩包格式，所以不适用分段读取内容，会乱码
        for (int i=0;length>i;i=i+1024){
            GetObjectRequest getObjectRequest = new GetObjectRequest(mainBuckt,s3Key);
            getObjectRequest.setRange(i, i+1024);
            S3Object nosObject = s3Client.getObject(getObjectRequest);
            long contentLength = nosObject.getObjectMetadata().getContentLength();
            InputStream objectContent = nosObject.getObjectContent();
            BufferedReader reader = null;
            //new BufferedReader(new InputStreamReader(objectContent, "GBK"));
            //解压
            if (s3Key.endsWith(".lzo_deflate")) {
                //lzo解压缩
                LzoAlgorithm algorithm = LzoAlgorithm.LZO1X;
                LzoDecompressor deCompressor = LzoLibrary.getInstance().newDecompressor(algorithm, null);
                LzoInputStream stream = new LzoInputStream(objectContent, deCompressor);
                reader = new BufferedReader(new InputStreamReader(stream));
            } else if (s3Key.endsWith(".gz")) {
                GZIPInputStream gzipInputStream = new GZIPInputStream(objectContent);
                reader = new BufferedReader(new InputStreamReader(gzipInputStream));
            } else {
                reader = new BufferedReader(new InputStreamReader(objectContent));
            }
            while (true) {
                String line;
                try {
                    line = reader.readLine();
                    if (line == null) break;
                    System.out.println("\n" + line);
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
            System.out.println("ok!断点续传标记数:"+i);
            try {
                reader.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }

参考链接：网易云—S3对象存储帮助文档