我的方法接收一个缓冲的读卡器,并转换文件中的每一行。但是,我需要将这个转换的输出上传到一个S3桶。这些文件非常大,所以我希望能够将我的上传流到S3对象中。
要做到这一点,我想我需要使用多部分上传,但是我不确定我是否正确地使用它,因为似乎没有什么上传。
以下是我的方法:
public void transform(BufferedReader reader)
{
Scanner scanner = new Scanner(reader);
String row;
List partETags = new ArrayList<>();
InitiateMultipartUploadRequest request = new InitiateMultipartUploadRequest("output-bucket", "test.log");
InitiateMultipartUploadResult result = amazonS3.initiateMultipartUpload(request);
while (scanner.hasNext()) {
row = scanner.nextLine();
InputStream inputStream = new ByteArrayInputStream(row.getBytes(Charset.forName("UTF-8")));
log.info(result.getUploadId());
UploadPartRequest uploadRequest = new UploadPartRequest()
.withBucketName("output-bucket")
.withKey("test.log")
.withUploadId(result.getUploadId())
.withInputStream(inputStream)
.withPartNumber(1)
.withPartSize(5 * 1024 * 1024);
partETags.add(amazonS3.uploadPart(uploadRequest).getPartETag());
}
log.info(result.getUploadId());
CompleteMultipartUploadRequest compRequest = new CompleteMultipartUploadRequest(
"output-bucket",
"test.log",
result.getUploadId(),
partETags);
amazonS3.completeMultipartUpload(compRequest);
}