signature=03ac1030b0ebd0ce0374e882fe0aa1fb,Bug #21003: chunk signature mismatch for AWSv4 and Oracle...

Unfortunately this PR doesn't solve the problem for us. We attached the entire client and server log (running with your patch), and the wireshark dump from the client side.

The client starting the chunked upload and first two chunks:

STREAMING-AWS4-HMAC-SHA256-PAYLOAD

nhpAWS4Authentication: string-to-sign

AWS4-HMAC-SHA256

20170816T115603Z

20170816/us-east-1/s3/aws4_request

6a93f48bcc8d274117f48a50dc84c5cfd812b3fb7edc5be7d763459647423c62

nhp: > Authorization:

nhp: >

nhp: > write 16384 bytes

nhpAWS4EncWrite: string-to-sign

AWS4-HMAC-SHA256-PAYLOAD

20170816T115603Z

20170816/us-east-1/s3/aws4_request

f69993822618a847e0e101d9fff756471f3faed67808890e1b6fb536c80c98c0

e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

b3d650f1ee6a4f131076f307ac5eb6ff9563ab3714eacf963e093238733990d4

nhp: 2017-08-16 13:56:03.569239 send enter 0x7fd8353ec740 2677

nhp: 2017-08-16 13:56:03.569292 send return 2677 0 0 elapsed +0 00:00:00.000053

nhp: 2017-08-16 13:56:03.569300 send enter 0x7fd8353ec740 4

nhp: 2017-08-16 13:56:03.569310 send return 4 0 0 elapsed +0 00:00:00.000010

nhp: 2017-08-16 13:56:03.569315 send enter 0x7fd8353ec740 87

nhp: 2017-08-16 13:56:03.569328 send return 87 0 0 elapsed +0 00:00:00.000013

nhp: 2017-08-16 13:56:03.569333 send enter 0x7fd8353ec740 2

nhp: 2017-08-16 13:56:03.569343 send return 2 0 0 elapsed +0 00:00:00.000010

nhp: 2017-08-16 13:56:03.569349 send enter 0x7fd8353ec740 6

nhp: 2017-08-16 13:56:03.569357 send return 6 0 0 elapsed +0 00:00:00.000008

nhp: 2017-08-16 13:56:03.569362 send enter 0x7fd8353ec740 16384

nhp: 2017-08-16 13:56:03.569381 send return 16384 0 0 elapsed +0 00:00:00.000019

nhp: 2017-08-16 13:56:03.569387 send enter 0x7fd8353ec740 2

nhp: 2017-08-16 13:56:03.569393 send return 2 0 0 elapsed +0 00:00:00.000006

nhp: > write 16384 bytes

nhpAWS4EncWrite: string-to-sign

AWS4-HMAC-SHA256-PAYLOAD

20170816T115603Z

20170816/us-east-1/s3/aws4_request

d61f8acc559fdf781581b6eda77de720360a2786f1d8449aa10654cf73997487

e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

e33f24140499430b048f6600af4f41f3ccb0cb766d9f7661124cf8ba4b827523

nhp: 2017-08-16 13:56:03.569633 send enter 0x7fd8353ec740 4

nhp: 2017-08-16 13:56:03.569654 send return 4 0 0 elapsed +0 00:00:00.000021

nhp: 2017-08-16 13:56:03.569659 send enter 0x7fd8353ec740 89

nhp: 2017-08-16 13:56:03.569675 send return 89 0 0 elapsed +0 00:00:00.000016

nhp: 2017-08-16 13:56:03.569680 send enter 0x7fd8353ec740 2

nhp: 2017-08-16 13:56:03.569690 send return 2 0 0 elapsed +0 00:00:00.000010

nhp: 2017-08-16 13:56:03.569695 send enter 0x7fd8353ec740 6

nhp: 2017-08-16 13:56:03.569709 send return 6 0 0 elapsed +0 00:00:00.000014

nhp: 2017-08-16 13:56:03.569714 send enter 0x7fd8353ec740 16384

nhp: 2017-08-16 13:56:03.569731 send return 16384 0 0 elapsed +0 00:00:00.000017

nhp: 2017-08-16 13:56:03.569737 send enter 0x7fd8353ec740 2

nhp: 2017-08-16 13:56:03.569742 send return 2 0 0 elapsed +0 00:00:00.000005

The wireshark dump from the client:

Authorization: AWS4-HMAC-SHA256 Credential=5QB12EADJ3HTU0PR5TQZ/20170816/us-east-1/s3/aws4_request, SignedHeaders=content-encoding;content-type;date;host;x-amz-content-sha256;x-amz-date;x-amz-decoded-content-length;x-amz-meta-blocksize;x-amz-meta-chunkidformat;x-amz-meta-chunkprefix;x-amz-meta-chunks;x-amz-meta-chunksize;x-amz-meta-chunktype;x-amz-meta-copynumber;x-amz-meta-dbid;x-amz-meta-dbname;x-amz-meta-filename;x-amz-meta-filesize;x-amz-meta-filetype;x-amz-meta-incarnation;x-amz-meta-libraryname;x-amz-meta-node;x-amz-meta-opentime;x-amz-meta-sbtrequest;x-amz-meta-sbtretrycount;x-amz-meta-sbtversion;x-amz-meta-sessionid;x-amz-meta-status;x-amz-meta-system;x-amz-meta-tickloc;x-amz-meta-user, Signature=f69993822618a847e0e101d9fff756471f3faed67808890e1b6fb536c80c98c0

57

4000;chunk-signature=d61f8acc559fdf781581b6eda77de720360a2786f1d8449aa10654cf73997487

4000

.....

Then here's rgw detecting the chunked upload:

STREAMING-AWS4-HMAC-SHA256-PAYLOAD

2017-08-16 13:56:03.570349 7f4bdc4f5700 10 canonical request hash = 6a93f48bcc8d274117f48a50dc84c5cfd812b3fb7edc5be7d763459647423c62

2017-08-16 13:56:03.570358 7f4bdc4f5700 10 string to sign = AWS4-HMAC-SHA256

20170816T115603Z

20170816/us-east-1/s3/aws4_request

6a93f48bcc8d274117f48a50dc84c5cfd812b3fb7edc5be7d763459647423c62

2017-08-16 13:56:03.570387 7f4bdc4f5700 10 body content detected in multiple chunks

2017-08-16 13:56:03.570388 7f4bdc4f5700 10 aws4 seed signature ok... delaying v4 auth

2017-08-16 13:56:03.570503 7f4bdc4f5700 10 date_k = aab84507d9ebbbeebff8608f793ad0cfc4c07b757c28829bc331a9866ea04ee7

2017-08-16 13:56:03.570515 7f4bdc4f5700 10 region_k = 301517aefddf7433ec49a49f2ba5a690e7a36a1c5176b4b02ffd324ba6202d85

2017-08-16 13:56:03.570518 7f4bdc4f5700 10 service_k = 5ee8c521c9d09722fed63a9435c3b00c69b5ac29566cf95d8edcaac9d037515c

2017-08-16 13:56:03.570528 7f4bdc4f5700 10 signing_k = ec979262faf07db88a9631f4fc75bb3954823ea84749c05fb8834088a65cd841

2017-08-16 13:56:03.570552 7f4bdc4f5700 10 generated signature = f69993822618a847e0e101d9fff756471f3faed67808890e1b6fb536c80c98c0

2017-08-16 13:56:03.570557 7f4bdc4f5700 15 string_to_sign=AWS4-HMAC-SHA256

20170816T115603Z

20170816/us-east-1/s3/aws4_request

6a93f48bcc8d274117f48a50dc84c5cfd812b3fb7edc5be7d763459647423c62

2017-08-16 13:56:03.570569 7f4bdc4f5700 15 server signature=f69993822618a847e0e101d9fff756471f3faed67808890e1b6fb536c80c98c0

2017-08-16 13:56:03.570570 7f4bdc4f5700 15 client signature=f69993822618a847e0e101d9fff756471f3faed67808890e1b6fb536c80c98c0

2017-08-16 13:56:03.570571 7f4bdc4f5700 15 compare=0

2017-08-16 13:56:03.570618 7f4bdc4f5700 10 date_k = aab84507d9ebbbeebff8608f793ad0cfc4c07b757c28829bc331a9866ea04ee7

2017-08-16 13:56:03.570627 7f4bdc4f5700 10 region_k = 301517aefddf7433ec49a49f2ba5a690e7a36a1c5176b4b02ffd324ba6202d85

2017-08-16 13:56:03.570631 7f4bdc4f5700 10 service_k = 5ee8c521c9d09722fed63a9435c3b00c69b5ac29566cf95d8edcaac9d037515c

2017-08-16 13:56:03.570634 7f4bdc4f5700 10 signing_k = ec979262faf07db88a9631f4fc75bb3954823ea84749c05fb8834088a65cd841

And failing on the first chunk:

2017-08-16 13:56:03.570755 7f4bdc4f5700 2 req 12:0.001047:s3:PUT /oracle-data-miropoto-1/file_chunk/0/SBTDB/unknown/2017-08-16/crap/ZyH3tH16jobK/0000000001:put_obj:executing

2017-08-16 13:56:03.570939 7f4bdc4f5700 20 parsed new chunk; signature=d61f8acc559fdf781581b6eda77de720360a2786f1d8449aa10654cf73997487, data_length=87, data_starts_in_stream=91

2017-08-16 13:56:03.570952 7f4bdc4f5700 30 AWSv4ComplMulti: stream_pos_was=91, to_extract=87

2017-08-16 13:56:03.570954 7f4bdc4f5700 30 AWSv4ComplMulti: to_extract=87, data_len=10

2017-08-16 13:56:03.570962 7f4bdc4f5700 30 AWSv4ComplMulti: to_extract=77, received=77

2017-08-16 13:56:03.570964 7f4bdc4f5700 20 AWSv4ComplMulti: filled=87

2017-08-16 13:56:03.571023 7f4bdc4f5700 20 AWSv4ComplMulti: string_to_sign=

AWS4-HMAC-SHA256-PAYLOAD

20170816T115603Z

20170816/us-east-1/s3/aws4_request

f69993822618a847e0e101d9fff756471f3faed67808890e1b6fb536c80c98c0

e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

3a79b501bbeaa09e83a00e71ecb07b3acd43acba55b805f393f543edf6c09376

2017-08-16 13:56:03.571054 7f4bdc4f5700 20 AWSv4ComplMulti: ERROR: chunk signature mismatch

2017-08-16 13:56:03.571058 7f4bdc4f5700 20 AWSv4ComplMulti: declared signature=d61f8acc559fdf781581b6eda77de720360a2786f1d8449aa10654cf73997487

2017-08-16 13:56:03.571059 7f4bdc4f5700 20 AWSv4ComplMulti: calculated signature=bd1a20b0df8a661c5a736bad4703dfc0e0be28ac77c224fd71a9dd44d91697ce