HDDS-14779. [STS] IAM Session Policy and ListBucket improvements

[fmorg-git]

Please describe your PR in detail:
We need to be able to tell whether an S3 STS call is authorized for listing files in a bucket only vs being able to download a file, because those are different actions in S3 (ListBucket vs GetObject). To differentiate, use LIST at key/object level for listing permission and use READ at key/object level for downloading permission. Currently, the OmMetadataReader authorizes only READ at key level for the listing operations. In order to not break existing functionality, have STS-specific checks that authorize against LIST permission instead. Further, because of how shallow listing works, in some cases (ex. root listing with a delimiter), it loses the context of what the original prefix was on the S3 request. For STS authorization, we need this original prefix, so introduce an optional listPrefix in the protocol to support this.



Here are a quick summary of the changes:

1. Support passing list prefix on root listing so STS can authorize against it.
2. Filter out object-related actions (ex GetObject, PutObject, etc) when Conditions are present since we only support s3:prefix Condition and s3:prefix is not applicable for object-related actions.
3. Keep track of the operator (i.e. StringEquals or StringLike) when using Conditions. If the operator is StringEquals and a wildcard (* or ?) is used, ignore that condition since Ranger can’t authorize against literal asterisk or question mark.
4. Change ListBucket to validate LIST permission on the object instead of READ so we can determine if we should allow listing the object or downloading the object (READ would be for downloading).

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-14779

How was this patch tested?

unit tests, smoke tests