AWSSDK.S3 > v3.7.10.1 AmazonS3Client Forbidden Response

[ajangles]

I am trying to instantiate the AmazonS3Client via DI in a ASP.NET 6 project and it seems any version above 3.7.10.1 will just throw a forbidden error when trying to access s3. I cannot find any information on what might have changed after this version?

This is debugging via vs2022, the credentials file is configured under %userprofile%/.aws/credentials, this is also deployed to an ECS instance and I'm seeing the same issue.

Methods mainly utilised are

• GetObjectMetadataRequest
• PutObjectAsync

Registering the service with DI:

services.AddAWSService<IAmazonS3>();

(services.AddDefaultAWSOptions(Configuration.GetAWSOptions()) or the like is not used, and didn't anyway change the outcome)

In appsettings.json there is the following:

"AWS": { "AWSAccessKey": "AWS_ACCESS_KEY_ID", "AWSSecretKey": "AWS_SECRET_ACCESS_KEY", "AWS_REGION": "ap-southeast-2", ... }

If the version is <= v3.7.10.1 then it works fine, just anything above this gives the error.. What am I missing?

[ashishdhingra]

@ajangles Good afternoon. Thanks for starting the discussion.

Few things to note:

• On your local environment, while debugging via Visual Studio, the application runs under the identity of the current logged in user assuming that you are using built-in Kestrel server. So if you have configured the credentials in your default profile, the credentials should work and you should be able to invoke S3 operations (assuming the credentials have proper IAM policies attached). For testing purposes, I created the below ASP.NET Core 6 project and it worked fine.
  .csproj

<Project Sdk="Microsoft.NET.Sdk.Web">

  <PropertyGroup>
    <TargetFramework>net6.0</TargetFramework>
    <Nullable>enable</Nullable>
    <ImplicitUsings>enable</ImplicitUsings>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="AWSSDK.Extensions.NETCore.Setup" Version="3.7.7" />
    <PackageReference Include="AWSSDK.S3" Version="3.7.104.28" />
  </ItemGroup>

</Project>

Program.cs

using Amazon.S3;
using Microsoft.Extensions.Configuration;

var builder = WebApplication.CreateBuilder(args);

// Add services to the container.
builder.Services.AddRazorPages();
builder.Services.AddDefaultAWSOptions(builder.Configuration.GetAWSOptions());
builder.Services.AddAWSService<IAmazonS3>();

var app = builder.Build();

// Configure the HTTP request pipeline.
if (!app.Environment.IsDevelopment())
{
    app.UseExceptionHandler("/Error");
    // The default HSTS value is 30 days. You may want to change this for production scenarios, see https://aka.ms/aspnetcore-hsts.
    app.UseHsts();
}

app.UseHttpsRedirection();
app.UseStaticFiles();

app.UseRouting();

app.UseAuthorization();

app.MapRazorPages();

app.Run();

Pages\Index.cshtml

@page
@model IndexModel
@{
    ViewData["Title"] = "Home page";
}

@if (Model.ObjectMetadata != null)
{
    <b>Object Content Length:</b> @Model.ObjectMetadata.ContentLength
}
<div class="text-center">
    <h1 class="display-4">Welcome</h1>
    <p>Learn about <a href="https://docs.microsoft.com/aspnet/core">building Web apps with ASP.NET Core</a>.</p>
</div>

Pages\Index.cshtml.cs

using Amazon.S3;
using Amazon.S3.Model;
using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.Mvc.RazorPages;

namespace ASPNetCore6S3DITest.Pages
{
    public class IndexModel : PageModel
    {
        private readonly ILogger<IndexModel> _logger;
        private readonly IAmazonS3 _amazonS3;

        public IndexModel(ILogger<IndexModel> logger, IAmazonS3 amazonS3)
        {
            _logger = logger;
            _amazonS3 = amazonS3;
        }

        [BindProperty]
        public GetObjectMetadataResponse ObjectMetadata { get; set; } = default;

        public async Task<IActionResult> OnGetAsync(string bucketName, string key)
        {
            if (!string.IsNullOrEmpty(bucketName) && !string.IsNullOrEmpty(key) && _amazonS3 != null)
            {
                ObjectMetadata =  await _amazonS3.GetObjectMetadataAsync(bucketName, key);
                
            }

            return Page();
        }
    }
}

The code works fine and the content length is displayed on the home page when query string parameters bucketName and key are passed. Kindly take note of the NuGet packages used in the project.

• The builder.Services.AddDefaultAWSOptions(builder.Configuration.GetAWSOptions()) reads the AWS options configured in appsettings.json under AWS key. Kindly refer Allowed values in appsettings file at Using AWSSDK.Extensions.NETCore.Setup and the IConfiguration interface. I'm unsure why you have configured AWSAccessKey, AWSSecretKey and AWS_REGION in appsettings.json. These are not supported.
• If you are executing your application under IIS, then you might leverage ProfilesLocation to pass absolute path of your credentials file. Kindly note that application pool identity should have proper access to read this file.
• When running under ECS, you should leverage Task IAM role with proper IAM permissions to invoke service operations.

Forbidden Response normally means credentials used do not have access to invoke service API operation.

You may also turn on verbose logging while debugging locally using below statements before making S3 call (or preferably before builder.Services.AddAWSService<IAmazonS3>();). This should log details AWS logs in VS launched console window during debugging:

Amazon.AWSConfigs.LoggingConfig.LogResponses = Amazon.ResponseLoggingOption.Always;
Amazon.AWSConfigs.LoggingConfig.LogTo = Amazon.LoggingOptions.SystemDiagnostics;
Amazon.AWSConfigs.AddTraceListener("Amazon", new System.Diagnostics.ConsoleTraceListener());

Thanks,
Ashish

[ajangles]

Hi Ashish, thank you for your reply, the verbose logging helped me find the answer. It seems that <= v3.7.10.1 you could set the path in the BucketName property, but that now no longer works

var request = new GetObjectMetadataRequest()
 {
          BucketName = bucketName, //eg - bucket/folderOne/folderTwo
          Key = fileKey, //eg - someFile.pdf 
};

I saw that it was escaping some of the '/' characters in the path incorrectly in the verbose logging:

Amazon Verbose: 0 : Single encoded /bucketName%2FfolderOne%2FfolderTwo/someFile.pdf with endpoint https://s3.ap-southeast-2.amazonaws.com/bucketName%2FfolderOne%2FfolderTwo for canonicalization: /bucketName%252FfolderOne%252FfolderTwo/someFile.pdf

Changing the request to the following has resolved the issue:

var request = new GetObjectMetadataRequest()
 {
          BucketName = bucketName, //eg - bucket
          Key = fileKey, //eg - folderOne/folderTwo/someFile.pdf 
};

[ashishdhingra]

@ajangles Thanks for the confirmation. You appear to be affected by the issue #2622 where existing issue was somehow allowing to suffix folder name (S3 doesn't have concept of folders) in bucket name parameter.

[github-actions[bot]]

Hello! Reopening this discussion to make it searchable.