Seperate docker builds for non-prod lake formats #39
Conversation
istreeter
force-pushed
the
separate-docker-builds
branch
4 times, most recently
from
6b305a4 to
8616c11
Compare
istreeter
force-pushed
the
separate-docker-builds
branch
from
8616c11 to
4d1b431
Compare
| externalId <- Option(conf.getTrimmed("fs.s3a.assumed.role.session.external.id")) | ||
| } yield StsAssumeRoleCredentialsProvider.builder | ||
| .stsClient { | ||
| StsClient.builder.defaultsMode(DefaultsMode.AUTO).build |
There was a problem hiding this comment.
I haven't used the AUTO mode before. Is it something we've tested? They suggest it's experimental, but I guess it's safe. Just wondered whether you've considered it.
There was a problem hiding this comment.
When working on common-streams, we decided to trust the defaults mode on the kinesis source/sink. That PR is here but unfortunately the conversation must have happened somewhere private.
The benefits are.... no need to litter our config files with retry settings for each sdk call. No need to implement the retry-handling. Our code can trust the sdk about what error classes are re-tryable.
Is it something we've tested?
No. To find out whether this is a good idea or not... I think we need to just get it into production on some low-priority apps, and see if proves to be stable over long time periods (many months).
They suggest it's experimental,
Out of interest, where did you read that suggestion? I was following this blog post and documentation such as this.
There was a problem hiding this comment.
Understood. I was wondering about the choice. This clarifies this. Thanks.
As for why it's experimental: same docs just specifically for AUTO mode.
No description provided.