1 GB Bottlerocket Remix

[stefansundin]

Hello everyone, just wanted to quickly show off something I made for fun. Not sure I want to maintain this in the long run but I am interested in learning more about Bottlerocket and the best way to do that is to experiment.

I have a weird hobby and that is to minimize cost on AWS. I like to run single instances that are self-contained and run a single application as cheaply as possible. You can run a small spot instance for about $1 per month, but that's only counting the compute resource, you have to pay about $0.80 per month for a 8 GB EBS volume ($0.40 if you are ok with magnetic). That's about $1.80 per month in total.

What is cool about Bottlerocket is that the root volume can be smaller than usual. Most traditional OS amis come with an 8 GB root volume which you can't make smaller. But Bottlerocket's root volume is 2 GB by default. It makes perfect sense since the root partition is read-only so there's no need to give it more space than necessary, just make the data volume bigger if needed.

Anyway, I was curious if it was possible to make the root volume 1 GB. It should be possible since Bottlerocket comes with two sets of partitions to enable upgrades (see here to learn how the partitions are set up), so what if you just remove half of the partitions. That should free up enough space right? I don't care about upgrading the instances in place, I can upgrade them by replacing them instead.

Yes, it is indeed possible to remove the "B" partitions. And it wasn't hard either. In very little time, I managed to build my own 1 GB ami and use it to run stuff. The Bottlerocket build system is great, so kudos for making it easy to use!

It turns out that I also had a lot of empty space left over on the 1 GB volume. I decided to turn the free space into a swap partition. I'm happy that my PR to enable swap with ECS was accepted (#2537), so I can keep using that when I use the official Bottlerocket release, but when I use my custom image I will just use the swap partition instead. Since my image can't do OS upgrades, I also decided to strip out some of the tooling used to support upgrading, which made it possible to make the swap partition a little bit larger. I was expecting this to be harder but it is awesome to see that there aren't huge dependencies between all the programs.

So with a 1 GB root volume and a 2 GB data volume, you can spend about $0.30 ($0.15 if you are ok with magnetic) on EBS volumes instead of $0.80. So with this you can run some of the most cost efficient instances on the planet. If you run smaller images than me then you can probably get away with a 1 GB data volume.

Okay, I've talked enough. Here's the link to my branch: https://github.com/stefansundin/bottlerocket/tree/1gb

See the last commit for the changes necessary. I hope it is useful for someone! Please let me know below if you have any questions.

Here's the final partition layout:

# partx -s /dev/nvme0n1
NR   START     END SECTORS SIZE NAME                 UUID
 1    2048    6143    4096   2M EFI-SYSTEM           cca50ddc-00ba-4fe7-bfdb-8ddc32e43f44
 2    6144   77823   71680  35M BOTTLEROCKET-BOOT-A  b92f002d-a062-4db9-9134-a89723a040e6
 3   77824 1060863  983040 480M BOTTLEROCKET-ROOT-A  61afcbe9-0929-4fe7-aef5-05dcf4b56a98
 4 1060864 1069055    8192   4M BOTTLEROCKET-HASH-A  e1d89f48-09ea-4a67-9320-a874b5583b25
 5 1069056 1105919   36864  18M BOTTLEROCKET-PRIVATE f95c1576-a5c7-452f-acb9-7201abcb92d2
 6 1105920 2095103  989184 483M BOTTLEROCKET-SWAP    ba8a5943-9037-490c-8b81-dbfe13b50715

P.S. It might be cool to have a third partition plan which skips the B partition bank. There's currently split and unified but what if there was an immutable plan indicating that it doesn't support upgrades? Just a thought.

[FernandoMiguel]

for all of us that use bottlerocket on EKS, it makes sense to not have the update partition, since the instance is always replaced with the newer AMI.
Wonder if it would be possible to have diff AMIs with and without the upgrade partition ?

[stefansundin]

I think it would definitely be possible. I haven't tried but I think you can compile my branch for the Kubernetes variant that you want. I think you just need to change is the partition sizes. Since Kubernetes doesn't like swap, I think you would just remove the swap partition that I added and give the root partition that space instead.

[webern]

This is a cool project. There does seem to be a use-case for immutable/replaceable Bottlerocket (i.e. without the update system and extra partitions). Thank you for opening a discussion about this.