buffer.resize pros and cons

[petrihakkinen]

I've been thinking about pros and cons of buffer.resize lately. Here are my thoughts in case they are useful. This could be extended to a RFC if the developers of Luau want to go in this direction.

Motivation:

1. Would allow buffers to be used when it's infeasible to compute the size of the buffer (e.g. serializing complex binary data currently requires a prepass for computing buffer size which is often infeasible)
2. Would allow shrinking buffers to optimal size if initial guesstimate was too large
3. It's not possible to implement resizing efficiently outside the VM. E.g. lets consider a user function which creates a new buffer and copies the data over. Freeing the old buffer would be subject to garbage collection which can happen much later. Thus memory usage of the app may increase considerably when resizing large buffers.
4. Would allow releasing memory immediately by calling buffer.resize(buf, 0)

Performance considerations:

Performance difference for buffer accesses would be likely negligible for buffers larger than 56 bytes (= 64 - header) because both versions need to access 2 cache lines (assuming 64 byte cache lines). For very small buffers though the current implementation only needs to access one cache line. Assumption is that such very small buffers are rare, so in practice I doubt the performance difference matters. A benchmark could be written to examine this more closely.

With resizable buffers, creating a new buffer would need 2 memory allocations, so creating and freeing a buffer will likely be somewhat slower. This is the only practical downside I can see.

Other:

• Since each buffer uniquely owns its data, should buffer data be allocated from Lua heap or externally malloced? I guess the 1 GB limit comes from the Lua heap, so with external allocation the 1 GB limit could be lifted.
• len in Buffer could be changed from int to size_t without downsides (since GC object needs to be at least 16 bytes).

[vegorov-rbx]

We went over resizable buffers in the initial RFC already.
They were considered and decided against.

[vegorov-rbx]

1Gb size limitation follows the current string size limitation.
2Gb-N (maybe 4Gb-N) is technical limit in our Luau VM implementation, but alternative VM implementations are free to implement an increased limit is they choose so.

[vegorov-rbx]

You can think of a buffer like a memory allocation in C++.
You do int* p = new int[1000]; and you cannot resize that.
That doesn't prevent std::vector from existing.

[petrihakkinen]

The big difference is that C++ has delete. Since there's no way to explicitly delete a buffer in Luau, trying to implement resize/realloc in user code will have a significant memory cost.

[vegorov-rbx]

That's a common thing for garbage-collectible languages.

[petrihakkinen]

But at the same time it's typical to have functions for manually closing a file or releasing scarce system resources. I'd say a memory buffer up to 1GB falls into this category.

btw. the RFC seems to keep the door open for buffer.resize:
"The proposed buffer object is not resizeable; this is possible to implement later using explicit buffer.resize call, however this may result in a performance impact for native implementation as the data will be read through a pointer redirection and will be more difficult to optimize; thus, this version of the RFC only proposes fixed length buffers. That said, if resizeable buffers are desired in the future, we would plan to enhance the current buffer type instead of making a parallel resizeable buffer type to reduce complexity."

[vegorov-rbx]

The possibility note refers to the fact that such an extension will not be ABI-breaking.
That doesn't mean there was a plan to introduce it later (there wasn't).

[petrihakkinen]

(I'm redacting my previous comment as I don't think it's productive. Seems that we have a different idea of usefulness of buffer.resize. That's fine, I can live with that.)