feat: extend manifest integration test

[acha-bill]

Uses feeds to test updates to a website

1. It creates a root feed manifest
2. To make an update to the feed, it uploads a tar website with some index.html and other files. Then uses this reference in the feed update.
3. It then tries to download the website via the root feed reference. It should resolve to the index.html
4. To update the website, we push another update to the feed with the new tar and then try to download it again via the root feed reference.
5. Lastly, it tries to download the other files in the website directly. e.g b25c89a401d9f26811680476619a1eb4a4e189e614bc6161cbfd8b343214917b/assets/styles/styles.css

[gacevicljubisa]

This part of code should be defered when buffer initialized

beekeeper/pkg/check/manifest/manifest.go

Lines 196 to 198 in 0b91024

	if err := tw.Close(); err != nil {
	return nil, err
	}

It will ensure that io.Writer is properly closed even if error occurs during writing.

[gacevicljubisa]

One improvement regarding Options:

beekeeper/pkg/check/manifest/manifest.go

Lines 36 to 46 in 4d02a31

	func NewDefaultOptions() Options {
	return Options{
	FilesInCollection: 10,
	GasPrice: "",
	MaxPathnameLength: 64,
	PostageAmount: 1,
	PostageDepth: 16,
	PostageLabel: "test-label",
	Seed: 0,
	}
	}

Maybe to put PostageLabel default value to manifest-check, or something like that to distingush it from other batch labels.
Maybe this would be nice to do for every check that uses PostageLabel option, to have the check name in it? What do you think @istae ?

Also, MaxPathnameLength shouldn't be less then 2, or it will panic.

[istae]

One improvement regarding Options:

beekeeper/pkg/check/manifest/manifest.go

Lines 36 to 46 in 4d02a31

	func NewDefaultOptions() Options {
	return Options{
	FilesInCollection: 10,
	GasPrice: "",
	MaxPathnameLength: 64,
	PostageAmount: 1,
	PostageDepth: 16,
	PostageLabel: "test-label",
	Seed: 0,
	}
	}

Maybe to put PostageLabel default value to manifest-check, or something like that to distingush it from other batch labels.
Maybe this would be nice to do for every check that uses PostageLabel option, to have the check name in it? What do you think @istae ?
Also, MaxPathnameLength shouldn't be less then 2, or it will panic.

The label can be used strategically in general, because the GetCreateMutableBatch function matches the label, and it can reuse the same batch created before to save on cost and time (by not waiting for a new batch to be created which can take few minutes).

[gacevicljubisa]

The label can be used strategically in general, because the GetCreateMutableBatch function matches the label, and it can reuse the same batch created before to save on cost and time (by not waiting for a new batch to be created which can take few minutes).

@istae Would this mean that we can resuse the same batch created with specific check? Or is it ok to use test-label for all of the checks?
What is your suggestion, to leave the same label for all of the checks?

[istae]

The label can be used strategically in general, because the GetCreateMutableBatch function matches the label, and it can reuse the same batch created before to save on cost and time (by not waiting for a new batch to be created which can take few minutes).

@istae Would this mean that we can resuse the same batch created with specific check? Or is it ok to use test-label for all of the checks? What is your suggestion, to leave the same label for all of the checks?

If the tests run one by one then it's more optimal to simply reuse the same batch.

[janos]

I agree with existing comments and would like to add that the check will be more responsive in certain situations if time.Sleep is replaced with select with cases on ctx.Done and time.After.