Lambda layers shouldn't be sorted at synthesis

[ctgardner]

Lambda extracts the layers in the order (low to high) listed by the function. Lambda merges folders with the same name, so if the same file appears in multiple layers, the function uses the version in the last extracted layer.

Source: https://docs.aws.amazon.com/lambda/latest/dg/invocation-layers.html#invocation-layers-accessing

Lambda layers are supposed to be extracted (and merged) in the order they're registered. This is the expected and current behaviour exhibited by AWS CDK (and CloudFormation), by default.

However, when @aws-cdk/aws-lambda:recognizeLayerVersion is enabled, the layers are sorted at synthesis. Potentially, this reorders layers and causes breaking changes to Lambda functions which rely upon the documented extract-and-merge behaviour.

aws-cdk/packages/@aws-cdk/aws-lambda/lib/function.ts

Lines 1058 to 1060 in f19ecef

	if (FeatureFlags.of(this).isEnabled(LAMBDA_RECOGNIZE_LAYER_VERSION)) {
	this._layers.sort();
	}

According to a test included with this change in #20150 (f19ecef), a reason for sorting layers is to ensure the same hash is calculated for Lambda functions which register the same layers in a different order, but are otherwise identical.

aws-cdk/packages/@aws-cdk/aws-lambda/test/function-hash.test.ts

Lines 210 to 230 in f19ecef

	test('with feature flag, we sort layers so order is consistent', () => {
	const app = new App({ context: { [LAMBDA_RECOGNIZE_LAYER_VERSION]: true } });
	const stack2 = new Stack(app, 'stack2');
	const fn1 = new lambda.Function(stack2, 'MyFunction', {
	runtime: lambda.Runtime.NODEJS_12_X,
	code: lambda.Code.fromInline('foo'),
	handler: 'index.handler',
	layers: [layer1, layer2],
	});
	const stack3 = new Stack(app, 'stack3');
	const fn2 = new lambda.Function(stack3, 'MyFunction', {
	runtime: lambda.Runtime.NODEJS_12_X,
	code: lambda.Code.fromInline('foo'),
	handler: 'index.handler',
	layers: [layer2, layer1],
	});
	expect(calculateFunctionHash(fn1)).toEqual(calculateFunctionHash(fn2));
	});

But because layers can overwrite each other, the order in which they're extracted is crucial and Lambda functions which register the same layers in a different order should always have a different hash.

Regardless of the previous point, why is it necessary to mutate the layers array to calculate the Lambda function hash? A sorted copy can be used to calculate the same hash. This would cause the layers to maintain the original ordering and be extracted in the expected order.

In summary, Lambda layers can overwrite each other when extracted, so the order in which they're extracted is crucial. Lambda functions which list the same layers in a different order aren't necessarily identical and shouldn't have the same hash. Per the documentation, layers should be extracted in the order listed by the function. Extracting Lambda layers in any other order is unexpected and is a potentially breaking change.