feat(stepfunctions): add support JSONata and variables

[WinterYukky]

Issue #32262

Closes #32262.

Reason for this change

For to use JSONata and variables of Step Function feature on AWS CDK. JSONata is new query language of Step Function, It is simple and powerful language. JSONata and variables is recommend for new state machine.

Description of changes

JSONata support

Add jsonPath() and jsonata() factory methods to state constructs. For example,

// For JSONPath
sfn.Pass.jsonPath(stack, "JSONPathPass", {
    outputPath: sfn.JsonPath.stringAt('$foo'),
});

// For JSONata
sfn.Pass.jsonata(stack, "JSONataPass", {
    outputs: {
        count: "{% $states.input.count + 1 %}"
    },
});

One option would be to simply add JSONata-specific properties to the Props of the existing State construct, but in this case, the JSONata-specific properties will be displayed to the JSONPath user. Conversely, it was thought that the development experience would deteriorate if JSONata users were shown JSONPath-specific properties. As a countermeasure, we decomposed the existing Props into JSONPath-specific properties and created jsonPath() and jsonata() factory methods to separate type suggestions for JSONPath users and JSONata users.

The existing initialization method, the constructor, is backward compatible because it accepts JSONPath and JSONata properties. However, to use this interface directly is a lot of noise. This noise is a source of confusion for SFn beginners, and I thought it was necessary to solve this problem.

Therefore, we use the factory methods for each query language.
※ The output property is used in the image example, but it is actually outputs. Check out this comment for the reasons for this decision.

jsonPath() does not have a JSONata-specific property outputs.

jsonata() does not have JSONPath-specific properties such as xxxPath.

Variables

Add assign to state constructs. assign can be used from either JSONata or JSONPath.

For example,

sfn.Pass.jsonata(this, 'AssignExamplePass', {
  assign: {
     count: "{% $states.input.count + 1 %}"
  },
});

Description of how you validated changes

Added unit test. Integration tests are not yet.

Checklist

• My code adheres to the CONTRIBUTING GUIDE and DESIGN GUIDELINES

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

[aws-cdk-automation]

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed add Clarification Request to a comment.

[melalawi]

Gave it a first pass-through. Good stuff! 😸

One thing we enforce is that a state machine whose QueryLanguage is set to JSONata will not accept any states with JSONPath as their QueryLanguage. Can this also be enforced here?

[WinterYukky]

@melalawi Thank you for your reviewing😀
Yeah, I implementing to throw error when top level QueryLanguage is JSONata but state level QueryLanguage is JSONPath.

aws-cdk/packages/aws-cdk-lib/aws-stepfunctions/lib/states/state.ts

Lines 566 to 576 in 08d810d

	protected renderQueryLanguage(topLevelQueryLanguage?: QueryLanguage): any {
	topLevelQueryLanguage = topLevelQueryLanguage ?? QueryLanguage.JSONPATH;
	if (topLevelQueryLanguage === QueryLanguage.JSONATA && this.queryLanguage === QueryLanguage.JSONPATH) {
	throw new Error(`'queryLanguage' can not be 'JSONPath' if set to 'JSONata' for whole state machine ${this.node.path}`);
	}
	const queryLanguage = topLevelQueryLanguage === QueryLanguage.JSONPATH && this.queryLanguage === QueryLanguage.JSONATA
	? QueryLanguage.JSONATA : undefined;
	return {
	QueryLanguage: queryLanguage,
	};
	}

[GavinZZ]

Thank you for this huuge effort to enable JSONata with State Machine Chainable constructs. Appreciate the effort.

I'm still going through some of the changes, but overall it looks good and I don't see any breaking changes. I left some minor feedback on the parts that I've reviewed.

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

Pull request has been modified.

[GavinZZ]

Changes look good but the amount of integ tests to cover the JSONata usage is a bit worrying to me. I would like to request add some more integ tests and happy to approve.

[GavinZZ]

Is this intended change?

[WinterYukky]

Yes, of course. Until now, JSONPath only supported paths starting with $. or $$. (e.g. $.foo.bar). If any other path is specified, a validation error is thrown.

Variables, on the other hand, start with $ without a dot (e.g. $foo.bar). Without this change, you will be left unable to specify variables in JSONPath.

[GavinZZ]

I don't think this integ test is enough as the changes are huge. Can you please add more tests that integrate with different step function tasks (perhaps not every step function task is needed but we should definitely add a few more integ tests that test the deployment).

[gracelu0]

Thanks for all your effort on this, I just have a few suggestions to improve the README

[gracelu0]

did we mean to delete this example? The current flow doesn't quite make sense given the previous sentence: "Defining a workflow looks like this (for the Step Functions Job Poller
example):"

Pull request has been modified.

[aws-cdk-automation]

AWS CodeBuild CI Report

• CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
• Commit ID: 78400a3
• Result: SUCCEEDED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[javamate]

@WinterYukky Thank you for working on this! I have been searching for hours trying to find this support to no avail. Finally thought to check the GitHub issues/PRs and found this! It looks like it is close.