-
Notifications
You must be signed in to change notification settings - Fork 519
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Evidently server side integration and workshop (#353)
* Evidently resources and deps * Evidently initial integration * Fixes and metrics * Evidently workshop WIP * IAM permissions * Cold user feature and experiment cleanup on delete * Skip generator flag/optimization * Explicit timestamp for experiment outcomes * Default value for timestamp * Update architecture diagram * Fix for None timestamp * All timestamp to be passed in * Evidently workshop * Local dev doc improvement * Evidently IAM changes * Evidently workshop improvements * Evidently workshop improvements * Built-in experiment compatibility * Add link to Evidently workshop * Add cold user feature to list of features * Add troubleshooting/FAQ section * Remove deleted entry point
- Loading branch information
1 parent
34965ba
commit f32e193
Showing
46 changed files
with
1,781 additions
and
277 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,217 @@ | ||
| --- | ||
| AWSTemplateFormatVersion: 2010-09-09 | ||
|
|
||
| Description: > | ||
| This template deploys the Retail Demo Store Evidently project and features. | ||
| Parameters: | ||
| Uid: | ||
| Type: String | ||
|
|
||
| Resources: | ||
| EvidentlyProject: | ||
| Type: AWS::Evidently::Project | ||
| Properties: | ||
| Name: !Ref Uid | ||
| Description: Retail Demo Store features and experiments | ||
|
|
||
| FeatureHomeProductRecs: | ||
| Type: AWS::Evidently::Feature | ||
| Properties: | ||
| DefaultVariation: Personalize-UserPersonalization | ||
| Description: Home page product recommendations | ||
| EvaluationStrategy: ALL_RULES | ||
| Name: home_product_recs | ||
| Project: !Ref EvidentlyProject | ||
| Variations: | ||
| - VariationName: FeaturedProducts | ||
| StringValue: '{"type":"product"}' | ||
| - VariationName: Personalize-UserPersonalization | ||
| StringValue: !Sub '{"type":"personalize-recommendations","arn":"arn:${AWS::Partition}:personalize:${AWS::Region}:${AWS::AccountId}:recommender/retaildemostore-recommended-for-you"}' | ||
|
|
||
| FeatureHomeProductRecsCold: | ||
| Type: AWS::Evidently::Feature | ||
| Properties: | ||
| DefaultVariation: Personalize-PopularItems | ||
| Description: Home page product recommendations for new/cold users | ||
| EvaluationStrategy: ALL_RULES | ||
| Name: home_product_recs_cold | ||
| Project: !Ref EvidentlyProject | ||
| Variations: | ||
| - VariationName: FeaturedProducts | ||
| StringValue: '{"type":"product"}' | ||
| - VariationName: Personalize-PopularItems | ||
| StringValue: !Sub '{"type":"personalize-recommendations","arn":"arn:${AWS::Partition}:personalize:${AWS::Region}:${AWS::AccountId}:recommender/retaildemostore-popular-items"}' | ||
|
|
||
| FeatureHomeFeaturedRerank: | ||
| Type: AWS::Evidently::Feature | ||
| Properties: | ||
| DefaultVariation: Personalize-PersonalizedRanking | ||
| Description: Home page featured products carousel | ||
| EvaluationStrategy: ALL_RULES | ||
| Name: home_featured_rerank | ||
| Project: !Ref EvidentlyProject | ||
| Variations: | ||
| - VariationName: RankingNoOp | ||
| StringValue: '{"type":"ranking-no-op"}' | ||
| - VariationName: Personalize-PersonalizedRanking | ||
| StringValue: !Sub '{"type":"personalize-ranking","arn":"arn:${AWS::Partition}:personalize:${AWS::Region}:${AWS::AccountId}:campaign/retaildemostore-personalized-ranking"}' | ||
|
|
||
| FeatureProductDetailRelated: | ||
| Type: AWS::Evidently::Feature | ||
| Properties: | ||
| DefaultVariation: Personalize-Similar-Items | ||
| Description: Product detail related products carousel | ||
| EvaluationStrategy: ALL_RULES | ||
| Name: product_detail_related | ||
| Project: !Ref EvidentlyProject | ||
| Variations: | ||
| - VariationName: ProductsInSameCategory | ||
| StringValue: '{"type":"product"}' | ||
| - VariationName: OpenSearchMoreLikeThis | ||
| StringValue: '{"type":"similar"}' | ||
| - VariationName: Personalize-Similar-Items | ||
| StringValue: !Sub '{"type":"personalize-recommendations","arn":"arn:${AWS::Partition}:personalize:${AWS::Region}:${AWS::AccountId}:campaign/retaildemostore-related-items"}' | ||
|
|
||
| FeatureSearchResults: | ||
| Type: AWS::Evidently::Feature | ||
| Properties: | ||
| DefaultVariation: Personalize-PersonalizedRanking | ||
| Description: Search control auto-complete | ||
| EvaluationStrategy: ALL_RULES | ||
| Name: search_results | ||
| Project: !Ref EvidentlyProject | ||
| Variations: | ||
| - VariationName: RankingNoOp | ||
| StringValue: '{"type":"ranking-no-op"}' | ||
| - VariationName: Personalize-PersonalizedRanking | ||
| StringValue: !Sub '{"type":"personalize-ranking","arn":"arn:${AWS::Partition}:personalize:${AWS::Region}:${AWS::AccountId}:campaign/retaildemostore-personalized-ranking"}' | ||
|
|
||
| FeatureLiveStreamProductRecs: | ||
| Type: AWS::Evidently::Feature | ||
| Properties: | ||
| DefaultVariation: Personalize-PersonalizedRanking | ||
| Description: Live stream product recommendations | ||
| EvaluationStrategy: ALL_RULES | ||
| Name: live_stream_prod_recommendation | ||
| Project: !Ref EvidentlyProject | ||
| Variations: | ||
| - VariationName: RankingNoOp | ||
| StringValue: '{"type":"ranking-no-op"}' | ||
| - VariationName: Personalize-PersonalizedRanking | ||
| StringValue: !Sub '{"type":"personalize-ranking","arn":"arn:${AWS::Partition}:personalize:${AWS::Region}:${AWS::AccountId}:campaign/retaildemostore-personalized-ranking"}' | ||
|
|
||
| FeatureLiveStreamProductDiscounts: | ||
| Type: AWS::Evidently::Feature | ||
| Properties: | ||
| DefaultVariation: Personalize-PersonalizedRanking | ||
| Description: Live stream product discounts | ||
| EvaluationStrategy: ALL_RULES | ||
| Name: live_stream_prod_discounts | ||
| Project: !Ref EvidentlyProject | ||
| Variations: | ||
| - VariationName: RankingNoOp | ||
| StringValue: '{"type":"ranking-no-op"}' | ||
| - VariationName: Personalize-PersonalizedRanking | ||
| StringValue: !Sub '{"type":"personalize-ranking","arn":"arn:${AWS::Partition}:personalize:${AWS::Region}:${AWS::AccountId}:campaign/retaildemostore-personalized-ranking"}' | ||
|
|
||
| EvidentlyCleanupLambdaFunction: | ||
| Type: AWS::Lambda::Function | ||
| Properties: | ||
| Description: 'Retail Demo Store deployment utility function that cancels and deletes experiments to allow project to be fully deleted' | ||
| Code: | ||
| ZipFile: | | ||
| import boto3 | ||
| import cfnresponse | ||
| def handler(event, context): | ||
| print(event) | ||
| response_data = {} | ||
| response_status = cfnresponse.SUCCESS | ||
| try: | ||
| project_name = event['ResourceProperties']['EvidentlyProjectName'] | ||
| if event['RequestType'] == 'Delete': | ||
| evidently = boto3.client('evidently') | ||
| experiments_stopped = 0 | ||
| experiments_deleted = 0 | ||
| paginator = evidently.get_paginator('list_experiments') | ||
| for paginate_result in paginator.paginate(project = project_name): | ||
| for experiment in paginate_result['experiments']: | ||
| if experiment['status'] == 'RUNNING': | ||
| print(f"Experiment {experiment['name']} is still running; cancelling") | ||
| evidently.stop_experiment(desiredState='CANCELLED', | ||
| experiment=experiment['name'], | ||
| project=project_name | ||
| ) | ||
| experiments_stopped += 1 | ||
| print(f"Deleting experiment {experiment['name']}") | ||
| evidently.delete_experiment(experiment=experiment['name'], | ||
| project=project_name | ||
| ) | ||
| experiments_deleted += 1 | ||
| message = f"Stopped {experiments_stopped} experiments and deleted {experiments_deleted} experiments" | ||
| response_data['Message'] = message | ||
| else: | ||
| response_data['Message'] = "Nothing to do for this request type" | ||
| except Exception as e: | ||
| print("Error: " + str(e)) | ||
| response_status = cfnresponse.FAILED | ||
| response_data['Message'] = "Resource {} failed: {}".format(event['RequestType'], e) | ||
| cfnresponse.send(event, context, response_status, response_data) | ||
| Handler: index.handler | ||
| Runtime: python3.9 | ||
| Timeout: 120 | ||
| Role: !GetAtt EvidentlyCleanupLambdaExecutionRole.Arn | ||
|
|
||
| CustomEvidentlyCleanup: | ||
| Type: Custom::EvidentlyCleanup | ||
| DependsOn: [ FeatureHomeProductRecs, FeatureHomeProductRecsCold, FeatureHomeFeaturedRerank, FeatureProductDetailRelated, FeatureSearchResults, FeatureLiveStreamProductRecs, FeatureLiveStreamProductDiscounts ] | ||
| Properties: | ||
| ServiceToken: !GetAtt EvidentlyCleanupLambdaFunction.Arn | ||
| EvidentlyProjectName: !Ref Uid | ||
|
|
||
| EvidentlyCleanupLambdaExecutionRole: | ||
| Type: AWS::IAM::Role | ||
| Properties: | ||
| AssumeRolePolicyDocument: | ||
| Version: 2012-10-17 | ||
| Statement: | ||
| - Effect: Allow | ||
| Principal: | ||
| Service: lambda.amazonaws.com | ||
| Action: | ||
| - 'sts:AssumeRole' | ||
| Policies: | ||
| - PolicyName: LoggingPolicy | ||
| PolicyDocument: | ||
| Version: 2012-10-17 | ||
| Statement: | ||
| - Effect: Allow | ||
| Action: | ||
| - logs:CreateLogGroup | ||
| - logs:CreateLogStream | ||
| - logs:PutLogEvents | ||
| Resource: '*' | ||
| - PolicyName: Evidently | ||
| PolicyDocument: | ||
| Version: 2012-10-17 | ||
| Statement: | ||
| - Effect: Allow | ||
| Action: | ||
| - evidently:* | ||
| Resource: | ||
| - '*' | ||
|
|
||
| Outputs: | ||
| EvidentlyProjectName: | ||
| Description: Evidently project name | ||
| Value: !Ref Uid |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.