Let’s create a new failure and generate an insight for that. We will modify the Lambda functions’s resource-based policy by removing the permissions for API Gateway to access this function.
Follow this link to open the Lambda function
On the Permissions tab, access the Resource-based policy section.
Save a copy of the policy offline as a backup before making any changes.
Note down the
Sid values for the
AWS:SourceArn that ends with
aws lambda remove-permission --function-name ScanFunctionMonitorOper \ --statement-id <Sid-value-ending-with-prod/*/>
aws lambda remove-permission --function-name ScanFunctionMonitorOper \ --statement-id <Sid-value-ending-with-prod/*/*>
You should see several 5XX errors, as in the following screenshot:
After less than 8 minutes, you should see a new ongoing reactive insight on the DevOps Guru dashboard.
Let’s take a closer look at the insight. The following screenshot shows the anomalous metric 5XXError Average of API Gateway and its duration (this insight shows as closed because I had already restored permissions):
If you have configured to enable creating OpsItem in Systems Manager, you would see the link to OpsItem ID created in the insight, as shown above. This is an optional configuration, which will enable you to track the insights in the form of open tickets (OpsItems) in Systems Manager OpsCenter.
The recommendations provide guidance based upon the related events and anomalous metrics. After the insight has been generated, reviewed, and verified, restore the permissions by running the following command:
aws lambda add-permission --function-name ScanFunctionMonitorOper \ --statement-id APIGatewayProdPerm --action lambda:InvokeFunction \ --principal apigateway.amazonaws.com
If needed, you can insert the condition to point to the API Gateway ARN to allow only specific API Gateways to access the Lambda function.