Implementing synthetic monitoring using Amazon Nova Act

來源原標題

Implementing synthetic monitoring using Amazon Nova Act

作者

Sarath Krishnan

原始標籤

Advanced (300) · Amazon Nova · Technical How-to

原始連結

https://aws.amazon.com/blogs/machine-learning/implementing-synthetic-monitoring-using-amazon-nova-act/

摘要原文

Synthetic monitoring emulates real user journeys through automated transactions. Rather than waiting for customers to encounter problems, teams continuously validate critical workflows (logins, purchases, form submissions) on a scheduled basis. With this approach, you detect problems faster when performance degrades or UI interactions break. For customer-facing businesses, especially in ecommerce, synthetic monitoring safeguards interactions that matter most. Organizations validate that product search works correctly, pricing and inventory display as expected, cart operations succeed, and checkout confirmations complete without error. Continuous testing identifies problems before customers encounter them. The same approach applies across industries. Financial services organizations make sure account access and transaction flows remain functional. Travel and hospitality providers monitor booking and reservation journeys. Software as a service (SaaS) companies validate signup and subscription upgrade paths. Healthcare organizations monitor appointment scheduling portals where availability and transaction health are essential. This post walks through an agent-driven approach to synthetic monitoring using Amazon Nova Act and Amazon Bedrock AgentCore . Together, with these services you can move beyond brittle UI scripts toward resilient, managed journey validation. The post describes the architecture and patterns for agent-driven synthetic monitoring, with a sample repository containing the complete implementation. Most organizations monitor infrastructure health through metrics, logs, traces, and API-level checks. However, high-impact failures often occur at the user journey level, where backend signals may not immediately reveal the problem. Common examples: a frontend deployment breaks a checkout button, a third-party login page changes unexpectedly, or a UI regression makes critical elements unresponsive. Traditional browser automation frameworks like Selenium and Playwright rely on explicit Document Object Model (DOM) locators and selectors, which creates brittleness. Even small UI changes can break tests and require constant maintenance. Teams invest significant ongoing effort managing selectors, handling timing-related instability, and updating scripts as applications evolve, often spending more time on maintenance than on building new automations. Amazon Nova Act uses a multimodal large language model (LLM) that processes UI screenshots rather than DOM selectors, making it significantly more resilient to UI changes. Because the model reasons from what it sees on screen (not from CSS classes or element IDs), it adapts when styling changes without requiring script updates. In early enterprise customer use cases, Amazon Nova Act has demonstrated over 90% accuracy on browser workflows. This is a meaningful improvement over selector-based approaches that fail immediately on any element change. Teams should test on their own sites and design monitors with retry logic for cases where adaptation does not succeed. Enterprise monitoring platforms compound the maintenance challenge by pricing browser checks as premium add-ons. This forces teams to choose between monitoring breadth and budget, all separate from the real goal of validating critical customer journeys. This solution combines the following AWS services, each handling a distinct responsibility: The architecture follows this flow: The following diagram illustrates the architecture. Figure 1: Synthetic monitoring architecture with Amazon Nova Act and Amazon Bedrock AgentCore This architecture provides fully managed synthetic monitoring. Teams avoid maintaining brittle scripts or browser infrastructure. The following sections walk through setup requirements, journey definition, agent deployment, and scheduling infrastructure. Before implementing, confirm the following: Implementation begins by defining the most important customer journeys to validate. For ecommerce applications, these typically include homepage load, product search, product detail navigation, add-to-cart validation, and checkout readiness. Rather than focusing only on workflow completion, validate outcomes explicitly. The monitor should confirm that search results appear correctly, the cart contains items, and no error banners display. Explicit outcome validation reduces false positives from incomplete checks and false negatives from overly permissive assertions. The agent groups natural-language actions into logical journey steps and uses assertions at key checkpoints. Actions use act() to drive the UI through natural language. Assertions use act_get() with a boolean schema to validate outcomes. When a journey fails, the agent publishes the journey type, target URL, duration, completed steps, and failed steps. Detailed exception information remains in the runtime logs. The code uses the Nova Act SDK directly. When deployed to AgentCore Runtime, the execution environment manages browser session lifecycle through the Browser tool. In testing, a typical six-step journey completes in 2 to 4 minutes depending on page load times. Refer to the Nova Act User Guide for current method signatures and SDK version. The sample repository includes the complete agent definition with error handling and single-attempt step execution. Before the Amazon EventBridge schedule can invoke the agent, deploy it to AgentCore Runtime using the Nova Act CLI. The act workflow commands package the agent code, push the container image to Amazon ECR, and provision the runtime. Once deployed, the endpoint Amazon Resource Name (ARN) remains constant. Updates create new runtime versions without requiring you to update your Scheduler target. The act workflow show command returns the AgentCore Runtime ARN that you reference as the Scheduler target. The sample repository’s deploy.py script wraps these commands with prerequisite checks (Docker, AWS credentials), creates the SNS alert topic, and wires the Amazon EventBridge schedule, so a single python deploy.py performs an end-to-end deployment. (AgentCore also provides its own agentcore CLI for agent development. This sample standardizes on the Nova Act act workflow commands.) For a production-grade, repeatable deployment, the sample includes a CDK stack as a standalone alternative to deploy.py . It provisions the Amazon EventBridge schedule with least-privilege IAM permission, the SNS alert topic, and an SQS dead-letter queue that captures failed InvokeAgentRuntime calls from the Scheduler. The stack also creates two Amazon CloudWatch alarms: one on dead-letter-queue depth (failed Scheduler invocations), and one on missing scheduled runs (using the AWS/Scheduler InvocationAttemptCount metric, treating missing data as breaching). Note that these alarms detect infrastructure failures, such as a delivery failure to the agent or the schedule not firing. A functionally broken customer journey that still returns at the HTTP level surfaces through the agent’s SNS publish, not through these CloudWatch alarms. Focus synthetic monitors on high-value customer journeys rather than exhaustive page coverage. Over-monitoring creates noise that desensitizes teams to real failures. Start with 3-5 critical workflows (login, checkout, account access) and expand based on actual failure patterns and business impact. Monitor journeys that directly affect revenue or customer trust, not every possible user path. Similarly, avoid over-asserting on every page element. Validate meaningful outcomes (whether search results appear, whether the cart contains items) rather than checking every DOM element on the page. Overly granular assertions increase false positives without improving failure detection. The sample agent code uses single-attempt execution per step, which minimizes Browser session cost.