Console

Settings

Sign out

Notifications

Alexa

亚马逊应用商店

Ring

AWS

文档

Support

My Cases

Console

Alexa+ Smart Home Docs

Get started

Connect with Matter

Alexa Connect Kit

Add-ons

Certify

Resources

感谢您的访问。此页面目前仅提供英语版本。我们正在开发中文版本。谢谢您的理解。

Use the Alexa Automated Testing Service (Beta)

Smart home skills are now called smart home add-ons. No action is needed, and your existing integration will continue to operate normally.

Important: Amazon offers this feature as beta. The feature might change as Amazon receives feedback and iterates on the functionality.

Use the Alexa Automated Testing Service (ATS) in the Alexa developer console to test device control for your Alexa-connected smart home devices. ATS supports both smart home add-ons and Matter protocol implementations. The testing service lets you create test plans by using natural language instead of complex JSON definitions. Choose from pre-built templates or build custom tests that meet your specific requirements.

For details about other test tools, see Test and Debug Your Smart Home Add-on.

How it works

ATS provides pre-defined templates, or you can create custom test plans. To build a test, you describe what you want to test as natural-language utterances and ATS creates a test plan. For example, to test a light bulb with color control, you might add the utterance, "Alexa, set the ${deviceName} to blue."

When you run the test plan, ATS sends voice commands to your device through Alexa and validates the device response and state change reports. After the test completes, you receive a detailed test report with pass and fail results and error descriptions for any failures.

ATS provides pre-defined templates for the following Alexa interfaces:

You can use the testing service throughout the development lifecycle to perform the following tasks:

Test your smart home add-on or Matter integration during development.
Debug device integration issues.
Validate device behavior before you submit your product for Works with Alexa (WWA) certification.
Create regression tests for your add-on-enabled devices.

Note: You can only see information for the Amazon developer account that you use to sign in to the Alexa developer console. ATS doesn't show information from other developer accounts.

Prerequisites

Before you start, make sure that you have the following items:

An Amazon developer account.
An Alexa-enabled device, such as an Amazon Echo. Make sure that you sign in to the Echo device with the same credentials as your Amazon developer account.
For add-on-enabled devices, you must meet the following prerequisites:
- Your smart home add-on must be available in the developer console and you must configure the minimum configuration for a smart home add-on.
- Deploy your add-on code in an AWS Lambda function to provide communication between Alexa and your device.
- Enable your add-on in the Alexa app. Make sure that you sign in to the Alexa app with the same credentials as your Alexa developer account.
- Configure account linking and successfully link your device account with your Alexa account.
For Matter-enabled devices, you must meet the following prerequisites:
- Commission your device on the same Alexa fabric as your Echo device.
- Have your device local identifier available.
At least one test device connected to your developer account. Make sure your device is discoverable and responds to Alexa commands.
Register the Echo device and the device under test to the same account. Verify both devices in the Alexa app.

Steps to run a pre-built test

Follow these steps to create and run a test plan that uses a pre-built template. The testing service displays the Echo devices and devices under test registered to your Alexa app.

Important: During testing, make sure your Echo device is powered on, connected to the internet, not muted, and responding to voice commands. The Echo must be in the same room as or within range of your device under test.

Create a test plan from a template.
Run the test plan.
View the test results.

Step 1: Create a test plan from a template

To run automated tests, you first define a test plan. If your device supports one of the capabilities defined by a pre-built template, you can create a test plan that uses the template. If your device supports additional capabilities or test scenarios, you can customize the test plan. For more details, follow the instructions in Customize a pre-built test plan. If your device has capabilities not supported by a template, see Create a custom test plan.

Tip: To understand the test plan structure, use a pre-built template before you create custom test plans.

To create a test plan from a template in the Alexa developer console

Sign in to the Alexa developer console.
On the Skills tab, under SKILL NAME, scroll to find your smart home add-on.
Under ACTIONS, expand the dropdown list in your add-on's row, and then select Test.
On the banner that says, Try the Alexa Automated Testing Service (Beta) …., choose the ATS tool.
For Login Required, choose Log in with Amazon, and then enter your Amazon developer credentials to sign in to ATS.
For Click 'Allow' to Sign-In to Open Automation Kit, choose Allow.
Under Create a New Test Plan, for Select Template, select a template that matches a capability of your device, such as Power Controller, and then choose Create from Template.
On Create Test Plan, under Test Plan Name, enter a descriptive name for your test plan.
Valid values: 1–50 characters including letters, numbers, underscores, and hyphens.
To review the pre-built tests in the plan, expand each test case.
You can also see a preview of the test plan in the JSON Preview panel.
To save the test plan, choose Create Test Plan.

Step 2: Run the test plan

Select and run your test plan on ATS. You can run test plans based on pre-built or custom test cases.

Note: Wait for the test plan to complete before you run another test plan on the same device.

To run a test in the Alexa developer console

Under Run a Test, choose the following items:
1. For Test Plan Name, select the test plan that you created.
2. For Device Name, select the name of your device under test.
3. For Alexa Device, select the name of your Alexa-enabled device.
To run the test, choose Run Test.
Wait for the tests to complete.
If an error appears at the top of the page, correct the error, and then rerun the test plan.

Step 3: View test results

After the tests in the selected test plan complete, the testing service generates a test report that shows the overall test results, individual test case results, and error messages for any failures. The report also includes detailed step-by-step execution and device state information.

When a test case fails, use the error messages to help debug the issues. For more help, see Alexa Automated Testing Service Troubleshooting Guide.

Note: ATS stores test plans and test reports for 30 days and then automatically deletes them. Save your test plan and download test reports before they expire.

To view the test report for a specific test run

In the test run history table, navigate to the most recent entry of your test plan with the status COMPLETED.
Under ACTIONS in your test plan's row, choose View Report.
Under Test Report, view the test run ID, test plan name, and overall test status.
Under Device Information, view the device name.
Under Test Summary, view the total number of test cases run, number of test cases that succeeded, and the number that failed.
Under Test Cases, you can view the status of each test case: Success or Failed. If the test case failed, the report includes an error message.
To expand the test case results, choose the arrow icon ().
You can view the results of each step in the test case.
To return to the Alexa Automated Testing Service page, choose Back.
For failed test cases, update your add-on or device, and then rerun the test plan.

Test status values

During your test run, you might see one of the following statuses in the test run history table.

Status	Description
IN PROGRESS	The test cases in the specified test plan are running.
COMPLETED	Test run finished. The results are ready to view.
TERMINATED	An internal error occurred during test execution.

Download test reports

The testing service deletes test reports after 30 days. Follow these steps to download your test report.

To download your test report

In the test run history table, navigate to the most recent entry of your test plan with the status COMPLETED.
Under ACTIONS in your test plan's row, choose Download Report.
The JSON-formatted report downloads to your default download location.

View test plans

Before you run a test, you can view the test plan.

To view your test plan

Under Run a Test, for Test Plan Name, select the test plan that you want to review.
To view the test plan, choose Preview Test Plan.

Save your test plan

You might want to include your test plan in your CI/CD pipeline. Follow these steps to download your test plan. If your test plan expires, you can create a new test plan based on the saved version.

To save your test plan

In the test run history table, navigate to the most recent entry of your test plan with the status COMPLETED.
Under ACTIONS in your test plan's row, choose View Test Plan.
The JSON-formatted test plan opens in a separate browser tab or window.
To save your test plan to your PC or Mac, choose Save.

Was this page helpful?

Provide feedback

Last updated: May 01, 2026