Enhancing Your Tests with AI Prompts

Introduction

This tutorial will guide you through using the AI Prompt block in TV Labs to create more intelligent and adaptable test workflows for smart TV apps.

What to expect:

• ⏱️ Duration: 5-10 minutes
• 🎯 Audience: Users familiar with basic TV Labs automation
• 🧠 Outcome: Understand how to leverage AI for smarter testing
• 🖥️ Hands-on: Create and run a workflow using AI prompts

Understanding AI Prompts

The AI Prompt block is a powerful and flexible tool that uses advanced image recognition and natural language processing to analyze the screen content. This feature goes beyond simple text or color matching, allowing for more nuanced and context-aware testing.

Adding an AI Prompt to Your Workflow

1. In your workflow, add an AI Prompt block
2. Enter the prompt: Describe what you see on the screen.

Figure 1: Adding an AI Prompt block

Running a Workflow with AI Prompts

Follow the same steps as in the basic tutorial to run your workflow. When you reach the AI Prompt block, you'll see a detailed description of the screen content.

Figure 2: Completed workflow execution

Example AI Prompt Result

The AI Prompt block should provide a detailed description of the screen content, similar to this:

AI output

The screen displays a streaming service interface, likely from ITVX, showcasing the TV show "Fringe."

The main section highlights "Fringe" with an image of three characters and a brief description: "An FBI agent, a multi-scientist - the mind-bending crime epic."

There is a "Watch Now" button for immediate viewing.

Below the main feature, there is a section titled "Our Top Picks," displaying several other shows or movies with their respective cover images.

The titles visible include:

1. "Fringe" (repeated)
2. "Queen of Oz"
3. "The Only Way is Essex"
4. "The Tower"
5. An animated show featuring colorful characters (possibly "Trolls")

The left sidebar includes navigation options and a sponsorship note at the bottom.

Advanced AI Prompt Usage

One of the most powerful aspects of the AI Prompt is its ability to pass information to the next blocks in your workflow. This allows for dynamic, context-aware testing. Here are some examples:

1. Dynamic Navigation:

AI Prompt:
  "Is there a 'Continue Watching' section visible?"
Next Block (Conditional):
  If AI response contains "yes", navigate to that section.

2. Content Based Assertions:

AI Prompt:
  "Are there any error messages or warning signs on the screen?"
Next Block (Conditional):
  If AI detects errors, capture a screenshot and log details.

Detect on Screen Examples

To implement some of the advanced usage examples, the AI Detect on Screen block can be used. This block will return true or false depending on whether the element is detected on the current screen, allowing for assertions or conditional logic to execute.

Adding AI Detect on Screen to Your Workflow

1. In your workflow, add an AI Detect on Screen block
2. Enter the type of element you want to detect in the Element field, and optionally any text that relates to the element in the Text field.

Dynamic Navigation Example

1. Add an AI Detect on Screen block to your workflow
2. Enter button for the element, and Sign in for the text
3. Add a Conditional block to check the result, with the value true
4. Add a Press Keys block to navigate to the sign-in page

Figure 3: Using AI Detect on Screen to detect a Sign In button

Dynamic Content-Based Assertion Example

1. Add an AI Detect on Screen block to your workflow
2. Enter text for the element, and Plans start at just $X/month for the text
3. Add an Assert block to check the result, with the value true

Figure 4: Using AI Detect on Screen to detect dynamic content

Best Practices for AI Prompts

• Craft clear, specific questions that yield actionable information for your test workflow
• Use AI Prompts to handle scenarios that are difficult to test with traditional methods
• Combine AI Prompts with other blocks to create adaptive workflows
• Experiment with different prompts to see how they can enhance your testing strategy

Conclusion

By leveraging the AI Prompt block and AI Detect on Screen block, you can create more intelligent and adaptable test workflows. The AI's ability to understand and describe complex screen states allows your tests to make decisions and perform actions based on real-time content analysis, much like a human tester would.

Remember, the key to effective use of AI Prompts is crafting clear, specific questions that yield actionable information for your test workflow. Experiment with different prompts to see how they can enhance your testing strategy.

Happy testing with AI!