AI

AI Detect App Language

Restrictions: "none"

No docs available

Inputs

Name	Type	Description	Required	Dynamic
model	enum	The AI model to use for vision processing	No	No

Outputs

Name	Type	Description
default	string	The language of the app as a two-letter ISO 639-1 code

Ask AI (Yes/No)

Restrictions: "none"

No docs available

Inputs

Name	Type	Description	Required	Dynamic
question	text	A yes/no question to ask the AI about the screen	Yes	No
region	region	Specify an optional region to crop the screen to before passing it to the AI	No	No
model	enum	The AI model to use for vision processing	No	No

Outputs

Name	Type	Description
default	boolean	true if the AI answers yes to the question, false otherwise
viewpoint	image	The viewpoint of the AI

Prompt AI

Restrictions: "none"

No docs available

Inputs

Name	Type	Description	Required	Dynamic
prompt	text	Plain english prompt to the AI	Yes	No
region	region	Specify an optional region to crop the screen to before passing it to the AI	No	No
model	enum	The AI model to use for vision processing	No	No

Outputs

Name	Type	Description
default	string	Output text generated by the AI
viewpoint	image	The viewpoint of the AI

AI Navigate (Deprecated)

Restrictions: "none"

Navigates to a specific UI element using AI-generated instructions.

Inputs

Name	Type	Description	Required	Dynamic
target	string	Name of the UI element to navigate to	Yes	No
region	region	Specify an optional region to crop the screen to before passing it to the AI	No	No
additionalInstructions	text	Any additional context or instructions for the LLM. This could be hints about the UI, like: 'The selected element is highlighed by a yellow color'.	No	No
waitAfterInput	number	Wait after executing the navigation sequence before the next step.	No	No
confirm	boolean	Select item after completing navigation	Yes	No
captureBeforeConfirm	boolean	Capture a screenshot before pressing the OK button	Yes	No
promptType	enum	The method to use for the navigation	Yes	No
model	enum	The AI model to use for vision processing	No	No

Outputs

Name	Type	Description
default	string	Output text generated by the AI
llmResponse	string	Response from LLM for debuggin
viewpoint	image	The viewpoint of the AI
screenBeforeConfirm	image	The screen show before press the OK button

AI Navigation Agent (Deprecated)

Restrictions: "none"

Navigates to a specific UI element using AI-generated instructions with recursive prompting.

Inputs

Name	Type	Description	Required	Dynamic
target	string	Name of the UI element to navigate to	Yes	No
region	region	Specify an optional region to crop the screen to before passing it to the AI	No	No
additionalInstructions	text	Any additional context or instructions for the LLM. This could be hints about the UI, like: 'The selected element is highlighed by a yellow color'.	No	No
commandsPerIteration	number	Number of commands to execute in each iteration	Yes	No
maxIterations	number	Maximum number of iterations before stopping	Yes	No
waitAfterInput	number	Wait after executing each command in the navigation sequence.	No	No
model	enum	The AI model to use for vision processing	No	No

Outputs

Name	Type	Description
default	string	Output text generated by the AI
llmResponse	string	Response from LLM for debugging
viewpoint	image	The final viewpoint of the AI

AI Agent

Restrictions: "none"

AI agent that can navigate and interact with TV devices using vision and control capabilities.

Inputs

Name	Type	Description	Required	Dynamic
goal	text	Describe what you want the AI agent to accomplish.

Examples:

• "Navigate to Settings" (just move focus, don't press)
• "Open Settings" (navigate and press SELECT)
• "Select the Netflix app"
• "Find the search icon in the top right" | Yes | No | | screenshot | boolean | Allow the AI agent to capture screenshots for visual analysis | No | No | | remoteControlActions | remote_keys | Select which remote control commands the AI agent can use | No | No | | sleep | boolean | Allow the AI agent to wait for a number of seconds (e.g. after launching an app or pressing a button) | No | No | | maxAttempts | number | Maximum number of navigation attempts before giving up | No | No | | screenshotDelayMs | number | Ms to wait before each capture. Leave empty for no delay. | No | No | | model | enum | The AI model to use for vision processing and decision making | No | No |

Outputs

Name	Type	Description
success	boolean	Whether the navigation successfully completed the goal
actions	string	Sequence of actions taken (e.g., 'up,right,down,select')
attempts	number	Number of navigation attempts made
finalScreen	image	Screenshot after navigation completion
reasoning	string	AI's explanation of what it did and why
default	null

AI Detect on Screen

Restrictions: "none"

No docs available

Inputs

Name	Type	Description	Required	Dynamic
element_type	text	The type of element to look for on the screen, e.g. 'button' or 'text'	Yes	No
text	text	The text of the element to look for on the screen, e.g. 'Login' or 'Submit'	No	No
region	region	Specify an optional region to crop the screen to before passing it to the AI	No	No
model	enum	The AI model to use for vision processing	No	No

Outputs

Name	Type	Description
default	boolean	Whether the element was detected or not
viewpoint	image	The viewpoint of the AI