AI

Prompt AI

Restrictions: "none"

No docs available

Inputs

Name	Type	Description	Required	Dynamic
prompt	text	Plain english prompt to the AI	Yes	No
region	region	Specify an optional region to crop the screen to before passing it to the AI	No	No
model	enum	The AI model to use for vision processing	No	No

Outputs

Name	Type	Description
default	string	Output text generated by the AI
viewpoint	image	The viewpoint of the AI

AI Detect on Screen

Restrictions: "none"

No docs available

Inputs

Name	Type	Description	Required	Dynamic
element_type	text	The type of element to look for on the screen, e.g. 'button' or 'text'	Yes	No
text	text	The text of the element to look for on the screen, e.g. 'Login' or 'Submit'	No	No
region	region	Specify an optional region to crop the screen to before passing it to the AI	No	No
model	enum	The AI model to use for vision processing	No	No

Outputs

Name	Type	Description
default	boolean	Whether the element was detected or not
viewpoint	image	The viewpoint of the AI

AI Navigation Agent

Restrictions: "none"

Navigates to a specific UI element using AI-generated instructions with recursive prompting.

Inputs

Name	Type	Description	Required	Dynamic
target	string	Name of the UI element to navigate to	Yes	No
region	region	Specify an optional region to crop the screen to before passing it to the AI	No	No
additionalInstructions	text	Any additional context or instructions for the LLM. This could be hints about the UI, like: 'The selected element is highlighed by a yellow color'.	No	No
commandsPerIteration	number	Number of commands to execute in each iteration	Yes	No
maxIterations	number	Maximum number of iterations before stopping	Yes	No
waitAfterInput	number	Wait after executing each command in the navigation sequence.	No	No
model	enum	The AI model to use for vision processing	No	No

Outputs

Name	Type	Description
default	string	Output text generated by the AI
llmResponse	string	Response from LLM for debugging
viewpoint	image	The final viewpoint of the AI

AI Navigate

Restrictions: "none"

Navigates to a specific UI element using AI-generated instructions.

Inputs

Name	Type	Description	Required	Dynamic
target	string	Name of the UI element to navigate to	Yes	No
region	region	Specify an optional region to crop the screen to before passing it to the AI	No	No
additionalInstructions	text	Any additional context or instructions for the LLM. This could be hints about the UI, like: 'The selected element is highlighed by a yellow color'.	No	No
waitAfterInput	number	Wait after executing the navigation sequence before the next step.	No	No
confirm	boolean	Select item after completing navigation	Yes	No
captureBeforeConfirm	boolean	Capture a screenshot before pressing the OK button	Yes	No
promptType	enum	The method to use for the navigation	Yes	No
model	enum	The AI model to use for vision processing	No	No

Outputs

Name	Type	Description
default	string	Output text generated by the AI
llmResponse	string	Response from LLM for debuggin
viewpoint	image	The viewpoint of the AI
screenBeforeConfirm	image	The screen show before press the OK button

AI Detect App Language

Restrictions: "none"

No docs available

Inputs

Name	Type	Description	Required	Dynamic
model	enum	The AI model to use for vision processing	No	No

Outputs

Name	Type	Description
default	string	The language of the app as a two-letter ISO 639-1 code