tvlabs execute

Experimental

tvlabs execute is experimental. Its flags, the TVLABS_* environment variables it sets, and its exit codes may all change without a major version bump, and it is not yet covered by the deprecation policy the rest of the CLI follows.

Pin your CLI version if you depend on it from CI, and tell us what you need at [email protected] — feedback now is what settles the interface.

Running your own test harness against a real device

Use tvlabs execute to request a device, connect to it, and run a script of yours against it. Where tvlabs run executes automation workflows you built in the TV Labs web interface, execute runs your harness — Appium, WebDriver, Playwright, a shell script, whatever your suite already uses — and gets out of the way.

tvlabs execute ./run_suite.sh --target 'platform_key:android_tv AND platform_os_variant:fire_tv AND year>=2023'

The command requests a device matching --target, waits for it to warm up, opens the tunnel, then runs your entrypoint with environment variables describing the session. Your entrypoint's exit code becomes the command's exit code. When it finishes, the session is ended and the tunnel torn down.

Everything the command prints itself goes to stderr. Stdout belongs entirely to your entrypoint, so a harness whose output is machine-read — TAP, JUnit on stdout, a $(tvlabs execute ...) substitution — stays clean.

The entrypoint

The entrypoint is the first argument. It runs wherever the CLI runs — your shell, a CI runner, a container — not on the device. It reaches the device over the tunnel.

# Both spellings work; the positional form is shorter.
tvlabs execute ./run_suite.sh --target 'platform_key:roku'
tvlabs execute --entrypoint ./run_suite.sh --target 'platform_key:roku'

It is executed directly, without a shell, so it takes no arguments of its own — put what your harness needs in --env or inside the script. Relative paths resolve against the directory you ran the command from, not against --context.

Environment contract

Two rules hold for every variable below, and both exist so that what is behind these values can change — SSH forwards today, something else later — without your harness changing with it:

• Every value is ready to use. A host:port is complete and a port is one the local end really binds. Never assemble an address from parts, and never assume a default.
• A variable that does not apply to your device's platform is absent, not empty. So ${TVLABS_ADB_ADDR:?} is a working assertion, and a harness can branch on whether a variable is set instead of on the platform key.

Every run

Variable	Description
TVLABS_SESSION_ID	The session's UUID
TVLABS_SESSION_URL	Link to the session in the TV Labs web interface
TVLABS_PLATFORM	Platform key of the matched device, e.g. android_tv. This is the platform, not the OS variant — a Fire TV or Google TV device reports android_tv
TVLABS_DEVICE_NAME	The matched device's name in TV Labs
TVLABS_DEVICE_IP	The device's address on the sidecar's private network. Useful for logs and correlation — not dialable from your harness; use TVLABS_DEVICE_HOST and the ports below
TVLABS_DEVICE_HOST	Host to reach the device's forwarded ports on. Normally the tunnel's bind address (see --bind-address); under --containerized it is host.docker.internal, the container's route back to your machine. Every port below that you dial is reachable there; TVLABS_APP_PORT is the exception
TVLABS_APP_PORT	Port your own web app should listen on for the device to reach it at local.tvlabs.ai. This one is a reverse forward — the CLI connects to your app on loopback whatever --bind-address says — so it is a port you serve, not one you dial
TVLABS_OUTPUT_DIR	Directory the CLI creates for this session's artifacts, at <--context>/.tvlabs/<session id>. Write your reports there and your CI step can collect them from the workspace. Under --containerized this is a /workspace/... path; the files still land under --context on the host

Per platform

Each platform adds the addresses its own tooling needs:

Platform	Adds
android_tv, android_mobile	TVLABS_ADB_ADDR, TVLABS_ADB_SERIAL, TVLABS_INSPECTOR_PORT
tizen	TVLABS_SDB_ADDR, TVLABS_INSPECTOR_PORT
web_os	TVLABS_ARES_DEVICE, TVLABS_SSH_ADDR, TVLABS_INSPECTOR_PORT
roku	TVLABS_ECP_PORT, TVLABS_ROKU_DEV_PORT
vega, tivo	TVLABS_INSPECTOR_PORT
viera, smartcast	nothing
google_chrome, firefox, safari	nothing — the debugger and WebDriver are forwarded on fixed ports (9222 and 9515) that --inspector-port does not move

Variable	Description
TVLABS_ADB_ADDR	host:port of the adb server the device is attached to. Point a client at it — adb -H "${TVLABS_ADB_ADDR%:*}" -P "${TVLABS_ADB_ADDR##*:}", or ANDROID_ADB_SERVER_ADDRESS / ANDROID_ADB_SERVER_PORT. Nothing to adb connect
TVLABS_ADB_SERIAL	The device's serial as that adb server reports it, so adb -s "$TVLABS_ADB_SERIAL" shell … works without parsing adb devices
TVLABS_SDB_ADDR	host:port of the sdb server the Tizen device is attached to
TVLABS_ARES_DEVICE	Name of the ares device profile the CLI registered for this session — always pass it: ares-install -d "$TVLABS_ARES_DEVICE" app.ipk. An ares command without -d targets the LG emulator, not your device
TVLABS_SSH_ADDR	host:port of the webOS device's SSH service, the one the ares profile above dials
TVLABS_ECP_PORT	Local port forwarded to the Roku device's ECP (External Control Protocol) service
TVLABS_ROKU_DEV_PORT	Local port forwarded to the Roku Development Application Installer — where a sideload is POSTed

TVLABS_INSPECTOR_PORT

Set only on the platforms above, and only because something is listening on it there. It means "a debugger is reachable here" — which debugger, and what is serving it, differs by platform:

Platform	What answers on the port
tizen	The sidecar's debug proxy in front of the device's inspector
web_os	The device's own inspector
vega	The unit's Node inspector
tivo	The unit's debugger
android_tv, android_mobile	The CLI's own DevTools bridge, which speaks CDP to your app's WebView through the device's adb server — this is the address chrome://inspect reaches the device at

On roku, smartcast, viera and the browser platforms the variable is absent: their debuggers listen on fixed ports that --inspector-port does not move (smartcast on 9555, the browsers on 9222 with WebDriver on 9515, Roku's BrightScript console on 8085 and its debug server on 8080), or there is no inspector at all. Earlier CLI versions set the variable on every platform, which on those advertised a port nothing was listening on.

Reserved names, and credentials

Every TVLABS_* name in this contract is reserved on every platform, not just on the one you are targeting: passing one as --env is an error rather than a silent override, so a CI job's flags mean the same thing whichever device it matches. A reserved name your platform does not define is also removed from the inherited environment, so a nested run cannot leak a stale value into an inner one.

Your entrypoint inherits the rest of the CLI's environment, minus TV Labs credentials: TVLABS_API_KEY and any other TVLABS_* variable that names a key, token, secret, password or passphrase is dropped rather than handed to a test runner that may log its environment. Your own secrets — cloud keys, registry tokens — are untouched. If your harness genuinely needs the API key, pass it explicitly:

tvlabs execute ./run_suite.sh \
  --target 'platform_key:tizen' \
  --env "TVLABS_API_KEY=$TVLABS_API_KEY"

Everything else passes straight through:

tvlabs execute ./run_suite.sh \
  --target 'platform_key:tizen' \
  --env SUITE=smoke \
  --env RETRIES=2

Installing a build

Upload and install a build as part of device warmup with --build:

tvlabs execute ./run_suite.sh --target 'platform_key:tizen' --build ./app.wgt

To reuse a build you already uploaded with tvlabs upload, pass its id instead. The two are mutually exclusive — a mistyped path fails immediately rather than being sent to the server as a build id that doesn't exist.

tvlabs execute ./run_suite.sh --target 'platform_key:tizen' --build-id 408cb137-1b75-4322-b8f6-9eccf5b6b149

Local requirements

Your harness drives the device with the platform's own tools, and those run on this host, not on the device: adb for Android TV, Fire TV and Google TV, sdb for Tizen, the ares CLI for webOS. The tunnel also binds a local port for every service the platform forwards — the session's adb server on 5037 for Android, its sdb server on 26099 for Tizen, and the rest of that platform's ports — and it cannot start if any one of them is already taken, whether by a vendor server of your own or by an unrelated dev server.

When --target names a platform_key, the tool and those ports are checked before a device is requested, so a host that cannot run the platform exits 1 in under a second, with no device allocated and no session created:

port 5037 is already in use. The TV Labs CLI serves the session's adb server on that port, so stop your local one with `adb kill-server` and try again

If the target does not name one platform the CLI recognises, the platform is not known until the server matches a device, and the same checks run then instead — before the tunnel is opened. That case ends the session and still exits 1 rather than 125: the fault is on this host, and no retry can install a missing tool for you.

--webos-tools-dir is part of the answer for webOS: pass it and the check looks for ares-setup-device in that directory instead of on PATH, which is where setup will look for it too.

--containerized is the other answer — see below.

Containerized runs

--containerized runs your entrypoint in a container on your own machine instead of directly on the host. Two things come with that:

• The image owns the toolchain. Your harness gets what the image provides, not what happens to be installed here, and none of your environment.
• No fixed local ports. The device forwards bind ephemeral ports instead of the platform's documented ones, so a run no longer collides with an adb server, a dev server on 8080, or a second tvlabs run — and two runs can share one machine.

This is not a security boundary. It is your machine, your image and your code; the value is reproducibility and staying out of the way of your own tooling.

tvlabs execute ./ecp_probe.sh \
  --containerized \
  --target 'platform_key:roku' \
  --context .

Read the ports from the contract, never from a literal

This is the one rule the mode depends on. TVLABS_ECP_PORT is 8060 on a host run and something like 49213 in a container — both are correct, because both are what the local end actually bound. A harness that hardcodes 8060 works in one mode and not the other; one that reads the variable works in both, and gains collision immunity for free.

curl -fsS "http://$TVLABS_DEVICE_HOST:$TVLABS_ECP_PORT/query/device-info"

What changes

	Host	Containerized
TVLABS_DEVICE_HOST	--bind-address (default localhost)	host.docker.internal
TVLABS_OUTPUT_DIR	<--context>/.tvlabs/<session>	/workspace/.tvlabs/<session>
Forwarded ports	the platform's documented ports	ephemeral, reported in the contract
Environment	yours, minus TV Labs credentials	only the TVLABS_* contract and your --env

--context is mounted at /workspace and is the only host directory the container can see, so your entrypoint has to live inside it. Artifacts written to TVLABS_OUTPUT_DIR still land under --context on the host, through the mount — the path the harness sees is just spelled differently.

Exit codes, streamed stdout and stderr, and signal handling are unchanged: your harness's exit code is still the command's, and a Ctrl-C still gives it SIGTERM, a grace period, then SIGKILL. The signal reaches your harness's children here too, at the same instant it reaches the harness.

One thing does differ, and it bites a harness that flushes on the way out. Your entrypoint is the container's init child, so when it exits the container goes with it and anything it spawned is killed immediately — a child part-way through writing a report loses the rest of its grace period. If your harness has children whose output you need, wait for them in your SIGTERM handler rather than exiting straight away.

Requirements and limits

• macOS with Docker Desktop. Linux and Windows are not supported yet, and podman is not either.
• roku only. Ephemeral ports break the platforms whose host-side setup dials a fixed local port — Tizen's sdb, webOS's ares profile, Android's adb — so those keep the host path for now. A --target naming one is refused before a device is requested.
• --app-port does not reach into the container. That forward is reversed: the CLI dials your app on the host's loopback, so serve it on the host rather than inside the container.
• --inspector-port stays fixed, because it exists for you to point a browser at. It is the one local port a containerized run still holds.
• The image is minimal — bash, curl, jq, CA certificates — and deliberately so for now. If your harness needs node, python3, a JDK or git, build your own with FROM tvlabsai/execute-runner:<version> and pass --image.

Exit codes

Built so a CI retry policy can tell your failures from ours:

Code	Meaning
entrypoint's own	Your harness ran to completion. Its status is the command's status
1	A local error — bad flags, a --target the server rejects, or a host that cannot run the platform's tools (see Local requirements). Never worth retrying unchanged
124	No device was provisioned and connected within --match-timeout
125	The platform failed before your entrypoint started. Safe to retry
126	The session died under a running entrypoint. Any results it published are partial
130 / 143	Interrupted by SIGINT / SIGTERM

125 and 126 are the two worth wiring into a retry rule: 125 means nothing of yours ran, so a retry costs nothing, while 126 means it ran against a device that disappeared.

Options

Flag	Default	Description
--target	required	KQL device query
--entrypoint	—	Script or binary to run. The positional argument is the shorter spelling
--context	.	Working directory for the entrypoint
--env	—	KEY=VALUE passed to the entrypoint. Repeatable
--build	—	Path to a build file to upload and install during warmup
--build-id	—	Existing build id to install during warmup
--match-timeout	30m	How long to wait for a device. Does not bound your entrypoint's run
--inspector-port, -i	9222	Local port forwarded to the device's web inspector
--app-port, -p	50552	Port of a locally running web application to forward to the device
--bind-address, -b	localhost	Address to bind the tunnel to, and the TVLABS_DEVICE_HOST your entrypoint dials
--webos-tools-dir	—	Directory containing the ares-* CLI tools for webOS, when they are not on PATH
--containerized	false	Run the entrypoint in a container on this machine. See Containerized runs
--image	—	Container image for --containerized. Defaults to tvlabsai/execute-runner:<cli version>

note

--match-timeout bounds provisioning only. Your entrypoint's own run is not bounded here — a long one is bounded by the session's server-side timeout. Name your own timeout inside the harness if you need one.

Signals and cleanup

Your entrypoint runs in its own process group, so a Ctrl-C or a CI job cancellation reaches the whole tree — including anything your harness spawned — rather than orphaning children against a device whose session is about to end. It gets SIGTERM first with a grace period to flush results, then SIGKILL.

Because it is a background process group, an interactive read from a terminal would stop it. Stdin is therefore passed through when it is a pipe or a file — so a CI job feeding its harness on stdin keeps working — and detached when it is a terminal.

GitHub Actions example

name: TV Device Tests
on: [push, pull_request]

jobs:
  device-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install TV Labs CLI
        run: /bin/bash -c "$(curl -fsSL https://tvlabs.ai/install.sh)"

      - name: Run suite against a Fire TV
        env:
          TVLABS_API_KEY: ${{ secrets.TVLABS_API_KEY }}
        run: |
          tvlabs execute ./run_suite.sh \
            --target 'platform_key:android_tv AND platform_os_variant:fire_tv AND year>=2023' \
            --build ./app.apk \
            --env SUITE=smoke