Skip to main content

CI/CD Integration

Integrate TV Labs testing into your CI/CD pipelines to enable automated testing on real devices as part of your development workflow.

Overview​

TV Labs provides two separate routes for CI/CD integration:

Route 1: Visual No-Code Workflows​

Execute visual automation projects using tvlabs run.

Key advantage: Same workflow runs across all TV platforms (Roku, Samsung, LG, Apple TV) without platform-specific code changes.

Best for teams who:

  • Create visual workflows in the TV Labs web interface
  • Need to integrate existing TV Labs projects into CI/CD
  • Want to save time on test automation setup

Route 2: Custom Appium Scripts​

Run custom Appium test scripts via the TV Labs Appium proxy. Provides full programmatic control with platform-specific automation capabilities.

Best for teams who:

  • Write custom test automation code
  • Need full programmatic control over test execution
  • Want to integrate existing Appium test suites

Choose the route that best fits your team's testing approach.

Route 1: Visual No-Code Workflows with tvlabs run​

The tvlabs run command executes automation projects that include visual workflows, target configurations, and build deployments.

GitHub Actions Example​

name: TV Labs Automation Tests
on: [push, pull_request]

jobs:
  tv-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install TV Labs CLI
        run: /bin/bash -c "$(curl -fsSL https://tvlabs.ai/install.sh)"

      - name: Upload app build
        env:
          TVLABS_API_KEY: ${{ secrets.TVLABS_API_KEY }}
        run: |
          BUILD_ID=$(tvlabs upload -i ./dist/app.zip)
          echo "BUILD_ID=$BUILD_ID" >> $GITHUB_ENV

      - name: Run TV automation tests
        env:
          TVLABS_API_KEY: ${{ secrets.TVLABS_API_KEY }}
        run: |
          tvlabs run --project ${{ vars.PROJECT_ID }} \
            --app ${{ vars.APP_ID }} \
            --device ${{ vars.DEVICE_ID }} \
            --build ${{ env.BUILD_ID }} \
            --entrypoint ${{ vars.WORKFLOW_ID }}

Route 2: Custom Appium Scripts​

Use the TV Labs Appium proxy to run Appium tests on real devices. Supports both CLI and REST API integration methods.

Build Upload Options​

Before running Appium tests, you need to upload your application build. TV Labs provides two methods:

  1. Using TV Labs CLI - tvlabs upload command
  2. Using REST API - Direct API calls without CLI dependency

Both methods return a build ID that you use in your Appium test capabilities.

GitHub Actions with Appium​

name: TV Labs Appium Tests
on: [push, pull_request]

jobs:
  appium-tests:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install dependencies
        run: npm ci

      - name: Install TV Labs CLI
        run: /bin/bash -c "$(curl -fsSL https://tvlabs.ai/install.sh)"

      - name: Upload app build (CLI method)
        env:
          TVLABS_API_KEY: ${{ secrets.TVLABS_API_KEY }}
        run: |
          BUILD_ID=$(tvlabs upload -i ./dist/app.zip)
          echo "BUILD_ID=$BUILD_ID" >> $GITHUB_ENV

      - name: Run Appium tests
        env:
          TVLABS_API_KEY: ${{ secrets.TVLABS_API_KEY }}
          TVLABS_BUILD_ID: ${{ env.BUILD_ID }}
        run: npm test

      - name: Upload test results
        uses: actions/upload-artifact@v4
        if: always()
        with:
          name: test-results
          path: |
            test-results/
            screenshots/

Alternative Upload Method: Instead of using the CLI, you can upload builds using the REST API directly. See the Build Upload documentation for details on the API method.

Sample Appium Test Script​

// test/tv-test.js
import { remote } from 'webdriverio';

describe('TV App Tests', () => {
  let driver;

  beforeEach(async () => {
    const capabilities = {
      'tvlabs:build': process.env.TVLABS_BUILD_ID,
      'tvlabs:constraints': { platform_key: 'roku' },
    };

    driver = await remote({
      hostname: 'appium.tvlabs.ai',
      port: 4723,
      protocol: 'https',
      capabilities,
      headers: { Authorization: `Bearer ${process.env.TVLABS_API_KEY}` },
      connectionRetryTimeout: 300_000,
    });
  });

  afterEach(async () => {
    await driver?.deleteSession();
  });

  it('should navigate through the app', async () => {
    await driver.executeScript('roku: pressKey', [{ key: 'Right' }]);
    await driver.executeScript('roku: pressKey', [{ key: 'Select' }]);
    await driver.takeScreenshot();
  });
});

Advanced Configuration​

Parallel Testing Across Multiple Devices​

Route 1 (Visual No-Code Workflows) enables running identical tests across multiple TV platforms simultaneously. Visual workflows are platform-agnostic, requiring no code changes between Roku, Samsung, LG, and Apple TV.

name: Parallel Device Testing
on: [push, pull_request]

jobs:
  parallel-tv-tests:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        device_config:
          - name: 'Roku TV'
            device_id: ${{ vars.ROKU_DEVICE_ID }}
          - name: 'Samsung TV'
            device_id: ${{ vars.SAMSUNG_DEVICE_ID }}
          - name: 'LG TV'
            device_id: ${{ vars.LG_DEVICE_ID }}
          - name: 'Apple TV'
            device_id: ${{ vars.APPLE_TV_DEVICE_ID }}

    steps:
      - uses: actions/checkout@v4

      - name: Install TV Labs CLI
        run: /bin/bash -c "$(curl -fsSL https://tvlabs.ai/install.sh)"

      - name: Upload app build
        env:
          TVLABS_API_KEY: ${{ secrets.TVLABS_API_KEY }}
        run: |
          BUILD_ID=$(tvlabs upload -i ./dist/app.zip)
          echo "BUILD_ID=$BUILD_ID" >> $GITHUB_ENV

      - name: Run visual tests on ${{ matrix.device_config.name }}
        env:
          TVLABS_API_KEY: ${{ secrets.TVLABS_API_KEY }}
        run: |
          tvlabs run --project ${{ vars.PROJECT_ID }} \
            --app ${{ vars.APP_ID }} \
            --device ${{ matrix.device_config.device_id }} \
            --build ${{ env.BUILD_ID }}

Technical advantages:

  • Platform Agnostic: Same workflow runs on all TV platforms without modification
  • Simplified Maintenance: One test maintains compatibility across all platforms
  • True Parallel Execution: Identical tests run simultaneously across different TV OS types

Route 2 (Appium) requires separate test scripts for each platform with platform-specific commands.

Regional Testing​

Run tests in different geographic regions using the --region flag:

- name: Test in different regions
  strategy:
    matrix:
      region:
        - new_york_usa
        - buenos_aires_argentina
        - sydney_australia
        - sao_paulo
        - canada_montreal
        - santiago_chile
        - malaysia
        - mexico
        - lima_peru
        - manila_phillipines
        - singapore
        - stockholm_sweden
        - virginia_usa
        - london_uk
  env:
    TVLABS_API_KEY: ${{ secrets.TVLABS_API_KEY }}
  run: |
    tvlabs run --project ${{ vars.PROJECT_ID }} \
      --app ${{ vars.APP_ID }} \
      --device ${{ vars.DEVICE_ID }} \
      --region ${{ matrix.region }}

All available regions can be retrieved from the REST API route /api/v1/teleport/regions.

Development Status​

Current features:

  • Basic CI/CD integration including GitHub Actions
  • Build artifact management (CLI and REST API)
  • Regional testing capabilities
  • Flexible integration options (with or without CLI)

Coming Soon:

  • CI/CD integration with GitLab, Azure DevOps, and other platforms
  • TV Labs Github Action (tvlabs/setup-tvlabs-cli@v1)

Getting Help​

If you have questions about CI/CD integration or need assistance setting up your pipelines: