Network Capture

Overview

Network capture for HTTP traffic is available for Appium sessions. After a capture has completed an HTTP archive (HAR) file will be available for download by members of your organization via the TV Labs sessions dashboard.

Supported Platforms

Network capture is currently available on the following platforms. Additional platform support will be rolled out incrementally.

Platform	Capture Method	Notes
Roku	Traffic Interception	Certificate install required in app bundle. See special considerations.
Android Mobile	Traffic Interception	Manifest configuration required. See special considerations.
Android TV	Traffic Interception	Manifest configuration required, and certificate install required in app bundle. See special considerations.
Fire TV	Traffic Interception	See Android TV
Google TV	Traffic Interception	See Android TV
tvOS	Traffic Interception
iOS	Traffic Interception
WebOS	Chrome DevTools Protocol
Tizen	Chrome DevTools Protocol

Network capture can be started in two different ways described below.

Using Capabilities

Network capture can be enabled on session start by setting the tvlabs:log_network capability to true in the Appium session capabilities.

{
  "tvlabs:constraints": "platform_key:web_os",
  "tvlabs:build": "...",
  "tvlabs:log_network": true
}

Using Execute Method

Network capture can be started or stopped in a middle of a session by using an Appium execute method. See more details on the Execute Methods page.

Viewing Network Logs

On the Sessions Dashboard

To view the network logs for a session in the TV Labs sessions dashboard, navigate to the session in the TV Labs sessions dashboard. Scroll down and find the "Network" tab, and click "View Requests" on the HTTP archive you wish to view. You may also click the "Download Selected" button to download the selected HTTP archives as a HAR file.

Download HAR API

An API endpoint to download HAR files for a session is available at /api/v1/download/http-archive/<archive_id>. This endpoint requires authentication with a valid API key. On success this endpoint returns a redirect to a download of the HAR file.

Example script to download all HAR files for a given Appium session ID:

#!/bin/usr/env bash

APPIUM_SESSION_ID="$1"
DIRECTORY="$APPIUM_SESSION_ID-har"

mkdir -p "$DIRECTORY"
pushd "$DIRECTORY"

if [ -z "$APPIUM_SESSION_ID" ]; then
  echo "Usage: $0 <appium-session-id>"
  exit 1
fi

if [ -z "$TVLABS_API_KEY" ]; then
  echo "error: TVLABS_API_KEY is required, please configure it in your environment"
  exit 1
fi

# Get all archive IDs from the session and download each HAR file
curl -s -H "Authorization: Bearer $TVLABS_API_KEY" "https://tvlabs.ai/api/v1/sessions/appium/$APPIUM_SESSION_ID" \
  | jq -r '.data.http_archives[].id' \
  | xargs -I {} curl -L -H "Authorization: Bearer $TVLABS_API_KEY" -o "{}.har" "https://tvlabs.ai/api/v1/download/http-archive/{}"

# List the downloaded HAR files
ls -la *
popd

See the API documentation for more details.

Third Party Tools

There are many alternative online tools available to view HAR files, but one of the easiest ways is to import them into the Chrome DevTools. Open the DevTools within Chrome, select the "Network" tab, and look for the "Import HAR" button

Special Platform Considerations

Android

This section describes how to package your Android application to allow network capture of requests made by the application. Due to limitations imposed by the Android platform on API level 24 or greater (Android 7.0), network capture requires additional changes to your application's manifest to allow the use of the TV Labs certificate (a user certificate). If your application is targeting API level 24 or greater, you will need to add a trust anchor to your application's manifest.

In your AndroidManifest, add a reference to a network_security_config XML file if it doesn't already exist:

<?xml version="1.0" encoding="utf-8"?>
<manifest ... >
  <application android:networkSecurityConfig="@xml/network_security_config" ... >
      ...
  </application>
</manifest>

If you added the network_security_config file reference in the previous step, create the file and add the following to allow the usage of user certificates in your test build:

<?xml version="1.0" encoding="utf-8"?>
<network-security-config>
  <base-config cleartextTrafficPermitted="true">
    <trust-anchors>
      <certificates src="system" />
      <certificates src="user" />
    </trust-anchors>
  </base-config>
</network-security-config>

See the Android Developer security configuration documentation for the latest details.

Android TV

Android TV platforms (including Google TV and Fire TV) disable install flows for CA certificates, so the TV Labs certificate must be embedded in the application package to use network capture with these devices. After setting up the Network Security Configuration outlined in the Android section above, follow the instructions below to build your application with the TV Labs certificate.

Downloading the Certificate

TV Labs provides an API endpoint to download the certificate bundle containing the TV Labs certificate. This example stores the certificate under certs/tvlabs-ca-bundle.pem in the application file tree.

curl -H "Authorization: Bearer $TVLABS_API_KEY" -o certs/tvlabs-ca-bundle.pem https://tvlabs.ai/api/v1/certificate/android_tv

This certificate is downloaded in the PEM format, but Android applications expect DER format. You can use the following command to convert the certificate file to DER format.

openssl x509 -in certs/tvlabs-ca-bundle.pem -out certs/tvlabs.cer -outform DER

Place the DER format certificate output at certs/tvlabs.cer in the application app/src/main/res/raw directory, and update your Network Security Configuration to include the new certificate.

<?xml version="1.0" encoding="utf-8"?>
<network-security-config>
  <base-config cleartextTrafficPermitted="true">
    <trust-anchors>
      <certificates src="system" />
      <certificates src="@raw/tvlabs" />
    </trust-anchors>
  </base-config>
</network-security-config>

Rebuild your application as normal, and your application package is now ready to be used with network interception on Android TV devices!

Roku

This section describes how to package your Roku BrightScript application with the TV Labs certificate. Due to limitations imposed by the Roku platform installing client certificates on the device is impossible, meaning to use network capture the TV Labs certificate must be embedded in the application package along with application logic when making network requests.

Downloading the Certificate

TV Labs provides an API endpoint to download the certificate bundle containing the TV Labs certificate. This example stores the certificate under certs/tvlabs-ca-bundle.pem in the application file tree.

curl -H "Authorization: Bearer $TVLABS_API_KEY" -o certs/tvlabs-ca-bundle.pem https://tvlabs.ai/api/v1/certificate/roku

Using the Certificate in Application Code

The ifHttpAgent interface provides a SetCertificatesFile method that can be used to set the certificate file used for HTTP requests made by the implementing object. Most Roku objects, including roSGNode, roVideoPlayer and roUrlTransfer objects implement this interface, see the Roku documentation for more details. If you are using a third-party library, please refer to the library's documentation for the correct way to set the certificate file.

Set the certificate file to the system default. This will be replaced with the TV Labs certificate in the next section:

req = CreateObject("roUrlTransfer")
req.SetCertificatesFile("common:/certs/ca-bundle.crt")
req.InitClientCertificates()
' ... make the request w/ certificate ...

important

If the server in your request does not use mTLS, the InitClientCertificates() function call will cause the request to timeout because the server will never request the client certificate. In this case, skip the InitClientCertificates() function call to avoid the timeout.

Swapping the Certificate

The Perl script below can be used to replace all instances of the certificate path values in the application code.

# Find and replace all instances of the certificate path in `.brs` files in the current directory tree
find . -name "*.brs" -type f -exec perl -pi -e 's|common:/certs/ca-bundle.crt|pkg:/certs/tvlabs-ca-bundle.pem|g' {} +

After running the replacement script, the application code will look like this:

req.SetCertificatesFile("pkg:/certs/tvlabs-ca-bundle.pem")

Package the application using your normal process, ensuring the TV Labs certificate is included in the certs directory. Your application package is now ready to be used with network interception!