Back to blog

Consistent & Reliable Media Experience Powered by Dyte

Consistent & Reliable Media Experience Powered by Dyte

We at Dyte aim to provide a consistent and reliable video/audio experience to all of our users. It is kind of logical to assume that different users would use different devices and browsers. Dyte must support a variety of devices and browser types in order to provide a consistent and reliable experience to all users. And that's what makes it challenging:

  • Each browser type implements WebRTC specifications in a unique way.
  • Until recently, WebRTC specifications were not standardized.
  • There are numerous media failure scenarios that the SDK must consider in order to ensure reliable media delivery.

Let's take a look at some of the most common issues we encountered and how we dealt with them:

Issue Possible cause Action by Dyte SDK
Acquiring audio/video tracks fail 1. Some other application might have already acquired and locked the device
2. Faulty hardware
3. Driver/software might be buggy
If an alternative device exists try to acquire media from that device, if not throw an error for the user to check if the device is available
Acquiring audio/video tracks fail 1. Browser implementation for the MediaTrackConstraints may vary Use adapter.js, a shim to insulate apps from spec changes and prefix differences of browser-specific implementation of the WebRTC APIs
Acquiring audio/video tracks is successful but no audio / empty video in the track 1. Virtual Devices - The device might not provide the required media every time
2. Faulty hardware
3. Driver/software might be buggy
Check the track for silence / no data and re-acquire if necessary. If reacquire also produces a faulty track, switch to an alternative device if available
Acquiring audio/video tracks is successful but the track produces silent/empty data later in the middle of the call 1. Device disconnection/reconnection
2. Faulty hardware
3. Driver/software might be buggy
Listen for track-ended events, periodically check the track for silence / no data and re-acquire if necessary. If reacquire also produces a faulty track, switch to an alternative device if available

All of these issues are very critical and, if not handled properly, break the core feature of Dyte, thus requiring intensive testing around it. The next challenge was to automate the testing for all these scenarios. It is difficult to have access to physical devices in CI pipelines and therefore recreating these failure scenarios in a test environment is difficult.

Few more media-related behaviors that need to be working as expected and are harder to test without physical devices in a CI environment

Event Action by Dyte SDK
External speakers connected or disconnected (no mic) In Chromium, browsers switch the output to the connected/alternative device (Only chromium browsers support setting the output device)
External headset connected or disconnected (with mic) Release the current device if there is one and acquire the microphone of the newly connected/alternative device

We needed something that would be able to emulate the behavior of real devices in our CI environment.

We built Device Emulator, a javascript library to emulate devices with different capabilities.

DeviceEmulator.js introduces methods on the MediaDevices class to add, remove and control emulated devices. It uses AudioContext and oscillator node to generate the synthetic audio tracks and canvas for generating synthetic video devices. Methods like setSinkId, getUserMedia, getDisplayMedia, and enumerate devices are overridden to make them work with the emulated devices out of the box, all you have to do is import the library on top-level to polyfill the device emulator methods. Minimal example usage of the library:

import '@dyte-in/device-emulator';

// Add video input device. Similarly, we can also add audio input device
const videoInputId = navigator.mediaDevices.addEmulatedDevice('videoinput');
Using emulated video device as input
Using emulated video device as input
// Video track will produce empty data
navigator.mediaDevices.silenceDevice(videoInputId);
Silencing the video input device (video track produces empty black frames)
Silencing the video input device (video track produces empty black frames)
// Video track of the stream will end and subsequent getUserMedia 
// calls for this deviceId will throw error
navigator.mediaDevices.brickDevice(videoInputId);
Trying to acquire video track from a broken device (throws error)
Trying to acquire video track from a broken device (throws error)
// Add audio output device
const audioOutputId = navigator.mediaDevices.addEmulatedDevice('audiooutput');
Using emulated audio output device to redirect audio
Using emulated audio output device to redirect audio

To sum up what our library can do

  • Insert / Remove emulated audio (input and-or output) or video devices
  • Inject failure on a media track
  • Inject failure on a device

With a combination of these, it is now possible to simulate various edge conditions and experiment with different device capabilities to test how our SDK performs under different circumstances and properly handle all known points of failure. We heavily make use of this library along with Cypress, a reliable end-to-end testing framework that operates directly in the browser, to write automated integration tests for every feature of Dyte SDK.

Where do we run these tests?

It is essential to test our SDKs on various browsers and operating systems. Fortunately, there exists BrowserStack, a platform that provides 3000+ real mobile devices and browsers and integrates really well with Cypress and Selenium. Setting it up was just adding the following JSON file to our existing Cypress codebase:

{
  "auth": {
    "username": "$BROWSERSTACK_USERNAME",
    "access_key": "$BROWSERSTACK_ACCESSKEY"
  },
  "browsers": [{
      "browser": "chrome",
      "os": "Windows 10",
      "versions": ["latest", "latest - 1"]
    },
    {
      "browser": "firefox",
      "os": "OS X Mojave",
      "versions": ["latest", "latest - 1"]
    },
    {
      "browser": "edge",
      "os": "OS X Catalina",
      "versions": ["latest"]
    }
  ],
  "run_settings": {
    "cypress_config_file": "./cypress.json",
    "cypress_version": "9",
    "project_name": "Web-Core E2E integration Tests",
    "build_name": "$CYPRESS_BUILD_NAME",
		"parallels": 5,
}

All the e2e tests now run in different browsers+devices and a detailed report that includes logs, screen capture, etc. can be viewed on the BrowserStack dashboard:

Untitled
BrowserStack Dashboard

Conclusion

Hope you got some insights into the testing process of the core features of Dyte SDK and how we make sure all our new releases are supported on a wide range of browsers and devices. Testing our product thoroughly to ship a reliable product is always our top priority as we are committed to providing the best audio/video calling experience to our customers! We love open source and we will soon be making the device emulator library publicly available so keep an eye on our GitHub page!

If you haven’t heard about Dyte yet, head over to https://dyte.io to learn how we are revolutionizing live video calling through our SDKs and libraries and how you can get started quickly on your 10,000 free minutes which renew every month. If you have any questions, you can reach us at support@dyte.io or ask our developer community.

Great! Next, complete checkout for full access to Dyte.
Welcome back! You've successfully signed in.
You've successfully subscribed to Dyte.
Success! Your account is fully activated, you now have access to all content.
Success! Your billing info has been updated.
Your billing was not updated.