Know how Dyte uses MoleculerJS to power microservices & APIs

MoleculerJS is a fast, easy-to-use, and fault-tolerant microservices framework for Javascript and Typescript. At Dyte, we have successfully used MoleculerJS to connect our different microservices and APIs to serve lightning-fast and error-free queries to all users and clients.

In this blog post, we will be giving a high-level overview of how we structure all of our services around Moleculer. Then, we will understand how we use Moleculer to collect critical insights and metrics to debug and optimize our infrastructure to serve millions of minutes of live video calls every month. So let’s get started!

An Overview of Dyte’s Backend Architecture 📐

Before getting into Moleculer’s awesome features, let me briefly introduce how Moleculer fits into Dyte’s infrastructure. Dyte’s REST API enables you to interact with the platform programmatically and lets you do useful tasks, like:

• Create/Update meetings
• Add participants to meetings
• Customize meeting UI components
• Customize participant permissions and roles
• Trigger/Query meeting recordings
• Register Webhooks to receive notifications from meetings
• Get meeting analytics/statistics and much more

That’s a lot of functionality! We at Dyte realized at an early stage that a monolithic backend service would not scale well as we introduced more and more features. Hence, we split most of the parts above into different services, following the microservice architecture. We explored many message queue and microservice frameworks like RabbitMQ, Apache Kafka, Istio. Still, we found that even though these were highly performant, using these frameworks directly required additional libraries/layers to work well with Javascript/ Typescript. And that’s when we came across MoleculerJS, which did precisely that!

Moleculer acts as a wrapper around frameworks like NATS, Redis, MQTT, Kafka, etc., and provides javascript APIs to communicate across different microservices.

The diagram above gives a high-level overview of how Dyte’s backend servers are designed around the microservice pattern. Each time we get an API request, the request is mapped to one of our API backend replicas running on our cluster, which then uses a Moleculer action (e.g., session.get()) to route the call to the appropriate microservice. Under the hood, Moleculer implements message passing between services using a transporter (in our case, Redis). Moleculer actions are robust and have inbuilt support for load balancing. The cool part about this feature is that it has zero extra infra cost, Moleculer maintains state in the transporter itself to balance your action invocations between replicas. Some key benefits that we are realizing through Moleculer are:

Ease of development and debugging 👨‍💻

At Dyte, we use the Moleculer-decorators npm package to make it even simpler to develop microservices. Want to convert a class into a microservice and expose its methods as actions on the service mesh?

class MyService {
  public async myMethod(param1: string, param2: number) {
    // Some cool stuff
    console.log(param1);
    console.log(param2);
  }
}

Just use @Service(), @Action(), and you are all set!

@Service({
  name: "My Service"
})
class MyService {
  @Action({
    params: {
      param1: { type: 'string' },
      param2: { type: 'number' },
    },
  })
  public async myMethod(ctx: Context<{ param1: string, param2: number }>) {
    // Some cool stuff
    console.log(ctx.params.param1);
    console.log(ctx.params.param2);
  }
}

The parameters are not only type-checked during compile-time, but during run time as well. Internally, Moleculer uses the powerful fastest-validator package to perform run-time data validation, which helps prevent vulnerabilities/crashes due to users passing in unexpected inputs.

Another valuable feature of Moleculer we widely use in Dyte during local development is the Moleculer-cli, which lets you trigger actions/events and view actions/services quickly.

Events 🔀

A large part of our infrastructure is event-driven, i.e., events are triggered by services, and other services need to listen for these events and then do operations accordingly. For example,

1. A participant joins a meeting, and then the “participant joined” event is triggered.
2. The stats service notes down joining times and records them in the database.
3. The auto-scaling service updates the participant count for the room to make better decisions about allocating future rooms to servers… and so on.

Moleculer has inbuilt support for events. One of its best features is that these are load-balanced across replicas of the instance, i.e., each event is guaranteed only to be received by one replica.

Fault Tolerance ⚠️

At Dyte, many times in production, we encounter service failures. For example, specific actions require a high number of DB calls / CPU processing. A high load on these actions can cause increased response times and sometimes even bottleneck the physical node on which the service runs, thus affecting other services, leading to cascading failures. Moleculer has two helpful features to mitigate these issues:

1. Retry Policies let us configure timeouts for action calls and rules to retry actions on timeouts with configurable backoff times and maximum limits.

// Retry policy settings. More info: https://moleculer.services/docs/0.14/fault-tolerance.html#Retry
    retryPolicy: {
        // Enable feature
        enabled: true,
        // Count of retries
        retries: 5,
        // First delay in milliseconds.
        delay: 100,
        // Maximum delay in milliseconds.
        maxDelay: 1000,
        // Backoff factor for delay. 2 means exponential backoff.
        factor: 2,
        // A function to check failed requests.
        check: (err: Errors.MoleculerError) => err && !!err.retryable,
    },

2. Circuit Breakers: In some cases, it is more beneficial to cut off calls to action to prevent cascading failures to other system components. This feature allows us to configure the maximum number of action failures before blocking all calls to the action.

// Settings of Circuit Breaker. More info: https://moleculer.services/docs/0.14/fault-tolerance.html#Circuit-Breaker
    circuitBreaker: {
        // Enable feature
        enabled: true,
        // Threshold value. 0.5 means that 50% should be failed for tripping.
        threshold: 0.5,
        // Minimum request count. Below it, CB does not trip.
        minRequestCount: 20,
        // Number of seconds for the time window.
        windowTime: 60,
        // Number of milliseconds to switch from open to half-open state
        halfOpenTime: 10 * 1000,
        // A function to check failed requests.
        check: (err: Errors.MoleculerError) => err && err.code >= 500,
    },

Metrics, metrics, and more metrics! 📊

Moleculer exposes many metrics crucial for Dyte to monitor how well our system performs and make optimizations accordingly. The best part, these metrics are exposable as Prometheus metrics, which means we could easily plug them into our existing monitoring system. The complete list of metrics that are exposed can be found here, but the most useful ones we found are:

• Action response times (p50, p90, p99)
• Action requests per minute
• Total errors/ error rate
• CPU/Memory Utilization

At Dyte, we integrated these statistics with Last9's powerful dashboards and anomaly detection alerting features. Here is an example of the response times from one of the actions on our internal monitoring panel:

Under 30-40ms, that looks a-okay!

Request Tracing with Jaeger 🔬

A feature of Moleculer we recently discovered was its ability to export tracing-related information to other popular services like Jaeger. Though the Prometheus metrics provided us with valuable high-level overviews of how our actions are performing, tracing gives us an in-depth analysis of each action call in a chain of sequential calls. This lets us identify which actions contribute more to the total response time of an API endpoint, just enabling us to make better decisions on architectural changes we must make to mitigate these bottlenecks.

Final Thoughts 🥳

Microservices are the future, and in my opinion, MoleculerJS is a hidden gem in this space, especially if you are working on a Javascript/Typescript project. We at Dyte are continuing to take advantage of its features and use them daily for speedy development. Most of all, it gives immense visibility on how well your systems are performing, thus letting you concentrate on what’s most important, reducing errors rates and serving your customers with minimum latency. I hope this helps, and I highly suggest you give it a shot.

In our next blog post on this topic, we will compare Moleculer with other popular microservice frameworks like Kafka, Istio, etc. Make sure you don't miss that. Till then, adios! 👋

If you haven’t heard about Dyte yet, head over to https://dyte.io to learn how we are revolutionizing live video calling through our SDKs and libraries and how you can get started quickly on your 10,000 free minutes which renew every month. If you have any questions, you can reach us at support@dyte.io or ask our developer community.