Advanced

Circuit Breaker Pattern

Prevent cascading failures and improve resilience with MoroJS's built-in circuit breaker pattern. Automatically detect and handle service failures gracefully.

What is a Circuit Breaker?

A circuit breaker prevents cascading failures by automatically detecting and handling service failures. When a service fails repeatedly, the circuit opens to prevent further requests.

Circuit Breaker States

CLOSED

Requests pass through normally. Failures are tracked.

OPEN

Requests fail immediately. Prevents cascade failures.

HALF_OPEN

Testing recovery. Limited requests allowed.

Basic Circuit Breaker Usagetypescript

1import { createApp, CircuitBreaker } from '@morojs/moro';
2
3const app = await createApp();
4
5// Create circuit breaker for external API
6const apiBreaker = new CircuitBreaker({
7  failureThreshold: 5,     // Open after 5 failures
8  resetTimeout: 30000,     // Try again after 30 seconds
9  monitoringPeriod: 10000  // Track failures over 10 seconds
10});
11
12app.get('/external-data', async (req, res) => {
13  try {
14    const data = await apiBreaker.execute(async () => {
15      const response = await fetch('https://api.example.com/data');
16      return await response.json();
17    });
18
19    res.json(data);
20  } catch (error: any) {
21    if (error.message === 'Circuit breaker is OPEN') {
22      return res.status(503).json({
23        error: 'Service temporarily unavailable',
24        retryAfter: 30
25      });
26    }
27    throw error;
28  }
29});
30
31await app.listen(3000);

How It Works

State Transitions

The circuit breaker transitions between three states based on the health of the protected service:

CLOSED → OPEN: When failure count reaches the threshold, the circuit opens to prevent further requests
OPEN → HALF_OPEN: After the reset timeout expires, the circuit enters half-open state to test recovery
HALF_OPEN → CLOSED: If test requests succeed, the circuit closes and normal operation resumes
HALF_OPEN → OPEN: If test requests fail, the circuit reopens and the reset timeout starts again

Circuit Breaker Configurationtypescript

1// Create circuit breakers for different services
2const breakers = {
3  database: new CircuitBreaker({
4    failureThreshold: 5,
5    resetTimeout: 30000
6  }),
7
8  externalApi: new CircuitBreaker({
9    failureThreshold: 3,
10    resetTimeout: 60000
11  }),
12
13  cache: new CircuitBreaker({
14    failureThreshold: 10,
15    resetTimeout: 5000
16  })
17};
18
19// Protected database operation
20async function getUser(id: string) {
21  try {
22    return await breakers.database.execute(async () => {
23      return await db.query('SELECT * FROM users WHERE id = ?', [id]);
24    });
25  } catch (error: any) {
26    if (error.message === 'Circuit breaker is OPEN') {
27      // Try cache fallback
28      return await breakers.cache.execute(async () => {
29        return await cache.get(`user:${id}`);
30      });
31    }
32    throw error;
33  }
34}

Usage Examples

Built-in Circuit Breakers for Jobstypescript

1// Background jobs automatically use circuit breakers
2app.scheduleJob({
3  name: 'external-api-sync',
4  schedule: '*/5 * * * *', // Every 5 minutes
5  handler: async () => {
6    // If this fails repeatedly, circuit breaker opens
7    const response = await fetch('https://api.example.com/data');
8    return await response.json();
9  },
10  options: {
11    enableCircuitBreaker: true, // Enabled by default
12    maxRetries: 3
13  }
14});

Manual Circuit Breaker with Fallbacktypescript

1async function fetchDataWithFallback() {
2  try {
3    return await breaker.execute(async () => {
4      return await fetchFromPrimaryAPI();
5    });
6  } catch (error: any) {
7    if (error.message === 'Circuit breaker is OPEN') {
8      // Use cached data or secondary source
9      return await fetchFromCache();
10    }
11    throw error;
12  }
13}

Multiple Circuit Breakerstypescript

1const breakers = new Map<string, CircuitBreaker>();
2
3breakers.set('auth', new CircuitBreaker({
4  failureThreshold: 3,
5  resetTimeout: 30000
6}));
7
8breakers.set('payment', new CircuitBreaker({
9  failureThreshold: 2,
10  resetTimeout: 60000
11}));
12
13breakers.set('email', new CircuitBreaker({
14  failureThreshold: 10,
15  resetTimeout: 30000
16}));
17
18// Use different breakers for different operations
19app.get('/users/:id', async (req, res) => {
20  const authBreaker = breakers.get('auth');
21
22  try {
23    const user = await authBreaker!.execute(async () => {
24      return await getUserFromAuthService(req.params.id);
25    });
26    res.json(user);
27  } catch (error) {
28    res.status(503).json({ error: 'Auth service unavailable' });
29  }
30});

Monitoring and Events

State Change Eventstypescript

1const breaker = new CircuitBreaker({
2  failureThreshold: 3,
3  resetTimeout: 5000
4});
5
6breaker.on('open', () => {
7  console.log('⚠️  Circuit breaker opened - service is failing');
8  // Alert monitoring system
9  alertMonitoring('circuit_breaker_open');
10});
11
12breaker.on('closed', () => {
13  console.log('✅ Circuit breaker closed - service recovered');
14  alertMonitoring('circuit_breaker_closed');
15});
16
17breaker.on('halfOpen', () => {
18  console.log('🔄 Circuit breaker half-open - testing recovery');
19  alertMonitoring('circuit_breaker_half_open');
20});

Health Check Endpointtypescript

1const breakers = new Map<string, CircuitBreaker>();
2
3app.get('/health/circuit-breakers', (req, res) => {
4  const status = Array.from(breakers.entries()).map(([name, breaker]) => ({
5    name,
6    state: breaker.getState(),
7    failures: breaker.getFailures(),
8    isOpen: breaker.isOpen()
9  }));
10
11  const anyOpen = status.some(b => b.isOpen);
12
13  res.status(anyOpen ? 503 : 200).json({
14    status: anyOpen ? 'degraded' : 'healthy',
15    breakers: status
16  });
17});

Configuration Examples

Different Breaker Configurationstypescript

1// Fast recovery for transient errors
2const fastBreaker = new CircuitBreaker({
3  failureThreshold: 2,
4  resetTimeout: 5000
5});
6
7// Slow recovery for persistent failures
8const slowBreaker = new CircuitBreaker({
9  failureThreshold: 10,
10  resetTimeout: 300000 // 5 minutes
11});
12
13// Sensitive breaker for critical operations
14const sensitiveBreaker = new CircuitBreaker({
15  failureThreshold: 1,  // Open immediately on failure
16  resetTimeout: 60000
17});
18
19// Tolerant breaker for flaky services
20const tolerantBreaker = new CircuitBreaker({
21  failureThreshold: 20,
22  resetTimeout: 10000
23});

Dynamic Adaptive Circuit Breakertypescript

1class AdaptiveCircuitBreaker {
2  private breaker: CircuitBreaker;
3  private successCount = 0;
4  private failureRate = 0;
5
6  constructor() {
7    this.breaker = new CircuitBreaker({
8      failureThreshold: 5,
9      resetTimeout: 30000
10    });
11  }
12
13  async execute<T>(fn: () => Promise<T>): Promise<T> {
14    try {
15      const result = await this.breaker.execute(fn);
16      this.successCount++;
17      this.adjustThreshold();
18      return result;
19    } catch (error) {
20      this.adjustThreshold();
21      throw error;
22    }
23  }
24
25  private adjustThreshold() {
26    // Adjust based on success rate
27    const total = this.successCount + this.breaker.getFailures();
28    if (total > 100) {
29      this.failureRate = this.breaker.getFailures() / total;
30
31      // Adjust threshold based on failure rate
32      if (this.failureRate > 0.5) {
33        // High failure rate - be more sensitive
34        this.breaker = new CircuitBreaker({
35          failureThreshold: 3,
36          resetTimeout: 60000
37        });
38      }
39    }
40  }
41}

API Reference

CircuitBreaker Class

typescript

class CircuitBreaker extends EventEmitter {
  constructor(options: CircuitBreakerOptions);

  // Execute protected function
  execute<T>(fn: () => Promise<T>): Promise<T>;

  // Get current state
  getState(): 'CLOSED' | 'OPEN' | 'HALF_OPEN';

  // Check if open
  isOpen(): boolean;

  // Get failure count
  getFailures(): number;

  // Manually reset
  reset(): void;

  // Events
  on(event: 'open', listener: () => void): this;
  on(event: 'closed', listener: () => void): this;
  on(event: 'halfOpen', listener: () => void): this;
  on(event: 'reset', listener: () => void): this;
}

interface CircuitBreakerOptions {
  failureThreshold: number;    // Number of failures before opening
  resetTimeout: number;         // Milliseconds before trying HALF_OPEN
  monitoringPeriod?: number;    // Milliseconds for failure tracking window
}

Best Practices

Set Appropriate Thresholds

Base thresholds on service characteristics. Stable services should have lower thresholds, while flaky services can be more tolerant.

typescript

// Good: Based on service characteristics
const databaseBreaker = new CircuitBreaker({
  failureThreshold: 5,
  resetTimeout: 30000
});

Provide Fallbacks

Always have a fallback strategy when the circuit is open. Use cached data, secondary services, or default responses.

typescript

// Good: Always have a fallback
try {
  return await breaker.execute(() => fetchFromDB(id));
} catch (error: any) {
  if (error.message === 'Circuit breaker is OPEN') {
    return await fetchFromCache(id);
  }
}

Monitor State Changes

Alert on circuit breaker state changes. Open circuits indicate service degradation that needs immediate attention.

typescript

breaker.on('open', () => {
  logger.error('Service degraded');
  sendAlert('CircuitBreakerOpen');
});

Separate Breakers

Use different circuit breakers for independent services. Don't let one service failure affect others.

typescript

// Good: Separate breakers
const breakers = {
  auth: new CircuitBreaker({...}),
  payment: new CircuitBreaker({...})
};

Circuit Breaker Pattern

What is a Circuit Breaker?

Circuit Breaker States

CLOSED

OPEN

HALF_OPEN

How It Works

State Transitions

Usage Examples

Monitoring and Events

Configuration Examples

API Reference

CircuitBreaker Class

Best Practices

Set Appropriate Thresholds

Provide Fallbacks

Monitor State Changes

Separate Breakers

Next Steps

Background Jobs

Performance

Testing