rpcx服务框架浅析12-断路器

RPCX分布式服务框架主要致力于提供高性能和透明化的RPC远程服务调用。

断路器

      (百度一下，到处都是概念说明，懒得打字下面全是百度抄的)在分布式环境下，特别是微服务结构的分布式系统中， 一个软件系统调用另外一个远程系统是非常普遍的。这种远程调用的被调用方可能是另外一个进程，或者是跨网路的另外一台主机, 这种远程的调用和进程的内部调用最大的区别是，远程调用可能会失败，或者挂起而没有任何回应，直到超时。更坏的情况是， 如果有多个调用者对同一个挂起的服务进行调用，那么就很有可能的是一个服务的超时等待迅速蔓延到整个分布式系统，引起连锁反应， 从而消耗掉整个分布式系统大量资源。最终可能导致系统瘫痪。

断路器（Circuit Breaker）模式就是为了防止在分布式系统中出现这种瀑布似的连锁反应导致的灾难。

一旦某个电器出问题，为了防止灾难，电路的保险丝就会熔断。断路器类似于电路的保险丝， 实现思路非常简单，可以将需要保护的远程服务嗲用封装起来，在内部监听失败次数， 一旦失败次数达到某阀值后，所有后续对该服务的调用，断路器截获后都直接返回错误到调用方，而不会继续调用已经出问题的服务， 从而达到保护调用方的目的, 整个系统也就不会出现因为超时而产生的瀑布式连锁反应。

RPCX断路器

RPCX断路器定义

type Breaker interface {
    Call(func() error, time.Duration) error
    Fail()
    Success()
    Ready() bool
}

消费方访问提供方获取client时进行判断，如下：

func (c *xClient) getCachedClient(k string) (RPCClient, error) {
    c.mu.RLock()
    breaker, ok := c.breakers.Load(k)
    if ok && !breaker.(Breaker).Ready() {
        c.mu.RUnlock()
        return nil, ErrBreakerOpen
    }
    .....
}

错误发生时

err := client.Connect(network, addr)
if err != nil {
    if breaker != nil {
        breaker.(Breaker).Fail()
    }
    c.mu.Unlock()
    return nil, err
}

常见断路器

ThresholdBreaker：trip if the function fails 10 times

circuit.NewThresholdBreaker(10)

CircuitBreaker：trip when error rate hits 95%, with at least 100 samples

circuit.NewRateBreaker(0.95, 100)

ConsecutiveBreaker：trip if 10 consecutive failures occur

circuit.NewConsecutiveBreaker(10)

ConsecCircuitBreaker：is window sliding CircuitBreaker with failure threshold.

NewConsecCircuitBreaker(5, 100*time.Millisecond)

Goland circuitbreaker包

Circuitbreaker provides an easy way to use the Circuit Breaker pattern in a Go program.

Circuit breakers are typically used when your program makes remote calls. Remote calls can often hang for a while before they time out. If your application makes a lot of these requests, many resources can be tied up waiting for these time outs to occur. A circuit breaker wraps these remote calls and will trip after a defined amount of failures or time outs occur. When a circuit breaker is tripped any future calls will avoid making the remote call and return an error to the caller. In the meantime, the circuit breaker will periodically allow some calls to be tried again and will close the circuit if those are successful.

Installation
      go get github.com/rubyist/circuitbreaker

Examples
Here is a quick example of what circuitbreaker provides

// Creates a circuit breaker that will trip if the function fails 10 times
cb := circuit.NewThresholdBreaker(10)

events := cb.Subscribe()
go func() {
  for {
    e := <-events
    // Monitor breaker events like BreakerTripped, BreakerReset, BreakerFail, BreakerReady
  }
}()

cb.Call(func() error {
    // This is where you'll do some remote call
    // If it fails, return an error
}, 0)

Circuitbreaker can also wrap a time out around the remote call.

// Creates a circuit breaker that will trip after 10 failures
// using a time out of 5 seconds
cb := circuit.NewThresholdBreaker(10)

cb.Call(func() error {
  // This is where you'll do some remote call
  // If it fails, return an error
}, time.Second * 5) // This will time out after 5 seconds, which counts as a failure

// Proceed as above

Circuitbreaker can also trip based on the number of consecutive failures.

// Creates a circuit breaker that will trip if 10 consecutive failures occur
cb := circuit.NewConsecutiveBreaker(10)

// Proceed as above

Circuitbreaker can trip based on the error rate.

// Creates a circuit breaker based on the error rate
cb := circuit.NewRateBreaker(0.95, 100) // trip when error rate hits 95%, with at least 100 samples

// Proceed as above

系列文章

• rpcx服务框架浅析1-角色说明
• rpcx服务框架浅析2-服务发布(以zookeeper作为注册中心的Provider例子)
• rpcx服务框架浅析3-服务发布流程
• rpcx服务框架浅析4-消费方服务引用
• rpcx服务框架浅析5-消费方(以zookeeper作为注册中心的Consumer例子)
• rpcx服务框架浅析6-注册中心
• rpcx服务框架浅析7-负载均衡策略
• rpcx服务框架浅析8-群集容错机制
• rpcx服务框架浅析9-序列化
• rpcx服务框架浅析10-扩展Plugin
• rpcx服务框架浅析11-调用拦截链
• rpcx服务框架浅析12-断路器
• rpcx服务框架浅析13-远程调用