分布式加锁

参考：https://www.jianshu.com/p/6d1fd5d8fe0a

参考：https://www.cnblogs.com/harry335/p/5473480.html

参考：https://blog.youkuaiyun.com/qq_35349114/article/details/84027745

 

分布式锁：排他性、避免死锁、高可用

实现方式为：数据库实现、基于Redis实现、基于ZooKeeper实现。

 

一、数据库实现

1、基于数据库的乐观锁实现

乐观锁机制其实就是在数据库表中引入一个版本号（version）字段来实现的。读取和更新

2、基于数据库的悲观锁实现

悲观锁也叫作排它锁，在Mysql中是基于 for update 来实现加锁的。通过 for update 操作，数据库在查询的时候就会给正在查询的记录加上排它锁。
当某条记录加上排它锁之后，其它线程是无法操作这条记录的。
如此，我们就可以认为获得了排它锁的这个线程是拥有了分布式锁，然后就可以执行我们想要做的业务逻辑，当逻辑完成之后，再释放锁即可。

SQL语句加锁：

 //使用数据库的悲观锁for update
            String sql = "select value from t_table_id where table_name=? for update";

 

for update是在数据库中上锁用的，可以为数据库中的行上一个排它锁。当一个事务的操作未完成时候，其他事务可以读取但是不能写入或更新。

例子：
比如一张表三个字段 ， id（商品id）, name（商品名字） , count(数量） 
当商品抢购时候会显示剩余商品件数，如果并发量大的时候，商品自减的值可能不准确。所以当我们在一个事务中对count字段进行修改的时候，其他事务应该只能读取指定id的count,而不能进行update等操作。这个时候就需要用到for update.

sql语句：
start transaction ; 
select * from table_name where id =1 for update ;

update table_name set count = count - 1 where id= 1;

此时如果另一个事务也想执行类似的操作：
start transaction ; 
select * from table_name where id =1 for update ; 
//下面的这行sql会等待,直到上面的事务回滚或者commit才得到执行。 
update table_name set count = count - 1 where id= 1;

*注：当选中某一个行的时候,如果是通过主键id选中的。那么这个时候是行级锁。 
其他的行还是可以直接insert 或者update的。如果是通过其他的方式选中行,或者选中的条件不明确包含主键。这个时候会锁表。其他的事务对该表的任意一行记录都无法进行插入或者更新操作。只能读取。*
 

采用悲观锁来实现同步
    在sql语句后加 for update就加上了锁，在查询的时候进行加锁，在加锁后不能进行查询。提交时候后其他人才能查询。
    public static int generate(String tableName){
            //使用数据库的悲观锁for update
            String sql = "select value from t_table_id where table_name=? for update";
            Connection conn = null;
            PreparedStatement pstmt = null;
            ResultSet rs = null;
            int value = 0;
            try{
                conn = DbUtil.getConnection();
                //设置自动提交为false
                DbUtil.beginTransaction(conn);
                pstmt = conn.prepareStatement(sql);
                pstmt.setString(1, tableName);
                rs = pstmt.executeQuery();
                rs.next();
    //            if(!rs.next()){
    //                throw new RuntimeException();
    //            }
                value = rs.getInt("value");
                value++;
                modifyValueField(conn,tableName,value);
                //提交事务
                DbUtil.commitTransaction(conn);
            }catch(Exception e){
                e.printStackTrace();
                //回滚事务
                DbUtil.rollbackTranscation(conn);
                throw new RuntimeException();
            }finally{
                DbUtil.close(rs);
                DbUtil.close(pstmt);
                DbUtil.resetConnection(conn);
                DbUtil.close(conn);
            }
            return value;
        }

 

 

二、基于Redis实现

基于Redis实现的锁机制，主要是依赖Redis自身的原子操作：

SET user_key user_value MX PX 100

NX：只在在键不存在时，才对键进行设置操作，SET key value NX 效果等同于 SETNX key value。
PX millisecond：设置键的过期时间为millisecond毫秒，当超过这个时间后，设置的键会自动失效。

当redis中不存在user_key这个键的时候，才会去设置一个user_key键，并且给这个键的值设置为 user_value，且这个键的存活时间为100ms。

解锁很简单，只需要删除这个key就可以了，不过删除之前需要判断，这个key对应的value是当初自己设置的那个。
另外，针对Redis集群模式的分布式锁，可以采用Redis的Redlock机制。

 

/**
 * @author http://blog.youkuaiyun.com/java2000_wl
 * @version <b>1.0.0</b>
 */
public class RedisBillLockHandler implements IBatchBillLockHandler {

    private static final Logger LOGGER = LoggerFactory.getLogger(RedisBillLockHandler.class);

    private static final int DEFAULT_SINGLE_EXPIRE_TIME = 3;
    
    private static final int DEFAULT_BATCH_EXPIRE_TIME = 6;

    private final JedisPool jedisPool;
    
    /**
     * 构造
     * @author http://blog.youkuaiyun.com/java2000_wl
     */
    public RedisBillLockHandler(JedisPool jedisPool) {
        this.jedisPool = jedisPool;
    }

    /**
     * 获取锁  如果锁可用   立即返回true，  否则返回false
     * @author http://blog.youkuaiyun.com/java2000_wl
     * @see com.fx.platform.components.lock.IBillLockHandler#tryLock(com.fx.platform.components.lock.IBillIdentify)
     * @param billIdentify
     * @return
     */
    public boolean tryLock(IBillIdentify billIdentify) {
        return tryLock(billIdentify, 0L, null);
    }

    /**
     * 锁在给定的等待时间内空闲，则获取锁成功 返回true， 否则返回false
     * @author http://blog.youkuaiyun.com/java2000_wl
     * @see com.fx.platform.components.lock.IBillLockHandler#tryLock(com.fx.platform.components.lock.IBillIdentify,
     *      long, java.util.concurrent.TimeUnit)
     * @param billIdentify
     * @param timeout
     * @param unit
     * @return
     */
    public boolean tryLock(IBillIdentify billIdentify, long timeout, TimeUnit unit) {
        String key = (String) billIdentify.uniqueIdentify();
        Jedis jedis = null;
        try {
            jedis = getResource();
            long nano = System.nanoTime();
            do {
                LOGGER.debug("try lock key: " + key);
                Long i = jedis.setnx(key, key);
                if (i == 1) { 
                    jedis.expire(key, DEFAULT_SINGLE_EXPIRE_TIME);
                    LOGGER.debug("get lock, key: " + key + " , expire in " + DEFAULT_SINGLE_EXPIRE_TIME + " seconds.");
                    return Boolean.TRUE;
                } else { // 存在锁
                    if (LOGGER.isDebugEnabled()) {
                        String desc = jedis.get(key);
                        LOGGER.debug("key: " + key + " locked by another business：" + desc);
                    }
                }
                if (timeout == 0) {
                    break;
                }
                Thread.sleep(300);
            } while ((System.nanoTime() - nano) < unit.toNanos(timeout));
            return Boolean.FALSE;
        } catch (JedisConnectionException je) {
            LOGGER.error(je.getMessage(), je);
            returnBrokenResource(jedis);
        } catch (Exception e) {
            LOGGER.error(e.getMessage(), e);
        } finally {
            returnResource(jedis);
        }
        return Boolean.FALSE;
    }

    /**
     * 如果锁空闲立即返回   获取失败 一直等待
     * @author http://blog.youkuaiyun.com/java2000_wl
     * @see com.fx.platform.components.lock.IBillLockHandler#lock(com.fx.platform.components.lock.IBillIdentify)
     * @param billIdentify
     */
    public void lock(IBillIdentify billIdentify) {
        String key = (String) billIdentify.uniqueIdentify();
        Jedis jedis = null;
        try {
            jedis = getResource();
            do {
                LOGGER.debug("lock key: " + key);
                Long i = jedis.setnx(key, key);
                if (i == 1) { 
                    jedis.expire(key, DEFAULT_SINGLE_EXPIRE_TIME);
                    LOGGER.debug("get lock, key: " + key + " , expire in " + DEFAULT_SINGLE_EXPIRE_TIME + " seconds.");
                    return;
                } else {
                    if (LOGGER.isDebugEnabled()) {
                        String desc = jedis.get(key);
                        LOGGER.debug("key: " + key + " locked by another business：" + desc);
                    }
                }
                Thread.sleep(300); 
            } while (true);
        } catch (JedisConnectionException je) {
            LOGGER.error(je.getMessage(), je);
            returnBrokenResource(jedis);
        } catch (Exception e) {
            LOGGER.error(e.getMessage(), e);
        } finally {
            returnResource(jedis);
        }
    }

    /**
     * 释放锁
     * @author http://blog.youkuaiyun.com/java2000_wl
     * @see com.fx.platform.components.lock.IBillLockHandler#unLock(com.fx.platform.components.lock.IBillIdentify)
     * @param billIdentify
     */
    public void unLock(IBillIdentify billIdentify) {
        List<IBillIdentify> list = new ArrayList<IBillIdentify>();
        list.add(billIdentify);
        unLock(list);
    }

    /**
     * 批量获取锁  如果全部获取   立即返回true, 部分获取失败 返回false
     * @author http://blog.youkuaiyun.com/java2000_wl
     * @date 2013-7-22 下午10:27:44
     * @see com.fx.platform.components.lock.IBatchBillLockHandler#tryLock(java.util.List)
     * @param billIdentifyList
     * @return
     */
    public boolean tryLock(List<IBillIdentify> billIdentifyList) {
        return tryLock(billIdentifyList, 0L, null);
    }
    
    /**
     * 锁在给定的等待时间内空闲，则获取锁成功 返回true， 否则返回false
     * @author http://blog.youkuaiyun.com/java2000_wl
     * @param billIdentifyList
     * @param timeout
     * @param unit
     * @return
     */
    public boolean tryLock(List<IBillIdentify> billIdentifyList, long timeout, TimeUnit unit) {
        Jedis jedis = null;
        try {
            List<String> needLocking = new CopyOnWriteArrayList<String>();    
            List<String> locked = new CopyOnWriteArrayList<String>();    
            jedis = getResource();
            long nano = System.nanoTime();
            do {
                // 构建pipeline，批量提交
                Pipeline pipeline = jedis.pipelined();
                for (IBillIdentify identify : billIdentifyList) {
                    String key = (String) identify.uniqueIdentify();
                    needLocking.add(key);
                    pipeline.setnx(key, key);
                }
                LOGGER.debug("try lock keys: " + needLocking);
                // 提交redis执行计数
                List<Object> results = pipeline.syncAndReturnAll();
                for (int i = 0; i < results.size(); ++i) {
                    Long result = (Long) results.get(i);
                    String key = needLocking.get(i);
                    if (result == 1) {    // setnx成功，获得锁
                        jedis.expire(key, DEFAULT_BATCH_EXPIRE_TIME);
                        locked.add(key);
                    } 
                }
                needLocking.removeAll(locked);    // 已锁定资源去除
                
                if (CollectionUtils.isEmpty(needLocking)) {
                    return true;
                } else {    
                    // 部分资源未能锁住
                    LOGGER.debug("keys: " + needLocking + " locked by another business：");
                }
                
                if (timeout == 0) {    
                    break;
                }
                Thread.sleep(500);    
            } while ((System.nanoTime() - nano) < unit.toNanos(timeout));

            // 得不到锁，释放锁定的部分对象，并返回失败
            if (!CollectionUtils.isEmpty(locked)) {
                jedis.del(locked.toArray(new String[0]));
            }
            return false;
        } catch (JedisConnectionException je) {
            LOGGER.error(je.getMessage(), je);
            returnBrokenResource(jedis);
        } catch (Exception e) {
            LOGGER.error(e.getMessage(), e);
        } finally {
            returnResource(jedis);
        }
        return true;
    }

    /**
     * 批量释放锁
     * @author http://blog.youkuaiyun.com/java2000_wl
     * @see com.fx.platform.components.lock.IBatchBillLockHandler#unLock(java.util.List)
     * @param billIdentifyList
     */
    public void unLock(List<IBillIdentify> billIdentifyList) {
        List<String> keys = new CopyOnWriteArrayList<String>();
        for (IBillIdentify identify : billIdentifyList) {
            String key = (String) identify.uniqueIdentify();
            keys.add(key);
        }
        Jedis jedis = null;
        try {
            jedis = getResource();
            jedis.del(keys.toArray(new String[0]));
            LOGGER.debug("release lock, keys :" + keys);
        } catch (JedisConnectionException je) {
            LOGGER.error(je.getMessage(), je);
            returnBrokenResource(jedis);
        } catch (Exception e) {
            LOGGER.error(e.getMessage(), e);
        } finally {
            returnResource(jedis);
        }
    }
    
    /**
     * @author http://blog.youkuaiyun.com/java2000_wl
     * @date 2013-7-22 下午9:33:45
     * @return
     */
    private Jedis getResource() {
        return jedisPool.getResource();
    }
    
    /**
     * 销毁连接
     * @author http://blog.youkuaiyun.com/java2000_wl
     * @param jedis
     */
    private void returnBrokenResource(Jedis jedis) {
        if (jedis == null) {
            return;
        }
        try {
            //容错
            jedisPool.returnBrokenResource(jedis);
        } catch (Exception e) {
            LOGGER.error(e.getMessage(), e);
        }
    }
    
    /**
     * @author http://blog.youkuaiyun.com/java2000_wl
     * @param jedis
     */
    private void returnResource(Jedis jedis) {
        if (jedis == null) {
            return;
        }
        try {
            jedisPool.returnResource(jedis);
        } catch (Exception e) {
            LOGGER.error(e.getMessage(), e);
        }
    }

三、基于Zookeeper实现

基于ZooKeeper，就是使用它的临时有序节点来实现的分布式锁。
其原理是当某客户端要进行逻辑的加锁时，就在zookeeper上的某个指定节点的目录下，去生成一个唯一的临时有序节点， 然后判断自己是否是这些有序节点中序号最小的一个，如果是，则算是获取了锁。如果不是，则说明没有获取到锁，那么就需要在序列中找到比自己小的那个节点，并对其调用exist()方法，对其注册事件监听，当监听到这个节点被删除了，那就再去判断一次自己当初创建的节点是否变成了序列中最小的。如果是，则获取锁，如果不是，则重复上述步骤，当释放锁的时候，只需将这个临时节点删除即可。

 

使用zookeeper实现的分布式锁

分布式锁，实现了Lock接口


复制代码 代码如下:


package com.concurrent;

import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.locks.Condition;
import java.util.concurrent.locks.Lock;

import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.KeeperException;
import org.apache.zookeeper.WatchedEvent;
import org.apache.zookeeper.Watcher;
import org.apache.zookeeper.ZooDefs;
import org.apache.zookeeper.ZooKeeper;
import org.apache.zookeeper.data.Stat;

/**
   DistributedLock lock = null;
 try {
  lock = new DistributedLock("127.0.0.1:2182","test");
  lock.lock();
  //do something...
 } catch (Exception e) {
  e.printStackTrace();
 } 
 finally {
  if(lock != null)
   lock.unlock();
 }
 * @author xueliang
 *
 */
public class DistributedLock implements Lock, Watcher{
 private ZooKeeper zk;
 private String root = "/locks";//根
 private String lockName;//竞争资源的标志
 private String waitNode;//等待前一个锁
 private String myZnode;//当前锁
 private CountDownLatch latch;//计数器
 private int sessionTimeout = 30000;
 private List<Exception> exception = new ArrayList<Exception>();

 /**
  * 创建分布式锁,使用前请确认config配置的zookeeper服务可用
  * @param config 127.0.0.1:2181
  * @param lockName 竞争资源标志,lockName中不能包含单词lock
  */
 public DistributedLock(String config, String lockName){
  this.lockName = lockName;
  // 创建一个与服务器的连接
   try {
   zk = new ZooKeeper(config, sessionTimeout, this);
   Stat stat = zk.exists(root, false);
   if(stat == null){
    // 创建根节点
    zk.create(root, new byte[0], ZooDefs.Ids.OPEN_ACL_UNSAFE,CreateMode.PERSISTENT); 
   }
  } catch (IOException e) {
   exception.add(e);
  } catch (KeeperException e) {
   exception.add(e);
  } catch (InterruptedException e) {
   exception.add(e);
  }
 }

 /**
  * zookeeper节点的监视器
  */
 public void process(WatchedEvent event) {
  if(this.latch != null) {  
            this.latch.countDown();  
        }
 }

 public void lock() {
  if(exception.size() > 0){
   throw new LockException(exception.get(0));
  }
  try {
   if(this.tryLock()){
    System.out.println("Thread " + Thread.currentThread().getId() + " " +myZnode + " get lock true");
    return;
   }
   else{
    waitForLock(waitNode, sessionTimeout);//等待锁
   }
  } catch (KeeperException e) {
   throw new LockException(e);
  } catch (InterruptedException e) {
   throw new LockException(e);
  } 
 }

 public boolean tryLock() {
  try {
   String splitStr = "_lock_";
   if(lockName.contains(splitStr))
    throw new LockException("lockName can not contains \\u000B");
   //创建临时子节点
   myZnode = zk.create(root + "/" + lockName + splitStr, new byte[0], ZooDefs.Ids.OPEN_ACL_UNSAFE,CreateMode.EPHEMERAL_SEQUENTIAL);
   System.out.println(myZnode + " is created ");
   //取出所有子节点
   List<String> subNodes = zk.getChildren(root, false);
   //取出所有lockName的锁
   List<String> lockObjNodes = new ArrayList<String>();
   for (String node : subNodes) {
    String _node = node.split(splitStr)[0];
    if(_node.equals(lockName)){
     lockObjNodes.add(node);
    }
   }
   Collections.sort(lockObjNodes);
   System.out.println(myZnode + "==" + lockObjNodes.get(0));
   if(myZnode.equals(root+"/"+lockObjNodes.get(0))){
    //如果是最小的节点,则表示取得锁
             return true;
         }
   //如果不是最小的节点，找到比自己小1的节点
   String subMyZnode = myZnode.substring(myZnode.lastIndexOf("/") + 1);
   waitNode = lockObjNodes.get(Collections.binarySearch(lockObjNodes, subMyZnode) - 1);
  } catch (KeeperException e) {
   throw new LockException(e);
  } catch (InterruptedException e) {
   throw new LockException(e);
  }
  return false;
 }

 public boolean tryLock(long time, TimeUnit unit) {
  try {
   if(this.tryLock()){
    return true;
   }
         return waitForLock(waitNode,time);
  } catch (Exception e) {
   e.printStackTrace();
  }
  return false;
 }

 private boolean waitForLock(String lower, long waitTime) throws InterruptedException, KeeperException {
        Stat stat = zk.exists(root + "/" + lower,true);
        //判断比自己小一个数的节点是否存在,如果不存在则无需等待锁,同时注册监听
        if(stat != null){
         System.out.println("Thread " + Thread.currentThread().getId() + " waiting for " + root + "/" + lower);
         this.latch = new CountDownLatch(1);
         this.latch.await(waitTime, TimeUnit.MILLISECONDS);
         this.latch = null;
        }
        return true;
    }

 public void unlock() {
  try {
   System.out.println("unlock " + myZnode);
   zk.delete(myZnode,-1);
   myZnode = null;
   zk.close();
  } catch (InterruptedException e) {
   e.printStackTrace();
  } catch (KeeperException e) {
   e.printStackTrace();
  }
 }

 public void lockInterruptibly() throws InterruptedException {
  this.lock();
 }

 public Condition newCondition() {
  return null;
 }

 public class LockException extends RuntimeException {
  private static final long serialVersionUID = 1L;
  public LockException(String e){
   super(e);
  }
  public LockException(Exception e){
   super(e);
  }
 }

}



并发测试工具


复制代码 代码如下:


package com.concurrent;

import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.concurrent.CopyOnWriteArrayList;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.atomic.AtomicInteger;

/**
  ConcurrentTask[] task = new ConcurrentTask[5];
  for(int i=0;i<task.length;i++){
      task[i] = new ConcurrentTask(){
    public void run() {
     System.out.println("==============");

    }};
  }
  new ConcurrentTest(task);
 * @author xueliang
 *
 */
public class ConcurrentTest {
 private CountDownLatch startSignal = new CountDownLatch(1);//开始阀门
 private CountDownLatch doneSignal = null;//结束阀门
 private CopyOnWriteArrayList<Long> list = new CopyOnWriteArrayList<Long>();
 private AtomicInteger err = new AtomicInteger();//原子递增
 private ConcurrentTask[] task = null;

 public ConcurrentTest(ConcurrentTask... task){
  this.task = task;
  if(task == null){
   System.out.println("task can not null");
   System.exit(1);
  }
  doneSignal = new CountDownLatch(task.length);
  start();
 }
 /**
  * @param args
  * @throws ClassNotFoundException 
  */
 private void start(){
  //创建线程，并将所有线程等待在阀门处
  createThread();
  //打开阀门
  startSignal.countDown();//递减锁存器的计数，如果计数到达零，则释放所有等待的线程
  try {
   doneSignal.await();//等待所有线程都执行完毕
  } catch (InterruptedException e) {
   e.printStackTrace();
  }
  //计算执行时间
  getExeTime();
 }
 /**
  * 初始化所有线程，并在阀门处等待
  */
 private void createThread() {
  long len = doneSignal.getCount();
  for (int i = 0; i < len; i++) {
   final int j = i;
   new Thread(new Runnable(){
    public void run() {
     try {
      startSignal.await();//使当前线程在锁存器倒计数至零之前一直等待
      long start = System.currentTimeMillis();
      task[j].run();
      long end = (System.currentTimeMillis() - start);
      list.add(end);
     } catch (Exception e) {
      err.getAndIncrement();//相当于err++
     } 
     doneSignal.countDown();
    }
   }).start();
  }
 }
 /**
  * 计算平均响应时间
  */
 private void getExeTime() {
  int size = list.size();
  List<Long> _list = new ArrayList<Long>(size);
  _list.addAll(list);
  Collections.sort(_list);
  long min = _list.get(0);
  long max = _list.get(size-1);
  long sum = 0L;
  for (Long t : _list) {
   sum += t;
  }
  long avg = sum/size;
  System.out.println("min: " + min);
  System.out.println("max: " + max);
  System.out.println("avg: " + avg);
  System.out.println("err: " + err.get());
 }

 public interface ConcurrentTask {
  void run();
 }

}



测试


复制代码 代码如下:


package com.concurrent;

import com.concurrent.ConcurrentTest.ConcurrentTask;

public class ZkTest {
 public static void main(String[] args) {
  Runnable task1 = new Runnable(){
   public void run() {
    DistributedLock lock = null;
    try {
     lock = new DistributedLock("127.0.0.1:2182","test1");
     //lock = new DistributedLock("127.0.0.1:2182","test2");
     lock.lock();
     Thread.sleep(3000);
     System.out.println("===Thread " + Thread.currentThread().getId() + " running");
    } catch (Exception e) {
     e.printStackTrace();
    } 
    finally {
     if(lock != null)
      lock.unlock();
    }

   }

  };
  new Thread(task1).start();
  try {
   Thread.sleep(1000);
  } catch (InterruptedException e1) {
   e1.printStackTrace();
  }
  ConcurrentTask[] tasks = new ConcurrentTask[60];
  for(int i=0;i<tasks.length;i++){
   ConcurrentTask task3 = new ConcurrentTask(){
    public void run() {
     DistributedLock lock = null;
     try {
      lock = new DistributedLock("127.0.0.1:2183","test2");
      lock.lock();
      System.out.println("Thread " + Thread.currentThread().getId() + " running");
     } catch (Exception e) {
      e.printStackTrace();
     } 
     finally {
      lock.unlock();
     }

    }
   };
   tasks[i] = task3;
  }
  new ConcurrentTest(tasks);
 }
}