NAPI 方式的实现

Linux 内核协议栈中报文接收的设计思路：

NAPI接口和旧接口两者有一下相同点：

  （1）、对报文的处理都应该放在软中断中处理。

  （2）、两者都有存储报文的队列，NAPI的队列是由网卡来管理的，旧接口的队列是由内核管理的。

每个NAPI设备都有一个轮询函数来由软中断调用，来进行轮询处理报文。我们可以建立一个虚拟的NAPI设备，让她的轮询函数来轮询的处理旧接口的报文队列，这样NAPI和旧接口的软中断处理函数就可以共用一个。收包软中断处理函数只是进行调用相应NAPI的轮询函数进行处理，并不关心是虚拟NAPI还是非虚拟NAPI。具体轮询细节由相关NAPI的轮询函数自己实现。

如上所述，我们现在可以知道每个CPU上需要有如下几个元素：

1、一个报文的接收队列，由旧接口来使用。

2、一个虚拟的NAPI设备。

3、一个NAPI的链表，上面挂着有报文需要处理的NAPI设备。

Linux内核具体实现：

结构体定义如下

1 2 3 4 5 6

struct softnet_data { 　　struct sk_buff_head   input_pkt_queue; //旧接口的输入队列 　　struct list_head  poll_list; //有需要处理报文的NAPI设备 　　struct napi_struct    backlog;//虚拟的NAPI设备 backlog };

定义一个per_cpu变量 softnet_data

1	DECLARE_PER_CPU(struct softnet_data,softnet_data);

softnet_data 的初始化如下：

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

static int __init net_dev_init(void) {     int i, rc = -ENOMEM;     /*      *  Initialise the packet receive queues.      */     for_each_possible_cpu(i)     {         struct softnet_data *queue;         queue = &per_cpu(softnet_data, i);         /*初始化输入队列*/         skb_queue_head_init(&queue->input_pkt_queue);         queue->completion_queue = NULL;         /*初始化pool 链表头*/         INIT_LIST_HEAD(&queue->poll_list);         /*把backlog的轮询函数初始化为 process_backlog          该函数来处理传统接口使用的输入队列的报文*/         queue->backlog.poll = process_backlog;         /*backlog 轮询函数一次可以处理的报文上限个数*/         queue->backlog.weight = weight_p;         queue->backlog.gro_list = NULL;         queue->backlog.gro_count = 0;     } }

NAPI的数据结构：

每个NAPI需要如下几个元素：

1、一个轮询函数，来轮询处理报文。

2、需要有一个状态，来标识NAPI的调度状态，是正在被调度还是没有被调度。

3、每个NAPI的一次轮询能处理的报文个数应该有一个上限，不能无限制的处理下去，防止独占CPU资源导致其他进程被饿死。

4、每个NAPI应该和一个网络设备进行关联。

5、NAPI设计时考虑了多队列的情况。一个网络设备可以有多个报文队列，这样一个网络设备可以关联多个NAPI，每个NAPI只处理特定的队列的报文。这样在多核情况下，多个CPU核可以并行的进行报文的处理。一般专用的网络多核处理器存在这种情况。

6、每个NAPI在有报文的情况下应该挂到softnet_data的pool_list上去。

如上所述，NAPI的结构体定义如下

1 2 3 4 5 6 7 8 9 10

struct napi_struct {     struct list_head  poll_list;//挂到softnet_data的pool_list上     unsigned long  state;//NAPI的调度状态     int  weight;//一次轮询的最大处理报文数     int  (*poll)(struct napi_struct *, int);//轮询函数     struct net_device   *dev;//指向关联的网络设备     struct list_head  dev_list;//对应的网络设备上关联的NAPI链表节点         /*其他字段是gso功能用，这里先不讨论*/ };

NAPI的调度状态：

NAPI_STATE_SCHED设置时表示该NAPI有报文 需要接收。即把NAPI挂到softnet_data 时要设置该状态，处理完从softnet_data 上摘除该NAPI时要清除该状态。

一些NAPI的函数详解：

1、netif_napi_add（）,把网络设备net_device 和NAPI结构相绑定。

1 2 3 4 5 6 7 8 9 10 11 12 13 14

void netif_napi_add(struct net_device *dev,       struct napi_struct *napi,       int (*poll)(struct napi_struct *, int), int weight) {     INIT_LIST_HEAD(&napi->poll_list);     napi->poll = poll;     napi->weight = weight;     /*把NAPI加入到网络设备相关联的NAPI链表上去。*/     list_add(&napi->dev_list, &dev->napi_list);     napi->dev = dev;     /*绑定时设置NAPI是已调度状态，禁用该NAPI,以后手动的来清除该标识来使     能NAPI.*/     set_bit(NAPI_STATE_SCHED, &napi->state); }

2、使能和禁用NAPI：

1 2 3 4 5 6 7 8 9 10 11 12

static inline void napi_disable(struct napi_struct *n) {     /*先设置NAPI状态为DISABLE*/     set_bit(NAPI_STATE_DISABLE, &n->state);                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      /*循环的等待NAPI被调度完成,变成可用的，设置成SCHED状态*/     while (test_and_set_bit(NAPI_STATE_SCHED, &n->state))         msleep(1);                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     /*清除DISABLE状态*/     clear_bit(NAPI_STATE_DISABLE, &n->state); }

3、调度NAPI

1 2 3 4 5 6 7 8 9 10 11 12 13 14

void __napi_schedule(struct napi_struct *n) {     unsigned long flags;     /*链表操作必须在关闭本地中断的情况下操作，防止硬中断抢占*/     local_irq_save(flags);                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         /*把NAPI加入到本地CPU的softnet_data 的pool_list 链表上*/     list_add_tail(&n->poll_list,                   &__get_cpu_var(softnet_data).poll_list);     /*调度收包软中断*/     __raise_softirq_irqoff(NET_RX_SOFTIRQ);                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         local_irq_restore(flags); }

4、NAPI执行完毕，如果NAPI一次轮询处理完队列的所以报文，调用该函数。

1 2 3 4 5 6 7 8 9 10 11 12

void __napi_complete(struct napi_struct *n) {     BUG_ON(!test_bit(NAPI_STATE_SCHED, &n->state));     BUG_ON(n->gro_list);                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         /*把NAPI从softnet_data的pool_list上摘除下来*/     list_del(&n->poll_list);                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        /*使能NAPI，允许它下次可以被再次调度*/     smp_mb__before_clear_bit();     clear_bit(NAPI_STATE_SCHED, &n->state); }

一般网卡驱动的NAPI的步骤：

1、不同的网卡驱动都会为网络设备定义自己的结构体。有的是自己的结构体中即包括net_device和napi,有的是把自己的结构体放到net_device的priv部分。

2、实现NAPI的轮询函数 rx_pool(struct napi_struct *napi，int weigh);

int rx_pool(struct napi_struct *napi,int weigh)

{

   int rx_cnt = 0;

   struct sk_buff *skb;

   while(rx_cnt < weigh)

   {

       skb = netdev_alloc_skb();

       copy_skb_form_hw(skb);//把报文从硬件缓存中读到内存skb中*/

       rx_cnt ++;

       netif_receive_skb(skb);//送协议栈直接处理

   }

   if(rx_cnt < weigh)

   {

       /*处理报文小与允许处理最大个数，表示一次轮询处理完全部的报文。

       使能收包中断*/

       napi_complete(napi)

       enable_rx_irq();

   }

   return rx_cnt;

}

3、把网络设备跟NAPI相关联，netif_napi_add(netdev,napi,rx_pool,weigh)；

4、注册网卡的收包中断：例：

   request_irq(irq,&rx_irq_handle, 0, netdev->name, netdev);

在中断处理函数中

static irqreturn_t rx_irq_handle(int irq, void *data)

{

   struct net_device *netdev = data

   struct napi_struct *napi = netdev_priv(netdev)->napi;

   disable_rx_irq();

__napi_schedule(napi);

}