DM8168的PCIE2.0使用笔记

1、概述

    参考：sprugx8b.pdf

         DM816x_AM389x_PCI_Express_Root_Complex_Driver_User_Guide.pdf

         DM816x_AM389x_PCI_Express_Endpoint_Boot_Driver_User_Guide.pdf

         TMS320dm8168.pdf

http://processors.wiki.ti.com/index.php/TI81XX_PSP_PCI_Express_Root_Complex_Driver_User_Guide#RC_Driver_Source_Files

    对其中有助于PCIe开发的细节进行提取，虽然文档总结费时间，但是对于经验扩散来说无可替代。

2、DM816x Features

列出几个关键的特征，应该在PCIe开发过程中牢记，部分没有列出参考<17.1.2  sprugx8b.pdf>

Ø  x2 Lanes

Ø  Maximum outbound payload size of 128bytes。对于TLP的负载设置应该用的到。

Ø  Maximum inbound payload size of 256bytes.

Ø  Maximum read request size of 256bytes.此参数在于EP的通信中很重要。

3、DM816x Feature Not Supported

Ø  不支持热插拔

Ø  不支持WAKE#

4、Memory Map & Registers Map

    1.5节，1.16节。sprugx8b.pdf

5、RC, EP模式选择寄存器

    17.1.4.1节sprugx8b.pdf，使用PCIe的前提是配置DM8168PCIe接口的工作模式，在这里我们配置成RC模式。其配置寄存器为：

baseaddr参考1.5.2.1节Table1-12 L4Standard Peripheral Memory Map, Control Module baseaddr0x4814000~0x4815FFFF. 结合Table1-207可知PCIe_CFG baseaddr为0x48140640, 参考1.16.1.3.16节设置PCIE_DEVTYPE为0b10，Root Complex(RC)operation.

6、REFCLK 100Mhz，误差+/-300ppm

    配置62005时钟的时候注意时钟误差范围不要超过300ppm，62005可以设置。

7、only one interface link

    根据17.1.5描述，PCIe子系统支持一个link，也就是说DM8168只能连接一个EP，所以DM8168不能当做switch使用了，而且lane 0必须优先使用，lane 1不能单独使用，x1 只能是lane0 ，x2 是lane0 lane1。

8、250Mhz Clock

   250Mhz is aclock for OCP interfaces of the subsystem, is present at SYSCLK5(17.2.1 sprugx8b.pdf), which in turn uses the 27-Mhz crystal input from the main pLL.

9、PCIe and OCP/Inter address translating

    只有两种类型的地址空间需要映射，分别是Configuration 和 Memory 地址空间。物理设备（physical device）的地址空间分为两部分，分别是Range0和Range1，Range0被PCI configuration任务使用，作为configuraton space。Range1用作memory映射。17.2.3

10、51000000-51003fff : pcie-regs

    51000000-51003fff: pcie-regs，此段地址空间保存8168PCIE register的值，如下图所示。

应该需要主控配置inbound translation bar，用mem_rdwr.out工具查看初始状态可以看出，内核并没有配置这段寄存器。如下图所示：

11、outbound memory映射失败的问题（针对xx卡）

8168内核扫描XX卡时内核打印：

ti81xx_pcie:PCI scan done.
pci0000:00:00.0: BAR 9: can't assign mem pref (size 0x20000000)
pci 0000:00:00.0: BAR 8: assigned [mem 0x20000000-0x206fffff]
pci 0000:00:00.0: BAR 7: assigned [io  0x40000000-0x40001fff]
pci 0000:01:00.0: BAR 9: can't assign mem pref (size 0x20000000)
pci 0000:01:00.0: BAR 8: assigned [mem 0x20000000-0x205fffff]
pci 0000:01:00.0: BAR 0: assigned [mem 0x20600000-0x2061ffff]
pci 0000:01:00.0: BAR 0: set to [mem 0x20600000-0x2061ffff] (PCI address[0x20600000-0x2061ffff])
pci 0000:01:00.0: BAR 7: assigned [io  0x40000000-0x40001fff]
pci 0000:02:05.0: BAR 9: can't assign mem pref (size 0x8000000)
pci 0000:02:06.0: BAR 9: can't assign mem pref (size 0x8000000)
pci 0000:02:08.0: BAR 9: can't assign mem pref (size 0x8000000)
pci 0000:02:09.0: BAR 9: can't assign mem pref (size 0x8000000)
pci 0000:02:05.0: BAR 8: assigned [mem 0x20000000-0x201fffff]
pci 0000:02:06.0: BAR 8: assigned [mem 0x20200000-0x202fffff]
pci 0000:02:08.0: BAR 8: assigned [mem 0x20300000-0x203fffff]
pci 0000:02:09.0: BAR 8: assigned [mem 0x20400000-0x205fffff]
pci 0000:02:05.0: BAR 7: assigned [io  0x40000000-0x40000fff]
pci 0000:02:09.0: BAR 7: assigned [io  0x40001000-0x40001fff]
pci 0000:02:04.0: PCI bridge to [bus 03-03]
pci 0000:02:04.0:   bridge window [io  disabled]
pci 0000:02:04.0:   bridge window [mem disabled]
pci 0000:02:04.0:   bridge window [mem pref disabled]
pci 0000:04:00.0: BAR 4: can't assign mem pref (size 0x4000000)
pci 0000:04:00.0: BAR 3: can't assign mem pref (size 0x2000000)
pci 0000:04:00.0: BAR 1: can't assign mem pref (size 0x1000000)
pci 0000:04:00.0: BAR 2: can't assign mem pref (size 0x1000000)
pci 0000:04:00.0: BAR 0: assigned [mem 0x20000000-0x20000fff]
pci 0000:04:00.0: BAR 0: set to [mem 0x20000000-0x20000fff] (PCI address[0x20000000-0x20000fff])
pci 0000:02:05.0: PCI bridge to [bus 04-04]
pci 0000:02:05.0:   bridge window [io  0x40000000-0x40000fff]
pci 0000:02:05.0:   bridge window [mem 0x20000000-0x201fffff]
pci 0000:02:05.0:   bridge window [mem pref disabled]
pci 0000:05:00.0: BAR 4: can't assign mem pref (size 0x4000000)
pci 0000:05:00.0: BAR 3: can't assign mem pref (size 0x2000000)
pci 0000:05:00.0: BAR 1: can't assign mem pref (size 0x1000000)
pci 0000:05:00.0: BAR 2: can't assign mem pref (size 0x1000000)
pci 0000:05:00.0: BAR 0: assigned [mem 0x20200000-0x20200fff]
pci 0000:05:00.0: BAR 0: set to [mem 0x20200000-0x20200fff] (PCI address[0x20200000-0x20200fff])
pci 0000:02:06.0: PCI bridge to [bus 05-05]
pci 0000:02:06.0:   bridge window [io  disabled]
pci 0000:02:06.0:   bridge window [mem 0x20200000-0x202fffff]
pci 0000:02:06.0:   bridge window [mem pref disabled]
pci 0000:06:00.0: BAR 4: can't assign mem pref (size 0x4000000)
pci 0000:06:00.0: BAR 3: can't assign mem pref (size 0x2000000)
pci 0000:06:00.0: BAR 1: can't assign mem pref (size 0x1000000)
pci 0000:06:00.0: BAR 2: can't assign mem pref (size 0x1000000)
pci 0000:06:00.0: BAR 0: assigned [mem 0x20300000-0x20300fff]
pci 0000:06:00.0: BAR 0: set to [mem 0x20300000-0x20300fff] (PCI address[0x20300000-0x20300fff])
pci 0000:02:08.0: PCI bridge to [bus 06-06]
pci 0000:02:08.0:   bridge window [io  disabled]
pci 0000:02:08.0:   bridge window [mem 0x20300000-0x203fffff]
pci 0000:02:08.0:   bridge window [mem pref disabled]
pci 0000:07:00.0: BAR 4: can't assign mem pref (size 0x4000000)
pci 0000:07:00.0: BAR 3: can't assign mem pref (size 0x2000000)
pci 0000:07:00.0: BAR 1: can't assign mem pref (size 0x1000000)
pci 0000:07:00.0: BAR 2: can't assign mem pref (size 0x1000000)
pci 0000:07:00.0: BAR 0: assigned [mem 0x20400000-0x20400fff]
pci 0000:07:00.0: BAR 0: set to [mem 0x20400000-0x20400fff] (PCI address[0x20400000-0x20400fff])
pci 0000:02:09.0: PCI bridge to [bus 07-07]
pci 0000:02:09.0:   bridge window [io  0x40001000-0x40001fff]
pci 0000:02:09.0:   bridge window [mem 0x20400000-0x205fffff]
pci 0000:02:09.0:   bridge window [mem pref disabled]
pci 0000:01:00.0: PCI bridge to [bus 02-07]
pci 0000:01:00.0:   bridge window [io  0x40000000-0x40001fff]
pci 0000:01:00.0:   bridge window [mem 0x20000000-0x205fffff]
pci 0000:01:00.0:   bridge window [mem pref disabled]
pci 0000:00:00.0: PCI bridge to [bus 01-07]
pci 0000:00:00.0:   bridge window [io  0x40000000-0x40001fff]
pci 0000:00:00.0:   bridge window [mem 0x20000000-0x206fffff]
pci 0000:00:00.0:   bridge window [mem pref disabled]
PCI: enabling device 0000:00:00.0 (0140 -> 0143)
pci 0000:00:00.0: BAR 9: assigned [mem 0x20700000-0x20afffff pref]
pci 0000:01:00.0: BAR 9: assigned [mem 0x20700000-0x20afffff 64bit pref]
pci 0000:02:05.0: BAR 9: assigned [mem 0x20700000-0x208fffff 64bit pref]
pci 0000:02:09.0: BAR 9: assigned [mem 0x20900000-0x20afffff 64bit pref]
bio: create slab <bio-0> at 0
gpio_vr gpio_vr: failed to get the gpios

 

从打印可以看出< pci 0000:00:00.0:BAR 9: can't assign mem pref (size 0x20000000)>有512M的空间没有分配成功。

在DM816x_AM389x_PCI_Express_Root_Complex_Driver_User_Guide.pdf中有说明：

也就是说内核初始分配空间256MB，而ti6678申请的空间溢出。这个问题可以通过增大outbound Memory解决，在：kernel/arch/arm/mach-omap2/devices.c中修改如下：

将其增加到1G（实际最大能到2G，先改成1G试试），从新编译内核烧写。打印如下：

ti81xx_pcie: PCI scan done.

pci 0000:00:00.0: BAR 9: assigned [mem0x20000000-0x3fffffff pref]

pci 0000:00:00.0: BAR 8: assigned [mem0x40000000-0x406fffff]

pci 0000:00:00.0: BAR 7: assigned [io  0x40000000-0x40001fff]

pci 0000:01:00.0: BAR 9: assigned [mem0x20000000-0x3fffffff pref]

pci 0000:01:00.0: BAR 8: assigned [mem0x40000000-0x405fffff]

pci 0000:01:00.0: BAR 0: assigned [mem0x40600000-0x4061ffff]

pci 0000:01:00.0: BAR 0: set to [mem0x40600000-0x4061ffff] (PCI address [0x40600000-0x4061ffff])

pci 0000:01:00.0: BAR 7: assigned [io  0x40000000-0x40001fff]

pci 0000:02:05.0: BAR 9: assigned [mem0x20000000-0x27ffffff pref]

pci 0000:02:06.0: BAR 9: assigned [mem0x28000000-0x2fffffff pref]

pci 0000:02:08.0: BAR 9: assigned [mem0x30000000-0x37ffffff pref]

pci 0000:02:09.0: BAR 9: assigned [mem0x38000000-0x3fffffff pref]

pci 0000:02:05.0: BAR 8: assigned [mem0x40000000-0x401fffff]

pci 0000:02:06.0: BAR 8: assigned [mem0x40200000-0x402fffff]

pci 0000:02:08.0: BAR 8: assigned [mem0x40300000-0x403fffff]

pci 0000:02:09.0: BAR 8: assigned [mem0x40400000-0x405fffff]

pci 0000:02:05.0: BAR 7: assigned [io  0x40000000-0x40000fff]

pci 0000:02:09.0: BAR 7: assigned [io  0x40001000-0x40001fff]

pci 0000:02:04.0: PCI bridge to [bus 03-03]

pci 0000:02:04.0:   bridge window [io  disabled]

pci 0000:02:04.0:   bridge window [mem disabled]

pci 0000:02:04.0:   bridge window [mem pref disabled]

pci 0000:04:00.0: BAR 4: assigned [mem 0x20000000-0x23ffffffpref]

pci 0000:04:00.0: BAR 4: set to [mem0x20000000-0x23ffffff pref] (PCI address [0x20000000-0x23ffffff])

pci 0000:04:00.0: BAR 3: assigned [mem0x24000000-0x25ffffff pref]

pci 0000:04:00.0: BAR 3: set to [mem0x24000000-0x25ffffff pref] (PCI address [0x24000000-0x25ffffff])

pci 0000:04:00.0: BAR 1: assigned [mem0x26000000-0x26ffffff pref]

pci 0000:04:00.0: BAR 1: set to [mem0x26000000-0x26ffffff pref] (PCI address [0x26000000-0x26ffffff])

pci 0000:04:00.0: BAR 2: assigned [mem 0x27000000-0x27ffffffpref]

pci 0000:04:00.0: BAR 2: set to [mem0x27000000-0x27ffffff pref] (PCI address [0x27000000-0x27ffffff])

pci 0000:04:00.0: BAR 0: assigned [mem0x40000000-0x40000fff]

pci 0000:04:00.0: BAR 0: set to [mem0x40000000-0x40000fff] (PCI address [0x40000000-0x40000fff])

pci 0000:02:05.0: PCI bridge to [bus 04-04]

pci 0000:02:05.0:   bridge window [io  0x40000000-0x40000fff]

pci 0000:02:05.0:   bridge window [mem 0x40000000-0x401fffff]

pci 0000:02:05.0:   bridge window [mem 0x20000000-0x27ffffffpref]

pci 0000:05:00.0: BAR 4: assigned [mem0x28000000-0x2bffffff pref]

pci 0000:05:00.0: BAR 4: set to [mem0x28000000-0x2bffffff pref] (PCI address [0x28000000-0x2bffffff])

pci 0000:05:00.0: BAR 3: assigned [mem0x2c000000-0x2dffffff pref]

pci 0000:05:00.0: BAR 3: set to [mem0x2c000000-0x2dffffff pref] (PCI address [0x2c000000-0x2dffffff])

pci 0000:05:00.0: BAR 1: assigned [mem0x2e000000-0x2effffff pref]

pci 0000:05:00.0: BAR 1: set to [mem0x2e000000-0x2effffff pref] (PCI address [0x2e000000-0x2effffff])

pci 0000:05:00.0: BAR 2: assigned [mem0x2f000000-0x2fffffff pref]

pci 0000:05:00.0: BAR 2: set to [mem0x2f000000-0x2fffffff pref] (PCI address [0x2f000000-0x2fffffff])

pci 0000:05:00.0: BAR 0: assigned [mem0x40200000-0x40200fff]

pci 0000:05:00.0: BAR 0: set to [mem0x40200000-0x40200fff] (PCI address [0x40200000-0x40200fff])

pci 0000:02:06.0: PCI bridge to [bus 05-05]

pci 0000:02:06.0:   bridge window [io  disabled]

pci 0000:02:06.0:   bridge window [mem 0x40200000-0x402fffff]

pci 0000:02:06.0:   bridge window [mem 0x28000000-0x2fffffffpref]

pci 0000:06:00.0: BAR 4: assigned [mem0x30000000-0x33ffffff pref]

pci 0000:06:00.0: BAR 4: set to [mem0x30000000-0x33ffffff pref] (PCI address [0x30000000-0x33ffffff])

pci 0000:06:00.0: BAR 3: assigned [mem0x34000000-0x35ffffff pref]

pci 0000:06:00.0: BAR 3: set to [mem0x34000000-0x35ffffff pref] (PCI address [0x34000000-0x35ffffff])

pci 0000:06:00.0: BAR 1: assigned [mem0x36000000-0x36ffffff pref]

pci 0000:06:00.0: BAR 1: set to [mem0x36000000-0x36ffffff pref] (PCI address [0x36000000-0x36ffffff])

pci 0000:06:00.0: BAR 2: assigned [mem0x37000000-0x37ffffff pref]

pci 0000:06:00.0: BAR 2: set to [mem0x37000000-0x37ffffff pref] (PCI address [0x37000000-0x37ffffff])

pci 0000:06:00.0: BAR 0: assigned [mem0x40300000-0x40300fff]

pci 0000:06:00.0: BAR 0: set to [mem0x40300000-0x40300fff] (PCI address [0x40300000-0x40300fff])

pci 0000:02:08.0: PCI bridge to [bus 06-06]

pci 0000:02:08.0:   bridge window [io  disabled]

pci 0000:02:08.0:   bridge window [mem 0x40300000-0x403fffff]

pci 0000:02:08.0:   bridge window [mem 0x30000000-0x37ffffffpref]

pci 0000:07:00.0: BAR 4: assigned [mem0x38000000-0x3bffffff pref]

pci 0000:07:00.0: BAR 4: set to [mem0x38000000-0x3bffffff pref] (PCI address [0x38000000-0x3bffffff])

pci 0000:07:00.0: BAR 3: assigned [mem0x3c000000-0x3dffffff pref]

pci 0000:07:00.0: BAR 3: set to [mem0x3c000000-0x3dffffff pref] (PCI address [0x3c000000-0x3dffffff])

pci 0000:07:00.0: BAR 1: assigned [mem0x3e000000-0x3effffff pref]

pci 0000:07:00.0: BAR 1: set to [mem0x3e000000-0x3effffff pref] (PCI address [0x3e000000-0x3effffff])

pci 0000:07:00.0: BAR 2: assigned [mem0x3f000000-0x3fffffff pref]

pci 0000:07:00.0: BAR 2: set to [mem0x3f000000-0x3fffffff pref] (PCI address [0x3f000000-0x3fffffff])

pci 0000:07:00.0: BAR 0: assigned [mem0x40400000-0x40400fff]

pci 0000:07:00.0: BAR 0: set to [mem0x40400000-0x40400fff] (PCI address [0x40400000-0x40400fff])

pci 0000:02:09.0: PCI bridge to [bus 07-07]

pci 0000:02:09.0:   bridge window [io  0x40001000-0x40001fff]

pci 0000:02:09.0:   bridge window [mem 0x40400000-0x405fffff]

pci 0000:02:09.0:   bridge window [mem 0x38000000-0x3fffffffpref]

pci 0000:01:00.0: PCI bridge to [bus 02-07]

pci 0000:01:00.0:   bridge window [io  0x40000000-0x40001fff]

pci 0000:01:00.0:   bridge window [mem 0x40000000-0x405fffff]

pci 0000:01:00.0:   bridge window [mem 0x20000000-0x3fffffffpref]

pci 0000:00:00.0: PCI bridge to [bus 01-07]

pci 0000:00:00.0:   bridge window [io  0x40000000-0x40001fff]

pci 0000:00:00.0:   bridge window [mem 0x40000000-0x406fffff]

pci 0000:00:00.0:   bridge window [mem 0x20000000-0x3fffffffpref]

PCI: enabling device 0000:00:00.0 (0140-> 0143)

 

资源分配成功。