pci dma

hank you very much, Neal. This helped incredibly much. In a lot of the online resources everything is still referred to only as DMA so again apologies that I misunderstood something.



I guess I cannot tag this as answered, but hopefully this sentence marks it.
________________________________________
From: Neal Palmer [neal@dinigroup.com]
Sent: 20 August 2014 21:52
To: BRUNNER K.J.
Cc: pci-sig@dinigroup.com
Subject: Re: [pci-sig] Mutiple DMA streaming FPGAs

J,

   There really isn't a bus arbiter in PCIE.  You don't really tag data as
being DMA.  There really isn't bus master control of PCIE...

   Your device sends memory write packets that are addressed to the
motherboards DRAM area.  That is PCIE's definition of "DMA".

   Packets from multiple PCIE devices get mux'd together in the host
chipset/CPU and get routed to the DRAM.  So your bandwidth concerns really
end up being DRAM bandwidth concerns.
   but you are only doing gen2 4lanes.  That is DRAMATICALLY less bandwidth
than Gen3 16lanes that GPU cards can do now...  Probably won't be an
issue.

   Biggest problems are probably getting a motherboard that allows the
largest packet size (yes expensive server motherboards generally have
larger packet size and therefore higher bandwidth).   And getting the
operating system to allocate contiguous chunks of memory that are large
enough to make your hardware addressing simple.

-- Neal

On Wed, 20 Aug 2014, BRUNNER K.J. wrote:

> Dear Community,
>
>
>
> I am very for having to post this question, but after hours/days of
>
> searching I cannot find a definite answer,  hence I am hoping for
>
> someone with experience to answer my question.
>
>
>
> I work in real time diagnostic systems and want to use 2 ML605
>
> boards to stream a data stream into memory via the onboard PCIe
>
> link. This has to happen as fast as is somehow possible, i.e. at the
>
> theoretical maximum for the PCIe gen2 x4 speed (~1.6GB/s). As I
>
> understand this requires DMA streams into the hosts RAM, involving
>
> the FPGA to assume bus master control of the PCIe bus and tagging
>
> the data as streaming data. As far as I know even though multiple
>
> devices can try to assume bus master control at the same time, only
>
> one will actually receive this, which is determined by the arbiter. Here
>
> is my question:
>
>
>
> What happens if I do have 2 FPGAs which try to simultaneously stream
>
> using DMA (continuously with multiple consecutive requests)? Will this
>
> reduce(halfen) the bandwidth, as they can only stream if the other
>
> one is not, i.e. at best one sends then the other and then the frist one
>
> again? As far as I know the PCI DMA channels do not exist anymore on
>
> PCIe so multiple devices could at best be multiplexed by the arbiter, is
>
> this possible without bandwidth loss? How would this physically work,
>
> as the bus speed is set by the slowest device in the link, which is
>
> the gen 2 FPGA PCIe controller and thus the arbiter could not operate
>
> at twice the speed in order to multiplex between 2 devices?!?
>
>
> Although this does sound like an FPGA board question it literally is
>
> concerning the PCIexpress busses capabilies, which is why I am
>
> bothering you.
>
> I apologize for the long message and hope you can help me.
>
> If this is for some reason the wrong board to ask I also apologize
>
> and hope ou can lead me further to where I can get this answered.
>
>
> Thank you very much in advance,
>
>    J. Brunner
>

-- Neal Palmer

The Dini Group
7469 Draper Ave
La Jolla, CA 92037
(858) 454-3419 x16
(858) 454-1728 (Fax)

_______________________________________________
pci-sig mailing list
pci-sig@dinigroup.com
http://www.dinigroup.com/mailman/listinfo/pci-sig