question about DMA data transfer

[Vasilich2004]

Data transfer methods surprised me twice J

 

First time I was surprised when I wrote program and found that there are 3 data transfer choices DMA, Interrupts, and Programmed  I/O. Actually, there are 2 data transfer types Programmed  I/O and DMA. Also there are 2 ways how program will know that data are ready by polling and interrupt.

Finally I read a manual (for example, <a href="http://www.ni.com/pdf/manuals/371022j.pdf" target="_blank"> http://www.ni.com/pdf/manuals/371022j.pdf</a> page 10-4) and I could realized that

- Programmed &nbsp;I/O means ?polling and Programmed &nbsp;I/O?

- Interrupts means ?interrupt and Programmed &nbsp;I/O?

- DMA means ?interrupt and DMA?

It was interesting mixture.

&nbsp;

Next time I was surprised and am still surprising ?

Let me bring example. I need to measure 1 (only one) AI signal on 1 MHz rate. I take PCI-6251 (up to 1.25 MHz rate) and should be happy. I also have another parameter. I need to measure 50 ms or 50 kSamples. Any M-series PCI card has only 4 KSamples buffer. To work all 50 ms continuously the card should use algorithm, like

split the buffer on two parts 2 + 2 KSamples and work in schedule - when card?s ADC fills first 2 kSamples data transfer mechanism moves another 2 kSamples to PC memory. Of course, I use DMA data transfer method (it means ?interrupt and DMA?). But CPU will get interrupt each 2 ms and Windows is not real time system. So I should lose data time to time.

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; I got PCI-6251 card and tested. I changed NI example ?Acq&amp;Graph Voltage-Int Clk-Dig Start.vi? slightly

- put measurement in continuously loop

- set ?Samples per Channel? = 700 000

- set ?Rate (Hz)? = 1 000 000

It worked and CPU was loaded not so much.

I started another program and loaded CPU. In this test CPU was loaded constantly from 70% to 100%. Again I didn?t notice then system missed data. Why? Has anyone idea? (Unfortunately, I have Core DUO CPU (= 2 CPUs) and the test maybe be wrong.)

&nbsp;

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Finally, there are 2 parameters in the card specification (<a href="http://www.ni.com/pdf/manuals/371291h.pdf" target="_blank"> http://www.ni.com/pdf/manuals/371291h.pdf</a>) . One parameter is that DMA is ?scatter- gather? type (I know that). Another parameter is ?scan list memory = 4095 entries? (It seems I know it also but don?t see how it can be used by the card). Looking on the parameters a stupid idea came to my mind. Maybe driver (using CPU) allocates memory in PC (can be not continuous) and starts measurement. Then the card measures, transfers data (as PCI DMA master, so CPU is not involved) to PC memory according list, and sets interrupt when measurement was finished and data was transferred.

&nbsp;

It would be interesting if someone from NI explains how really works the card and driver. Thanks.

[S_Hong]

Hi Vasilich2004,
&nbsp;
If you set up your continuous acquisition properly. you should not be missing any data. Whenever you create a DAQmx task and use the Timing function/VI to set finite or continuous acquisition/generation, DAQmx creates a buffer in computer memory for acquired or to-be-generated samples.&nbsp;When you start your task, your settings are then configured in hardware and your DAQ device&nbsp;will be acquiring/generating data until the task is complete or a stop command is sent. For hardware timed acquisition/generation, since everything will be running off of the clock, there is no need to worry about skipping data&nbsp;as&nbsp;all this is taken care of by our drivers. For example, in data acquisition, the hardware will automatically store the data in a FIFO to be then transfered via the specified data transfer mechanism (DMA (default), IRQ,&nbsp; Programmed I/O) to your buffer (computer memory). Now once the DAQmx read function is called, you are effectively transfer your samples to your application. The only thing to watch out for when configuring your application is to ensure that data is read out of the buffer fast enough before the data is overwritten.
&nbsp;
I hope this answers your question,

[Vasilich2004]

Hi S_Hong,
&nbsp;
&gt;&gt;The only thing to watch out for when configuring your application is to ensure that data is read out of the buffer fast enough before the data is overwritten.
&nbsp;
Actually, that was my question. Why the buffer wasn't overwritten if CPU loading in my test was very high (70-100%)?
&nbsp;

[S_Hong]

Hi Vasilich2004,
&nbsp;
Buffer overflow errors will depends on many factors such as your buffer size, your data acquisition rate, processor speed, etc... It may be possible that even though your CPU may be working at 100% that your settings are such that there will never be an overflow of data. Another possibility is that there&nbsp;is a memory leak (memory is being written faster than being read out) and it's just a matter of time before you get a buffer overflow error. Have you tried waiting longer to see if you get this error?&nbsp;

[Vasilich2004]

Hi S_Hong,
&nbsp;
I am slightly doubt ...
&nbsp;
&gt;&gt; Buffer overflow errors will depends on many factors such as your buffer size, your data acquisition rate, processor speed, etc... It may be possible that even though your
&gt;&gt; CPU may be working at 100% that your settings are such that there will never be an overflow of data. Another possibility is that there&nbsp;is a memory leak (memory is being
&gt;&gt; written faster than being read out) and it's just a matter of time before you get a buffer overflow error. Have you tried waiting longer to see if you get this error?&nbsp;
- Buffer size in PCI card is defined. According manual, any M-series card has 4095 KSamples.
- Data acquisition rate is 1 MHz. So if CPU is DMA transfer master then CPU will get Interrupt less than each&nbsp; 4 ms ( = 4095 kSamples / 1MHz Samples/sec) if CPU is DMA master of data transfer.
- Processor speed, of course, is important. But it is important for overall performance. If system has ~100% CPU loading then there is enough propability to miss the interrupt, (Practically, CPU is not loaded so much in DMA data transfer. CPU gets interrupt and does DMA initialization.&nbsp;Data is transfered without CPU.&nbsp;When DAM trasfer finishes then CPU gets another interrupt. It takes very-very small amount of CPU time. So main problem is that CPU doesn't miss (or pospone &gt; 4 ms) first interrupt..)
- "It may be possible that even though your &gt;&gt; CPU may be working at 100% that your settings are such that there will never be an overflow of data."&nbsp; I did test with another (not NI card) and 85% CPU (in Windows OS) loading was critical.
- memory leak means another&nbsp;problem
-&nbsp;where did you suppose to get "overflow" error? According manual it has place in the PCI card and program can't wait longer there. If you thought about delay in the program between loops then nothing wouldn't happen because PCI card will miss external trigger that start acquisition.
&nbsp;
So I still have question. What is DMA master on PCI bus? PCI card or CPU? Can you point NI developers on this topic?
Then
If PCI card&nbsp;is DMA master than ...

If CPU is DMA master than ...
:smileywink:

[Brad K]

Hi Vasilich2004,When you do an acquisition with the PCI-6251 and you have AI.DataXferMech set to DMA, the PCI-6251 is the bus master for transferring AI data. The AI engine on the PCI-6251 writes samples into the 4095 sample input FIFO, and the DMA engine on the PCI-6251 reads them from the input FIFO and writes them to a buffer in system memory. This buffer can be much larger than the input FIFO, so the PCI-6251's DMA engine doesn't have to interrupt the CPU as often as you described.Running a process that consumes 100% CPU does not cause interrupts to be missed because interrupts have a higher priority than user processes. However, if your application doesn't read data from the buffer in system memory as quickly as the PCI-6251's DMA engine fills the buffer, the buffer will fill up and the AI task will return an overflow error¹.You also asked about the "scan list memory" specification. Each virtual channel in an AI task requires an entry in the scan list memory, so this limits the number of channels you can add to the task². Note that your scan list can contain repeated channels such as "Dev1/ai0:31,Dev1/ai0:31,Dev1/ai0:31".Brad¹ Unless you configure the task to allow overwriting the contents of the buffer² Ignoring the "DIV" counter that is sometimes used with SCXI to exceed the maximum number of scan list entries on E Series

[Vasilich2004]

Thank you, Brad
I got correct answer that confirmed my supposition and explained my question.
It was so diffucult ... :smileywink: