1. Brownout

    Thread Starter Well-Known Member

    Jan 10, 2012
    2,375
    998
    Hello,

    I'm trying out a PCIe demonstration design for Xilinx devices. I'll confugure the device on a development board and plug it into an open PCIe slot on a PC running Fedora. I was reading over the documentation, and in the summary of the demo design chapter, it says this:


    I have no earthly idea what those utilities might be. Has anyone any ideas?
     
  2. tshuck

    Well-Known Member

    Oct 18, 2012
    3,531
    675
    Page 397 talks about using LSPCI...this should let you see the configuration of the device, if I read that properly..
     
    Brownout likes this.
  3. thatoneguy

    AAC Fanatic!

    Feb 19, 2009
    6,357
    718
    Use lspci with the -v command.

    You will be able to see what your device is in the list, even though there isn't a driver attached to it. If there is a driver attached to it, it will be listed.

    If you don't have the interface driver for it, you'll need to write one as a loadable module.


    Code ( (Unknown Language)):
    1.  
    2. lspci -v
    3.  
    4. 02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 02)
    5.         Subsystem: Giga-byte Technology GA-EP45-DS5 Motherboard
    6.         Flags: bus master, fast devsel, latency 0, IRQ 1278
    7.         I/O ports at de00 [size=256]
    8.         Memory at fdfff000 (64-bit, prefetchable) [size=4K]
    9.         Memory at fdfe0000 (64-bit, prefetchable) [size=64K]
    10.         [virtual] Expansion ROM at fdf00000 [disabled] [size=64K]
    11.         Capabilities: [40] Power Management version 3
    12.         Capabilities: [50] MSI: Enable+ Count=1/2 Maskable- 64bit+
    13.         Capabilities: [70] Express Endpoint, MSI 01
    14.         Capabilities: [b0] MSI-X: Enable- Count=2 Masked-
    15.         Capabilities: [d0] Vital Product Data
    16.         Capabilities: [100] Advanced Error Reporting
    17.         Capabilities: [140] Virtual Channel
    18.         Capabilities: [160] Device Serial Number 12-34-56-78-12-34-56-78
    19.         Kernel driver in use: r8169
    20.         Kernel modules: r8169
    21.  
    22. 03:07.0 Ethernet controller: 3Com Corporation 3c905B 100BaseTX [Cyclone] (rev 64)
    23.         Subsystem: 3Com Corporation 3C905B Fast Etherlink XL 10/100
    24.         Flags: bus master, medium devsel, latency 32, IRQ 21
    25.         I/O ports at cf00 [size=128]
    26.         Memory at fdcff000 (32-bit, non-prefetchable) [size=128]
    27.         [virtual] Expansion ROM at fdb00000 [disabled] [size=128K]
    28.         Capabilities: [dc] Power Management version 1
    29.         Kernel driver in use: 3c59x
    30.         Kernel modules: 3c59x
    31.  
    32. [/size][/size][/size]
     
    Brownout likes this.
  4. Brownout

    Thread Starter Well-Known Member

    Jan 10, 2012
    2,375
    998
    I am so not a driver writer. We will have a driver for the target platform, but what I'm trying to do is just get a baseline system running on a PC for now. All I want is something to write and read to the attached memory of the example design, but I don't have the resources to create a driver. (end of sob story)
     
  5. thatoneguy

    AAC Fanatic!

    Feb 19, 2009
    6,357
    718
    If you are using a PCIe IP block, it should have come with code and a makefile to at least see it on the bus from the CPU's point of view.

    If you wrote the PCIe block rather than using a complete IP module, I'd suggest trying a complete IP module, though they can get into "ludicrous range" for pricing.
     
  6. Brownout

    Thread Starter Well-Known Member

    Jan 10, 2012
    2,375
    998
    It should. But it doesen't. My development kit is supplied with a software driver, but I am using a different design than the one delivered with the kit. I'm not sure the driver will work, but I will try it when I get the lab computer I've requested.
     
  7. Brownout

    Thread Starter Well-Known Member

    Jan 10, 2012
    2,375
    998
    I found this:

    Interesting. I'm not sure what it all means, but I'll figure it out.
     
  8. thatoneguy

    AAC Fanatic!

    Feb 19, 2009
    6,357
    718
    The format of the output of lscpi is:

    [[[[<domain>]:]<bus>]:][<slot>][.[<func>]]

    So you are seeing in the output pcibus:slot.function in the example I posted above.
     
  9. Brownout

    Thread Starter Well-Known Member

    Jan 10, 2012
    2,375
    998
    I use that information to find the resource in the directory hierarchy, then open the resource for basic reading and writing. I got everything except for the "Mmap" and "Unmap" but I'm sure I will find the info easily by gurggling it.
     
  10. thatoneguy

    AAC Fanatic!

    Feb 19, 2009
    6,357
    718
    Memory map your functions on the board, then free the system memory once the application is complete.
     
  11. Brownout

    Thread Starter Well-Known Member

    Jan 10, 2012
    2,375
    998
    Ah! So memory map creates an entry in system memory space for the device!? Ok, that makes sense.
     
  12. thatoneguy

    AAC Fanatic!

    Feb 19, 2009
    6,357
    718
    Memory mapped functions are the simplest to implement.

    In the end, you have a function and it is mapped to an address, and the output of that function is at another address. So you write to one address, read the result on another address. The OS and PCIe IP on the board handle the actual bus transfer. If the result is an output from the card, then that should change states, and no read is required. It depends on how you set up your hardware, and which IP block you are using.
     
  13. Brownout

    Thread Starter Well-Known Member

    Jan 10, 2012
    2,375
    998
    I'm still not getting the mmap stuff. Why couldn't I just do this:

    >lspci -vv
    >01:00.0 Memory controller: Xilinx Corporation Device 0505

    Code ( (Unknown Language)):
    1.  
    2. #include <studio.h>
    3. #include <stdlib.h>
    4. int main() {
    5. FILE *dev;
    6. int data;
    7. if((dev = open("/sys/bus/pci/devices/0000:01:00.0", "wb") == NULL) {
    8. printf("Failed to open device\n");
    9. return 1;
    10. }
    11. write(dev, 0xFF);
    12. close(dev);
    13. }
    14.  
     
  14. Brownout

    Thread Starter Well-Known Member

    Jan 10, 2012
    2,375
    998
    Ok, here's what I tried:

    Code ( (Unknown Language)):
    1.  
    2. #include <stdio.h>
    3. #include <stdlib.h>
    4. int main() {
    5. FILE * dev;
    6. int data;
    7. if(dev = open("sys/bus/pci/devices/0000:02:00.0", "w+b") == NULL) {
    8. printf("Failed to open device\n");
    9. return 1;
    10. }
    11. write(dev, 0xaa);
    12. printf("Data written to device 0xaa\n");
    13. data = read(dev);
    14. printf("Program Terminating with data %x\n", data);
    15. close(dev);
    16. return 0;
    17. }
    18.  
    Where lspci returned: 02:00.0 Xilinx...

    The program ran through, but shows the read data = 0 in the last printf() statement.
     
  15. thatoneguy

    AAC Fanatic!

    Feb 19, 2009
    6,357
    718
    On the FPGA, configure the PCIe block to simply loop any input back to the output.

    See if that gives a response.

    Then work from there.
     
  16. Brownout

    Thread Starter Well-Known Member

    Jan 10, 2012
    2,375
    998
    PCIe transaction model doesn't allow that. Reads are split transactions, initiator sends read request, completer sends results. I am required to buffer the data to be read back.

    BTW, I fixed the typo in the path in the above code, but stil doens't work. I'm getting to give up on this and try something different.
     
  17. Brownout

    Thread Starter Well-Known Member

    Jan 10, 2012
    2,375
    998
    I talked to a real software engineer, and he told me that mmap is absolutely required. I had omitted it because I still don't understand it. Guess I better wrap my head around it if I want to make this work.

    Does anyone have an example of mmap'ing a device?
     
  18. Brownout

    Thread Starter Well-Known Member

    Jan 10, 2012
    2,375
    998
    Ok, a little progress. First of all, the line:

    was wrong. "open" returns -1 when it fails, and this was failing, but I didn't know it. Once I corrected the line, I got an indication that the device failed to open. After awhile, I realized that I needed root priviliges to run this program, due to tyring to opoen a device. The device opened, but the program exited when I tried to read to it.

    So, considering what the software engineer told me, I replaced the read with this:

    Code ( (Unknown Language)):
    1.  
    2. #include <stdio.h>
    3. #include <stdlib.h>
    4. #include <sys/types.h>
    5. #include <sys/stat.h>
    6. #include <unistd.h>
    7. #include <fcntl.h>
    8. #include <sys/mman.h>
    9.  
    10. int main() {
    11. FILE * dev;
    12. int rdata = 0;
    13. int wdata = 0xaa;
    14. int objs;
    15. int *map;
    16. #define FILESIZE (1 * sizeof(int))
    17.  
    18. if(dev = open("/sys/bus/pci/devices/0000:02:00.0/resource0", O_RDWR | O_CREAT | O_TRUNC, (mode_t) 0600) == -1) {
    19. printf("Failed to open device\n");
    20. return 1;
    21. }
    22.  
    23. map = mmap(0, FILESIZE, PROT_READ | PROT_WRITE, MAP_SHARED, dev, 0);
    24. if(map == MAP_FAILED) {
    25. printf("Error mapping device\n");
    26. return 1;
    27. }
    28.  
    29. if(munmap(map, FILESIZE) == -1) {
    30. printf("Error un-mapping the file\n");
    31. }
    32.  
    33. close(dev);
    34. return 0;
    35. }
    36.  
    37.  
    But now I get the "Map Failed" when I run. Could it be that the OS won't allow me to map a device?
     
    Last edited: Apr 3, 2013
  19. Brownout

    Thread Starter Well-Known Member

    Jan 10, 2012
    2,375
    998
    Please see the above post again. This time, I have all my code posted. Same error.
     
  20. Brownout

    Thread Starter Well-Known Member

    Jan 10, 2012
    2,375
    998
    I think I've narrowed down the problem with mmap. When I open a device, Linux treats it as a file with size = 0. The mmap function fails if the size of mapping requested (the macro "FILESIZE" in this case) is larger than the file(device) being mapped. I don't now how to fix this, but I might have another method.

    Still, I'm open to any ideas about how to fix the file size. FYI, I tried to map with zero specified for size to match the device, but no dice.
     
Loading...