How to have multiple devices transferring data on different COM ports

Thread Starter

surfline

Joined Aug 12, 2009
36
I have a Windows program that is collecting tons of data from 5 devices that are connected from Windows through USB, but using a virtual com port with FTDI chip on the device. So for example I have COM1, COM2, COM3, COM4, COM5 that I'm sending data too. I'm sending tons of data, test is running for 1-7 days with little pausing.

I have found that I'm getting BSOD / PC crashes, usually from the FTDI COM port driver, ftser2k.sys . I've updated the driver to the latest, and all other drives to the latest.

Originally, I was trying to do 10-15 devices, but that would never work. With 5 devices, I can get it to work occasionally. I'm also going through a USB hub.

Any other ideas to solve this instead of reducing number of devices testing in parallel? I would really like to be able to test at least 5 at a time.
 

MrSoftware

Joined Oct 29, 2013
1,936
It sounds like the FTDI driver has a bug. In theory, you should be able to stress the driver with tons of data and not get a blue screen. Step 1 would be contact FTDI and see what they recommend.

That said; try to understand the crash so you can possibly come up with a work around. What reason is reported for the crash? The information should still be visible in the system log, use the windows event viewer to find it. Look under the "Application" section, use ctl-f and search for "blue". For example:

upload_2019-2-14_9-17-21.png


Things you can try; make sure all of your COM ports are using the same FTDI chipset. Also make sure your USB hub provides plenty of power. If it runs out of power and a device unexpectedly disconnects due to lack of power, that could potentially cause a bluescreen. Make sure your hub is externally powered to help with this. There are a number of hubs out there, but this one has worked well for me in the past. Maybe switch to an add-on USB card as those can be better than the USB support on some motherboards.

Assuming it's not a hardware problem; are you dynamically allocating and freeing memory (new/delete)? Are you running multiple threads? Maybe you're handing a reference (pointer) to the driver, then deleting the memory before the driver is done with it. Maybe the FTDI driver doesn't support simultaneous operations to multiple COM ports (I don't know, maybe FTDI can answer this). Assuming the worst, try serializing your code. Use semaphores (critical sections) and make sure one operation is done before releasing any memory or moving to the next operation. For example, if you're receiving data from COM1, allow that operation to block all other COM port operations until the read is done. This could slow things down, but the first goal is to figure out what's going wrong and make it stable. Once it's reliable, then you can work on making it faster, if that's even necessary.
 

danadak

Joined Mar 10, 2018
4,057
1) Can you do the transfers via DMA ? This is best approach, basically a
HW approach. There are micros out there that can do this.

2) Are you doing the servicing in interrupt ? If so calling other f()'s in ISR ?
That's a bad practice as it causes a lot of stack push in the ISR, needless
stack push. Wastes MIPs.

3) Wherever possible use pointers for working with buffers for the UART channels.
That will save MIPs. Reduce latency.

4) Using circular buffer for each UART ?


Regards, Dana.
 

Picbuster

Joined Dec 2, 2013
1,022
I have a Windows program that is collecting tons of data from 5 devices that are connected from Windows through USB, but using a virtual com port with FTDI chip on the device. So for example I have COM1, COM2, COM3, COM4, COM5 that I'm sending data too. I'm sending tons of data, test is running for 1-7 days with little pausing.

I have found that I'm getting BSOD / PC crashes, usually from the FTDI COM port driver, ftser2k.sys . I've updated the driver to the latest, and all other drives to the latest.

Originally, I was trying to do 10-15 devices, but that would never work. With 5 devices, I can get it to work occasionally. I'm also going through a USB hub.

Any other ideas to solve this instead of reducing number of devices testing in parallel? I would really like to be able to test at least 5 at a time.
One of the things what is popping into my mind:
Virtual comport in MS create an overhead on the interrupt when baud rate ( usb virtual coms) is to high a MS stack overflow is possible.

I do not work with MS very often but there is a function allowing a crash dump.
Using that might highlight your problem.

I used the ftdi chip(s) a lot never had any problems upto 12 USB under linux and winxp\win7\8 but speed each 19200 baud.
Win10 however is not my favorite. Its showing all types of incompatibilities with existing applications including the microsoft ones.

At the other hand you could consider LINUX as your front end and run one TCP/IP to your windows application.

Picbuster
 

Thread Starter

surfline

Joined Aug 12, 2009
36
1) Can you do the transfers via DMA ? This is best approach, basically a
HW approach. There are micros out there that can do this.

2) Are you doing the servicing in interrupt ? If so calling other f()'s in ISR ?
That's a bad practice as it causes a lot of stack push in the ISR, needless
stack push. Wastes MIPs.

3) Wherever possible use pointers for working with buffers for the UART channels.
That will save MIPs. Reduce latency.

4) Using circular buffer for each UART ?


Regards, Dana.
I don't think its a memory issue. I've taken logs of the memory and CPU usage, and they are quite under utilized.

What is DMA? I'm using Windows C# program and creating a SerialPort() object, then using the serialPort.WriteLine(command) to send and then SerialPort.ReadExisting() to receive back
 

Thread Starter

surfline

Joined Aug 12, 2009
36
One of the things what is popping into my mind:
Virtual comport in MS create an overhead on the interrupt when baud rate ( usb virtual coms) is to high a MS stack overflow is possible.

I do not work with MS very often but there is a function allowing a crash dump.
Using that might highlight your problem.

I used the ftdi chip(s) a lot never had any problems upto 12 USB under linux and winxp\win7\8 but speed each 19200 baud.
Win10 however is not my favorite. Its showing all types of incompatibilities with existing applications including the microsoft ones.

At the other hand you could consider LINUX as your front end and run one TCP/IP to your windows application.

Picbuster
Thanks, can't do linux because i'm also writing some of the data to Excel.
 

Thread Starter

surfline

Joined Aug 12, 2009
36
It sounds like the FTDI driver has a bug. In theory, you should be able to stress the driver with tons of data and not get a blue screen. Step 1 would be contact FTDI and see what they recommend.

That said; try to understand the crash so you can possibly come up with a work around. What reason is reported for the crash? The information should still be visible in the system log, use the windows event viewer to find it. Look under the "Application" section, use ctl-f and search for "blue". For example:

View attachment 170174


Things you can try; make sure all of your COM ports are using the same FTDI chipset. Also make sure your USB hub provides plenty of power. If it runs out of power and a device unexpectedly disconnects due to lack of power, that could potentially cause a bluescreen. Make sure your hub is externally powered to help with this. There are a number of hubs out there, but this one has worked well for me in the past. Maybe switch to an add-on USB card as those can be better than the USB support on some motherboards.

Assuming it's not a hardware problem; are you dynamically allocating and freeing memory (new/delete)? Are you running multiple threads? Maybe you're handing a reference (pointer) to the driver, then deleting the memory before the driver is done with it. Maybe the FTDI driver doesn't support simultaneous operations to multiple COM ports (I don't know, maybe FTDI can answer this). Assuming the worst, try serializing your code. Use semaphores (critical sections) and make sure one operation is done before releasing any memory or moving to the next operation. For example, if you're receiving data from COM1, allow that operation to block all other COM port operations until the read is done. This could slow things down, but the first goal is to figure out what's going wrong and make it stable. Once it's reliable, then you can work on making it faster, if that's even necessary.
Thanks I'll try asking FTDI too
 

MrChips

Joined Oct 2, 2009
22,099
I would do it differently. I would use a single USB port and network all devices to the single USB-to-UART bridge. (I usually use Silicon Labs CP2102).

The master PC sends a data request with a device ID code attached.
The device with the corresponding ID code responds with its data. In this way you can have as many devices as desired only subject to bus loading limitations.
 

Thread Starter

surfline

Joined Aug 12, 2009
36
I would do it differently. I would use a single USB port and network all devices to the single USB-to-UART bridge. (I usually use Silicon Labs CP2102).

The master PC sends a data request with a device ID code attached.
The device with the corresponding ID code responds with its data. In this way you can have as many devices as desired only subject to bus loading limitations.
Interesting, so one USB channel can then go to numerous different devices in parallel through the bridge? Aren't you then bottle necked by the bridge?
Unfortunately, I'm too far into the project to make a major change like that
 

MrChips

Joined Oct 2, 2009
22,099
Interesting, so one USB channel can then go to numerous different devices in parallel through the bridge? Aren't you then bottle necked by the bridge?
Unfortunately, I'm too far into the project to make a major change like that
The PC prompts each device one at a time. Hence there is no bottle-neck nor contention for the bus. The devices are networked on the UART TTL side of the bridge.
 

MrSoftware

Joined Oct 29, 2013
1,936
Slowing down the data rate for the COM ports is a good idea.

To serialize your COM port access in c#, look at lock(), TryEnter(), etc.. There are a few ways you can approach it.
 

danadak

Joined Mar 10, 2018
4,057
DMA is Direct Memory Access, where HW takes incoming COM data
and moves it over HW busses directly to memory with no SW intervention.

Allows you to achieve many Mega Bytes/sec data transfers (depending on
processor being used.)..

Regards, Dana.
 

MrSoftware

Joined Oct 29, 2013
1,936
DMA would be driver level on the PC. He could check, but I don't think the standard c# COM interface is going to expose that option.
 

jfitter

Joined Mar 14, 2019
14
Did you write the application yourself? You must ensure that all called procedures are thread safe. This is not guaranteed in Windows/Linux/C/C++ and varies from one compiler to the other. Your problem has the signature of this. Just because you do I/O sequentially in your code does not mean it is executed sequentially by the OS. There are threadsafe versions of many procedures, or let someone else worry about it and use BOOST.
Just my ideas...
 

MrSoftware

Joined Oct 29, 2013
1,936
Boost is very useful (and lots of the Boost features are in the later c++ std libraries), but he's using c# and I don't think there is any port of boost for c#.
 

pmd34

Joined Feb 22, 2014
518
Hi surfline, I had a similar system I made actually running some process equipment. I did actually end up using RS485 from the individual data senders, that way you can use the same transmission wires, and actually daisy chain everything.
As MrChips suggested, the best thing is to then have a single USB interface which connects to multiple data lines. I used an FT230X and an atmel microcontroller (with two USARTS for simplicity) to do the multiplexing.

I actually programed it with a list of data senders, that it would poll one at a time, at the end of this it would send all the data together to FTDI chip along with some delimiter characters so I could number crunch and bin the data more easily.
 

killivolt

Joined Jan 10, 2010
789
I worked at a company as a Tech, back then they used RS232 master slave config we had at least up to 25 boxes daisy chained works well. RS485 is robust compared to a simple RS232. Distance is no longer a challenge.

All the software however was written in Linux to keep it proprietary, having the additional protection is worth it.


kv
 
Top