I'm an experienced hardware design engineer that has recently been venturing into microcontrollers. I have a good understanding of how they work, and have become an almost adequate C programmer of micros.

Where I'm lacking, and the subject of my question, is structuring my applications. I called the thread "real time" but I do not necessarily mean highly accurate real time programming. I'm talking typical things micros do - reading keypad or other inputs, updating an LCD, reading sensors, responding to serial communications, turning on and off external devices, etc...

Being a hardware guy, I naturally gravitate toward state machines. But an application made up of a large case statement state machine can quickly get out of hand. And I've programmed myself into a corner more than once.

I'm not looking for off-the-shelf RTOS's. What I'm looking for is a source or reference where I can get myself up-to-speed on how to structure my applications. I want to know the basic ways others are doing this and the pros and cons of them.

Can anyone point me to a good resource to get where I want to go?

Thanks.

 

I think you are on the right track! All of my embedded stuff (mostly PIC, lots of different projects) uses cooperative-multitasking, no matter how 'simple' the task.

CONS include a steeper learning curve. The basic kernel provides task-switching/scheduling, timer services, processor management etc. This means writing a framework that provides these services and providing a standard task design that operates under the kernel.

PROS: the payoff comes in ease of programming the tasks. No more delay loops and clock counting. Add/modify a task without fouling others. A consistent programming model that becomes the starting point for pretty much any job. Once the RTOS is done, debugging becomes just the task at hand. For me, a new project becomes installing the kernel, a 1-2 hour job. That provides me with task management, timing, optional serial RS-232/485, EEPROM management, internal ADC etc. as needed. It also provides the framework for the application (one task, initialized to state 0 or to its start addresses, depending on the PIC). Adding tasks is a matter of replicating the framework and adding a pointer to it in the main scheduler.

There are probably lots of references on the web but some to check out:
http://www.pumpkininc.com/content/doc/manual/SalvoUserManual.pdf Salvo is a commercial RTOS which I have used with success. The first few chapters have some good tutorial info on RTOS for embedded systems.

MicroC/OS-II The Real Time Kernel by Jean J. Labrosse is a textbook written around his actual buildable RTOS with source code examples and descriptions of the what and why. Its aimed at PCs mostly but could run on a bigger microcontroller (ARM etc). Not so much for PICs.

Any good book on general Operating System Concepts (Peterson, Tannenbaum et al). While describing OS's for general purpose computing, the principles are applicable to any RTOS.

Microchip.com - Search RTOS and check out the tutorials and apnotes.

You are right about a typical low end RTOS not being 'rea'l time but you can get lots done with standard OS timing services. For hardcore real time stuff you use the chip's hardware peripherials and IRQ.

Many low end RTOS schemes wind up being state machines anyway. For midrange (16F) PICs you don't have access to the stack so tasks must suspend at level 0 .. state machines are a good implementation, the state number identifies the 'resume' address. I like them for lots of reasons but you are right, a real big task can have lots of states and become cumbersome. My worst case ones have topped out at maybe 10-15 states. The 18F has some rudimentary stack operations so task switching is easier.

If you are using PIC, I can give you some specifics on ways to proceed.

 

Interrupts might be another way to handle critical real-time events. Many microcontrollers have interrupt capability to varying degrees. Of course, any real-time processing on the smaller MCUs will not be as flexible as it is on the bus oriented microprocessors which can be expanded to near limitless interrupt levels and priorities. However, if you do not let the applications get too complicated, and prioritize events properly (which are critical, which can be handled asynchronously, which can wait indefinitely, etc..), you can get quite a bit of 'real-time' performance out of even the smaller MCUs.

And then there is always the distributed processing model for really complex situations.

 

As a resource for you to read, I have heard good things about the book "Code complete".

As far as good beginner advice I think a good starting point is to have a main loop which repeats continuously, and it performs your tasks in sequence. Tasks can be things like checking buttons, lighting LEDs, setting output pins etc.

An interrupt can be used to handle things that must be dealt with at that instant, often things like timers and signal generation, and maybe some input tasks if they need to be detected fast.

Another excellent technique is to "flow" your process out on paper before you mess with writing code. A good design can often be completed and debugged on papaer as to the sequence of events and priorities, then writing the code for it is much much easier.

If you outline your project and its tasks, inputs, outputs etc maybe people could suggest a good real-time system to start with?

 

There are many tips on writing large complex code successfully.
If you have ever written a Mac app you would quickly learn how to structure your code using event driven functions, creating the main event loop and using GetNextEvent( ) etc.

You could also use OOP or C++. I prefer not to do this for an embedded MCU in order to avoid code bloat.

Here are some immediate tips:

Learn Structured Programming techniques and adhere to the principles.
Learn about various techniques for choice of variable names and labels.
Learn how to format, indent and comment your code for readability and maintenance.

Use header files (.h)
Break your project into various .c and .h files
Use const defines
Use enum, struct and typedef.

Creating successful "real-time" code has more to do with structured design than knowing how to handle interrupts etc.

 

Thanks guys ... all good info.

I understand about structured code and naming conventions, indenting, header files, etc... all good programming practice.

And I understand how to structure basic embedded applications and have written several. Endless loops and state machines made from switch/case statements. I've used interrupts to handle things before.

But, I would like to have a more complete understanding of how embedded programmers in general use all these things to structure embedded code. I'd like to get a handle on all the options and the pros and cons of each. I was hoping that someone could point to an article or whitepaper somewhere where this is covered. I looked at the "code complete" book and it seems like a great resourse, but is not targeted specifically at embedded code.

Thanks.

 

If state machines get over my head in terms of code complexity/length,
I put them into tables.

So in the end no longer the state machine is translated into C code, a specialized "instruction set" is designed and used, and a pattern is parsed from a table.

This can well make sense for some applications and can simply code, and it will look nice.

If it does not, use a fully featured RTOS...

One advantage also if you use EEPROM for the tables, you can simply plug different EEPROMs, for different behaviour. They are easy to program using GAL programmer, and putting files into the EEPROM space. Just write down the start offsets manually, or if these are many, put them into an index table as well.

I mean there is not really always a need to use SD card, USB drive reader, and fully featured file system.

So techniques like that could be used for 16F or smaller 18F chips, you only get a little overhead for I2C, some logic, and the parser.

Also I often tend to use program structure like digital logic, means many flip-flops which in program code means binary flags. I mean I think in digital logic, rather.

 

Thanks guys ... all good info.

I understand about structured code and naming conventions, indenting, header files, etc... all good programming practice.

And I understand how to structure basic embedded applications and have written several. Endless loops and state machines made from switch/case statements. I've used interrupts to handle things before.

But, I would like to have a more complete understanding of how embedded programmers in general use all these things to structure embedded code. I'd like to get a handle on all the options and the pros and cons of each. I was hoping that someone could point to an article or whitepaper somewhere where this is covered. I looked at the "code complete" book and it seems like a great resourse, but is not targeted specifically at embedded code.

Thanks.

So you are already miles ahead that most new programmers.
You already know the basic and advanced principles. There is not much more to add to the main program event driven loop.

Firstly, I would eliminate the concept of state machines. One of the principles of Mac programming is to create a modeless machine.

All interrupts and events are recorded using semaphores or flags. Very little processing is done within the interrupt handler.

The main event loop calls GetNextEvent( ) and extracts the next event using some event queue or priority scheme. DoEvent() performs the tasks requested by that event.

Here is my template for a main program:

void main(void)
{
  Initialize();
  while (1)
  {
    SystemTask();
    GetNextEvent();
    DoEvent();
  }
}

 

Mr Chips,

Understood. You're template would work well with an event driven application. Something happens, you respond, then you go do something else and wait for the next thing to happen. ISR's do very little other than respond to the event and set flags. As I said, works well in an event driven program.

However, a state machine is very usefull for a program that is more process driven rather than event driven. For example my recent temperature controller. Basically, it measures the temperature then goes to one of three states depending on where it is and how far the measured temperature is from the setpoint: COOLING state, HEATING state, OFF state. This is a natural for a state machine.

BUT , its never that simple. There are almost always events you have to deal with. In the case of the temperature controller the issue was serial commands coming in from the host computer to tell the temperature controller where to set the temperature, how much hysteresis to use, etc... Right now I'm checking the RX register everytime through the state machine and using it as a decision point to go to a COMMAND state that reads the commands via the serial port. Of course the issue here is that if I'm in the middle of reading a series of bytes from the host, and he decides to stop sending them, then I'm stalled waiting for them, and all my other processing is blocked. Its exactly this kind of issue that complicates things and makes the choice of how to structure the code difficult. I was thinking in this case maybe I should use the serial interrupt and read the data one byte at time into an array. In this scenario I read a byte from the serial port then return to regular activity. When the next byte come in I go read it, etc. Each byte is stored in the array until the command is fully populated at which point a flag is set. My state machine polls the flag each time through and only takes action when the flag is set indicating a complete command has been received. Of course then I have to deal with the issue of incomplete or wrong commands. And should I implement a handshake where I ACK or NACK back to the host after each received byte? It gets complicated quickly.

 

Rule #1, never wait for a process to be completed, i.e. never poll for completion and wait until completed.

For serial communications, always use received data interrupt and store the incoming characters in a buffer. The complete command is parsed outside of interrupts in the DoEvent( ) loop.

This way you are never stuck in the middle of an incomplete command and your system is still able to respond to any other interrupt or event.

Serial communications always include parity and error checking as well as invalid commands. What you do after that depends on the serial comm protocol. Usually an error is reported back to the sender or the command is simply ignored.

Again all of these issues are handled not at the programming stage but at the systems design stage. This simplifies the programming effort.

It is very important to separate design from implementation.

 

Don't forget a timeout for receiving multibyte data so the buffer will be reset if the complete data isn't there after the expected time.

 

Mr Chips,

BUT , its never that simple. There are almost always events you have to deal with. In the case of the temperature controller the issue was serial commands coming in from the host computer to tell the temperature controller where to set the temperature, how much hysteresis to use, etc... Right now I'm checking the RX register everytime through the state machine and using it as a decision point to go to a COMMAND state that reads the commands via the serial port. Of course the issue here is that if I'm in the middle of reading a series of bytes from the host, and he decides to stop sending them, then I'm stalled waiting for them, and all my other processing is blocked. Its exactly this kind of issue that complicates things and makes the choice of how to structure the code difficult. I was thinking in this case maybe I should use the serial interrupt and read the data one byte at time into an array. In this scenario I read a byte from the serial port then return to regular activity. When the next byte come in I go read it, etc. Each byte is stored in the array until the command is fully populated at which point a flag is set. My state machine polls the flag each time through and only takes action when the flag is set indicating a complete command has been received. Of course then I have to deal with the issue of incomplete or wrong commands. And should I implement a handshake where I ACK or NACK back to the host after each received byte? It gets complicated quickly.

Lots of good info above. Here's some more..

One solution is to break each function into individual tasks. The attached is derived from your descriptions and shows how a cooperative multitasking setup might be organized to solve your problems.

3 tasks support and generate data that the main task (the temperature controller itself) needs. These tasks are:
(0) A serial message receiver whose job it is to receive setpoints and post them for MAIN to refer to.
(1) An ADC task which generates the current temperature reading.
(2) A switch debouncer for RUN/STOP

Each of these tasks is responsible for its function and to advise MAIN when something is wrong.

(3) MAIN: The temp control task itself, which consumes those outputs and generates the control function.

Note that MAIN has NO direct access to the RX data stream, ADC, switches, only their conditioned outputs. Also note that the support tasks are really providing an interface layer between ugly real-time stuff and the higher level temp-control function.

Scheduling as drawn (a hand-rolled RTOS) you would have to provide a master task switcher (counts tasks 0-3) which calls the tasks in order, passing them their current state. The called task runs and returns the 'resume' state. Each called task is its own state machine (case-statement like you are using). So in this one, you have 4 task state-machines and one sequencer over them.

I showed it as state-machines because you can implement them on almost anything and get the benefit of at least 'multitasking-like' programming. A 'real' multitasking RTOS would allow you to write each task in a procedural fashion, suspending when waiting for I/O etc. then resuming where you left off while providing timing, messaging etc. Way more complicated to implement (but easier to program with it).

On the drawing, tasks suspend whenever you take the stay-in-state edge (loop back to same state in response to a condition). Leaving the state bubble on these edges returns to the scheduler so it can do the next task. The state number is returned to the scheduler so it can resume at the arrowhead of the edge. You'll see ALL waits are on these suspend edges.

Finally, don't beat me up on the design details because there aren't any. Its a quick, broad brush. Error detection etc. is shown only to give you an idea where to do it. There's no deadband and the switch debouncer looks suspiciously like it was block-copied from main..

Hope this helps..

 

Very interesting discussion.

There are a couple of ways to tackle complex real-time systems. One way is to do time budgeting to ensure that all tasks can be completed in a predetermined time slot. The other is via a multitasking RTOS with a task manager/scheduler that allocates execution time to each task.

I am in the middle of an embedded microcontroller project that is fairly complex and I will use this as an example of how I implement the first scheme.

My project is a digital sampling oscilloscope and spectrum analyzer.
The tasks are:

• digitize three analog channels at 3Msps on each channel
• compute the frequency spectrum using FFT
• generate VGA output to LCD screen
• handle mouse input
• handle keyboard input
• USB interface
• ethernet interface
• timer interrupts

The LCD screen must be updated every 16ms.
VGA display is generated on the fly line by line, interrupts every 35μs.
All tasks except the FFT are interrupt driven. The FFT task is the most time consuming but it (and all tasks outside of the interrupt handlers themselves) can be interrupted.

The main program loop has to fetch the digitized waveforms, perform the FFT, generate and update the new screen while handling all the other tasks. All of these in total are accomplished under 16ms.

One of the problems with a preemptive mulitasking RTOS is that all library functions have to be reentrant. For example, suppose the FFT program is in the middle of a calculation that uses a multiply function or a cosine function. Then another task is invoked that calls the same function. The complete state of every function used must be preserved from task to task for each function to be reentrant. The application heap can grow very quickly in such cases.

I have another similar project on the go that digitizes the waveform at 50Msps for gamma-ray spectroscopy.

So there you have it, two approaches - John's multitasking RTOS approach and my serial task execution approach.

 

.. if your processor architecture / OS supports it. Mr. Chips has a nice thing going and is right about using serial execution.

Out here in 8 bit PIC land, words like preemptive and reentrant are reduced to concepts we are left to wish for as penance for a misspent youth. My 18F task switcher actually does support serial task execution but the scheduler has to manually pull and push on the stack to do it.. Reentrancy is still out of the question in most cases, but coding serial ex in a task is way nicer than states, particularly when a task has to suspend at other than call-level 0. The midrange 16F stuff has no stack access so states are about it.

However you approach it, its instructive that in either approach, the overall concepts are similar i.e. lots of individual tasks dedicated to doing their specific function, releasing the CPU when they have to wait on something to allow other tasks to run. Consuming and producing. Execution supervised by some high level process, etc. You could replace the state-based tasks in the drawing boxes with serial execution and the basics would be the same.

In any case, applying any multitasking approach is preferable to superloop-delay constructs, IMHO.

Mr. Chips, your project sounds like a fun one..

 

It is fun, you bet.

I am not using any OS. I have to roll my own.
The processor architecture is absolutely unbelievable. I know nothing about 18F, PIC32 and dsPIC but I bet none will come anywhere close to this ARM chip that I am using. I will post a picture of the LCD screen one day.

Right now I'm working on a board layout for the first prototype.

 

Excellent info, thanks! I have plenty to think about for a while now. Thanks.