Looking for an introduction to CMOS-Microcontroller Interface

Thread Starter

DanaRSullivan

Joined Sep 10, 2008
6
Hello--I'm starting a project which requires a video camera to send simple image data to a microcontroller for processing. The camera doesn't need to do any fancy processing, image tracking, etc; it can just send the images. The first thing the microcontroller needs to do is convert the image data from the camera into something that represents the image as a collection of pixels. (I don't need color information, just greyscale.) For example, it could get an image from the camera and convert it into an array of integers that represent the darkness/brightness of each pixel. I did a little research and it looks like a CMOS camera module is what I need. I checked out the AVRCam and other cameras that have microcontrollers already on them, but it seemed like they do more than I need, and it'd be nice to do something cheaper.

My question is: where can I find a good introduction or tutorial about connecting a CMOS camera module to a microcontroller and getting the microcontroller to understand the data from the camera? Also, what's the best type of microcontroller for this job? I've been reading robotics books, so I know the basics of how to program a microcontroller, but I've never actually used one. I have a lot of programming experience, though. Ideally I'd like to program in C, but I'm willing to learn a new language if it makes things easier on the hardware end. I'm also OK with using a different kind of camera besides CMOS; that just seemed like the best from what I know. I've spent a lot of time Google searching for this stuff, but haven't found a nice, basic introduction.

Edit: I'm putting more info about the requirements here.
--All the processing the microcontroller needs to do is convert frames into 16-shade grayscale and output them as a list of 4-bit pixels with some sort of signature to mark the beginning and end of a frame.
--I'd like to get 20fps, but really as long as I don't go below 10 it'll be satisfactory.
--The microcontroller can grab frames at an inconsistent rate; if the fps swings up and down while it's running, that's not a problem.

Thanks in advance!
 
Last edited:

Papabravo

Joined Feb 24, 2006
21,159
Hello--I'm starting a project which requires a video camera to send simple image data to a microcontroller for processing. The camera doesn't need to do any fancy processing, image tracking, etc; it can just send the images. The first thing the microcontroller needs to do is convert the image data from the camera into something that represents the image as a collection of pixels. (I don't need color information, just greyscale.) For example, it could get an image from the camera and convert it into an array of integers that represent the darkness/brightness of each pixel. I did a little research and it looks like a CMOS camera module is what I need. I checked out the AVRCam and other cameras that have microcontrollers already on them, but it seemed like they do more than I need, and it'd be nice to do something cheaper.

My question is: where can I find a good introduction or tutorial about connecting a CMOS camera module to a microcontroller and getting the microcontroller to understand the data from the camera? Also, what's the best type of microcontroller for this job? I've been reading robotics books, so I know the basics of how to program a microcontroller, but I've never actually used one. I have a lot of programming experience, though. Ideally I'd like to program in C, but I'm willing to learn a new language if it makes things easier on the hardware end. I'm also OK with using a different kind of camera besides CMOS; that just seemed like the best from what I know. I've spent a lot of time Google searching for this stuff, but haven't found a nice, basic introduction.

Thanks in advance!
As I see it your basic problem is finding a device with enough memory to hold a single image. Do you know how to calculate that number? NO -- well here's an example. A standard VGA image with 640 x 480 pixels times 8 bits for each pixel is -- ta da -- 307,200 bytes. That number eliminates most of the microcontrollers that I know about.
 

Thread Starter

DanaRSullivan

Joined Sep 10, 2008
6
Thanks a lot; I'll start reading the documentation for that camera.

As I see it your basic problem is finding a device with enough memory to hold a single image. Do you know how to calculate that number? NO -- well here's an example. A standard VGA image with 640 x 480 pixels times 8 bits for each pixel is -- ta da -- 307,200 bytes. That number eliminates most of the microcontrollers that I know about.
Well, I know it's possible to do image processing on a microcontroller from a video camera, because I can buy something that already does that (here for example), and they're fairly common in robotics projects. I just want to make my own to save money, and for the learning experience. So if VGA images are too large, maybe the cameras used have a different image format or take smaller pictures.

I imagine 16 shades of grey would be fine for my project, so I'd only need 4 bits per pixel, and it'd be nice if I could find a black-and-white camera to not deal with the color data.
 

Papabravo

Joined Feb 24, 2006
21,159
You can slice and dice the numbers in a variety of ways but you won't be doing much image processing on a microcontroller with say 4K of RAM. You're going to need something with an adequate amount of memory running at a reasonable clock rate to capture and process the data. I put the question to the entire forum. Is there a microcontroller with a reasonable shot at meeting the OP's requirements?
 
Last edited:

Papabravo

Joined Feb 24, 2006
21,159
Thanks a lot; I'll start reading the documentation for that camera.



Well, I know it's possible to do image processing on a microcontroller from a video camera, because I can buy something that already does that (here for example), and they're fairly common in robotics projects. I just want to make my own to save money, and for the learning experience. So if VGA images are too large, maybe the cameras used have a different image format or take smaller pictures.

I imagine 16 shades of grey would be fine for my project, so I'd only need 4 bits per pixel, and it'd be nice if I could find a black-and-white camera to not deal with the color data.
I'd hardly call an ARM7TDMI a microcontroller. You can delude yourself into thinking that it is one, but that won't change the facts on the ground. Getting the GNU tools up and running also has a fairly steep learning curve. I wouldn't recommend it for any but the heartiest explorers.
 

Thread Starter

DanaRSullivan

Joined Sep 10, 2008
6
You can slice and dice the numbers in a variety of ways but you won't be doing much image procession on a microcontroller with say 4K of RAM. You're going to need something with an adequate amount of memory running at a reasonable clock rate to capture and process the data. I put the question to the entire forum. Is there a microcontroller with a reasonable shot at meeting the OP's requirements?
Thanks for your help, but maybe I should have been more specific; it doesn't seem like what I'm trying to do should take a lot of processing power. I need to take the frames, convert them into 16-shade grayscale, and output them as a list of 4-bit pixels with some sort of signature to mark the beginning and end of a frame. I'm going for 20fps, but 15 or even 10 wouldn't be that bad, and it's not a problem if the microcontroller isn't consistent with the rate at which it grabs frames. Considering the CMUCam2 can track colors at 50fps and do face recognition, it seems reasonable that I can find a microcontroller that does what I need. The main reason I'm not just using a CMUCam, besides money, is I want to try it out with different cameras and lenses and not be confined to the camera embedded in the controller.

I'm going to copy some of this into the OP so it's more precise.
 

beenthere

Joined Apr 20, 2004
15,819
Apologies for the link - those cameras all have EIA outputs. The bad news, though, is that a brief survey of available cameras indicates that lower resolution in B & W means 659 x 494 pixels, 60 FPS, and 10 bit outputs. That is going to require your system to soak up a bitstream running to 195,330,000 BPS.

Even knocking off some bits to decrease pixel size requires a device to do this pixel-by-pixel, and then you have to store each frame. Most of the problem is that you can only reduce bits/pixel and pixels/frame after the data are in the system. Sounds like two uC's, one to produce the smaller frames and store them in memory, and the other to do the video processing.
 

hgmjr

Joined Jan 28, 2005
9,027
I have been thinking that this may be an application for multiprocessing. Even so, this is a fairly involved undertaking for even a seasoned microcontroller designer.

hgmjr
 

Thread Starter

DanaRSullivan

Joined Sep 10, 2008
6
Apologies for the link - those cameras all have EIA outputs. The bad news, though, is that a brief survey of available cameras indicates that lower resolution in B & W means 659 x 494 pixels, 60 FPS, and 10 bit outputs. That is going to require your system to soak up a bitstream running to 195,330,000 BPS.

Even knocking off some bits to decrease pixel size requires a device to do this pixel-by-pixel, and then you have to store each frame. Most of the problem is that you can only reduce bits/pixel and pixels/frame after the data are in the system. Sounds like two uC's, one to produce the smaller frames and store them in memory, and the other to do the video processing.
Yeah, I think you're right...it will probably end up having an extra uC just for frame storage. When I have a good rough design for this, I'll post it in this thread for anyone else doing similar stuff. It's more complicated than I expected; with all the robotics kits around I assumed there were loads of people looking to easily deal with video frame processing in realtime, and the supply would follow the demand. I might have to simplify the project...

It's too bad cameras don't have programmable settings like "just give me x times y resolution and z shades of gray, please!"
 

beenthere

Joined Apr 20, 2004
15,819
The situation is like trying to drink from a firehose with the valve full open. Overwhelming.

Actually, you do have those programmable settings. All you need to do is grab the CCD element and figure out how to program a uC to read it out. The you can set the reso and frame rate to suit. Then all you have to do is the video processing.

Two things DSP chips have in common: fast clocks and lots of memory. And that's just for audio.
 

Papabravo

Joined Feb 24, 2006
21,159
Thanks for your help, but maybe I should have been more specific; it doesn't seem like what I'm trying to do should take a lot of processing power. I need to take the frames, convert them into 16-shade grayscale, and output them as a list of 4-bit pixels with some sort of signature to mark the beginning and end of a frame. I'm going for 20fps, but 15 or even 10 wouldn't be that bad, and it's not a problem if the microcontroller isn't consistent with the rate at which it grabs frames. Considering the CMUCam2 can track colors at 50fps and do face recognition, it seems reasonable that I can find a microcontroller that does what I need. The main reason I'm not just using a CMUCam, besides money, is I want to try it out with different cameras and lenses and not be confined to the camera embedded in the controller.

I'm going to copy some of this into the OP so it's more precise.
I still think you are ignoring the realities of video stream processing. It is a surprisingly large amount of data and it will be like drinking from a firehose. Just when you finish with one frame along comes the next one. Avoiding words with too many syllables -- You can't do this with a microcontroller, period. You need a platform with two to three orders of magnitude more memory than a microcontroller has and you need to be running at a clock rate of maybe 80 - 240 MHz. One more time with feeling -- microcontrollers can't do those things. Nothing that I've read in your posts suggests that you understand what is involved.
 

Thread Starter

DanaRSullivan

Joined Sep 10, 2008
6
I still think you are ignoring the realities of video stream processing. It is a surprisingly large amount of data and it will be like drinking from a firehose. Just when you finish with one frame along comes the next one. Avoiding words with too many syllables -- You can't do this with a microcontroller, period. You need a platform with two to three orders of magnitude more memory than a microcontroller has and you need to be running at a clock rate of maybe 80 - 240 MHz. One more time with feeling -- microcontrollers can't do those things. Nothing that I've read in your posts suggests that you understand what is involved.
The thing is, I can buy a system for about a hundred dollars that is basically a more powerful version of what I'm trying to build. For example, here is one that does image tracking at 17fps, among other things. If what I'm thinking of is impossible, then what they did would definitely be impossible.

It's true that I couldn't process every single frame, but I don't need to. A frame grabber will take a frame and send it to the microcontroller that does the processing, ignoring any other camera input until it's done, and then grab another one when it's ready. It only needs enough memory to store a single frame. The microcontroller does not need to handle information at the rate the camera outputs it.
 

Papabravo

Joined Feb 24, 2006
21,159
The thing is, I can buy a system for about a hundred dollars that is basically a more powerful version of what I'm trying to build. For example, here is one that does image tracking at 17fps, among other things. If what I'm thinking of is impossible, then what they did would definitely be impossible.

It's true that I couldn't process every single frame, but I don't need to. A frame grabber will take a frame and send it to the microcontroller that does the processing, ignoring any other camera input until it's done, and then grab another one when it's ready. It only needs enough memory to store a single frame. The microcontroller does not need to handle information at the rate the camera outputs it.
Yes you can buy such a thing for a low price precisely because it is made in volume and may or may not continue to be required by the original developer. My point is not that they are impossible, but that they are impossible with a MICROCONTROLLER. An ARM7TDMI is most definitely NOT a microcontroller. Regardless of the cost to purchase, the cost to develop is most certainly several orders of magnitude larger assuming that you can acquire the parts, fabricate a PC board, and then assemble the fine pitch surface mount parts. I haven't heard of a hobbyist putting Ball Grid Array (BGA) onto a board with simple tools.

That said, the processor in the previous link is a PIC clone, but I can't read the part number. It is hard to imagine the part having enough memory to grab an entire frame, but it might be able to use the CCD as the memory, while processing a small piece of the image at one time. Certainly as it was trying to send the entire frame over the serial port it would not be acquiring new frames. Given those compromises, if that meets your requirements, then by all means buy one and get on with your application.
 
Last edited:

Thread Starter

DanaRSullivan

Joined Sep 10, 2008
6
Thanks nanovate for providing this link: So far my plan is to use one of these with a 16-bit microcontroller from Mouser. It'll be about $20 more than getting the CMUCam, not including buying the camera, but for my project it'll be worth it to experiment with different cameras. I also found this article about image tracking in a robot using a GameBoy Camera; I'm not going to do it that way, but someone else who stumbles on this thread might find it useful.
 
Top