I am a hobbyist and relatively new to building electronics and never worked with camera sensors, so I'd love to get some help selecting the appropriate components for a potential project.

So what I'm attempting to research building is a handheld camera that produces an effect like this:


Essentially to accomplish this in a handheld format requires a multiple camera sensor array so all the images can be taken at the same moment and processed together in a gif for display and download.

I'm trying to figure out the hardware for this and how to design it. I'm having trouble selecting the components and determining the data flow. I need multiple cameras to fire at the exact (or nearly exact) same moment, store the images on board temporarily, and then send their images to a central cpu for processing. Any help or pointers is appreciated. Which components can you recommend I look into for the cameras and how do I accomplish the image cache from sensor to cpu?

 

Welcome to AAC.

I am a hobbyist and relatively new to building electronics and never worked with camera sensors

I don’t intend to discourage you when I say that this project may be too large and complex for your stated level of expertise.

First of all, how did you get to the step of choosing a sensor? What other work have you done?

Second, this is at least as big a software project as hardware, do you have programming experience?

I suggest you step back and do some research about what you will need. By that I mean take something simple like a Raspberry Pi camera and mechanically test your ideas. Take shots of something static so timing isn’t an issue and then work out how you will use the software to crate the image. And work out unexpected challenges.

Armed with the experimental data you can being to specify a system with some (at least ball park) numbers attached to the components. You will see how much CPU your processing takes. You will learn other things I couldn’t guess, which is the point.

Consider it Prototype 0.1, you will have to build several prototypes of varying sophistication as you work on this project. Prototyping is very important to the design process and initial prototypes often look nothing like the final design because their purpose is to investigate one or another aspect of the project.

Maybe you have already done some of this work, if so please excuse the assumptions, but the framing of your question suggests otherwise.

If you have done, that information is critical in helping just like it is critical for you in designing.

This is almost certainly going to be a very challenging and time consuming project but that doesn’t mean you can’t do it, it just means you have to take that into account and objectively consider the demands it will place on your skill or acquiring skills, and the investment of time.

 

From what I see, this is a software application. There is nothing in the test image that suggests anything but a simple single plane images.

 

He need to take more than one angle simultaneously, like stereo photography, only it appears to have three angles.

 

its an illusion.

 

Welcome to AAC.

Thanks!

I don’t intend to discourage you when I say that this project may be too large and complex for your stated level of expertise.

I understand this is a larger project, it's been on my mind for a few years. I'm only researching the possible design to understand what it would take at this point. I don't think it's too large and complex for my level of expertise, but I'm willing to learn what I need to if I go forward.

First of all, how did you get to the step of choosing a sensor? What other work have you done?

I've done nothing for this project. I have completed other hobby projects before, minimal electronics at this point, but I'm familiar with the process.

Second, this is at least as big a software project as hardware, do you have programming experience?

Yes, I'm a programmer with a degree in Computer Science, I'm sorry if I made myself look to be less experienced.

I suggest you step back and do some research about what you will need. By that I mean take something simple like a Raspberry Pi camera and mechanically test your ideas. Take shots of something static so timing isn’t an issue and then work out how you will use the software to crate the image. And work out unexpected challenges.

Armed with the experimental data you can being to specify a system with some (at least ball park) numbers attached to the components. You will see how much CPU your processing takes. You will learn other things I couldn’t guess, which is the point.

Consider it Prototype 0.1, you will have to build several prototypes of varying sophistication as you work on this project. Prototyping is very important to the design process and initial prototypes often look nothing like the final design because their purpose is to investigate one or another aspect of the project.

Maybe you have already done some of this work, if so please excuse the assumptions, but the framing of your question suggests otherwise.

If you have done, that information is critical in helping just like it is critical for you in designing.

I am definitely at the pre-prototyping stage looking to move into prototyping, I should have made that more clear in my post. I'm wondering what components at a prototyping cost I should consider. I've tried looking on various websites and various specs for camera modules, ESP32 for example, but I thought maybe if someone with more experience can see what I'm trying to do and point out a couple of possibilities, I can learn from their experience.

This is almost certainly going to be a very challenging and time consuming project but that doesn’t mean you can’t do it, it just means you have to take that into account and objectively consider the demands it will place on your skill or acquiring skills, and the investment of time.

I appreciate the concern and I'm certain my post looks amateurish because I am, in fact, an amateur. I have worked with Arduinos, raspberry pis, I've worked with breadboards, PCBs, etc. and studied electronics on my own, but haven't built a huge project like this, so I am aware of the undertaking and what it will take, if I do move forward. I made my post earlier on the phone and the brevity probably framed my question as you perceived it.

 

its an illusion.

So, I own the Nishika N8000, which is a 35mm 4 lens camera for making lenticular photos, but people have scanned in the images and put them in sequence in a gif to create the 3D illusion you see.
Try googling "Nishika N8000 3D gifs" and see the youtube video below. In the example of my first thread you can see the confetti move in front of the other pieces. While I'm not entirely certain this image was taken with a multi camera setup or a computer generated effect, the effect I'm going for is similar and achievable with multiple cameras. I'm just trying to put it into a handheld format like the N8000.

 

Welcome to AAC.



I don’t intend to discourage you when I say that this project may be too large and complex for your stated level of expertise.

First of all, how did you get to the step of choosing a sensor? What other work have you done?

Second, this is at least as big a software project as hardware, do you have programming experience?

I suggest you step back and do some research about what you will need. By that I mean take something simple like a Raspberry Pi camera and mechanically test your ideas. Take shots of something static so timing isn’t an issue and then work out how you will use the software to crate the image. And work out unexpected challenges.

Armed with the experimental data you can being to specify a system with some (at least ball park) numbers attached to the components. You will see how much CPU your processing takes. You will learn other things I couldn’t guess, which is the point.

Consider it Prototype 0.1, you will have to build several prototypes of varying sophistication as you work on this project. Prototyping is very important to the design process and initial prototypes often look nothing like the final design because their purpose is to investigate one or another aspect of the project.

Maybe you have already done some of this work, if so please excuse the assumptions, but the framing of your question suggests otherwise.

If you have done, that information is critical in helping just like it is critical for you in designing.

This is almost certainly going to be a very challenging and time consuming project but that doesn’t mean you can’t do it, it just means you have to take that into account and objectively consider the demands it will place on your skill or acquiring skills, and the investment of time.

So, you misunderstood my question about “other work”, I was referring to other work on this project. What I mean to say is that choosing components is a task for later in a project not at the outset. How would you know what you need?

That;s why I suggested you start with the RPi and its camera to get an idea of how it might work. How could someone advise you when you don’t even have a specification for resolution, or focal length of the lens?

I suppose if someone had already done this project they would know but if that was the case surely you could find those people on the web. I don’t know if you’ve searched for that but I expect you have.

THe sort of prototype I am suggesting is akin to making a small 3D model of something so you can visualize how the larger, actual version would work. Like are some parts in interference? Does it look the way you expected? Is it feasible when you try it?

Inb your case, successfully creating an image using your own hardware and software should give you a lot of information you currently don’t have. Starting off with a single sensor and using a static subject would give oyu a chance to develop a sense of the processing pipeline, the feasibility of using low resolution sensors, and other things I can’t predict.

Then you can try adding a second sensor and see how that works out, etc.

It is a lot easier to help someone solve problems they encounter on the way to doing something than to make suggestions without information. Who knows, maybe someone will just happen to have the information but I feel that is unlikely. You can surely, though, get plenty of help making a prototype work.

None of this is meant to be critical, I am jut relating my experience in creating systems like this. It’s the best advice OI have. If you don’t find it useful you should ignore it and I won’t take it as an insult either.

I wish you the best of luck in creating this. As you go along it would be very nice to hear updates if you are so inclined, it’s a very interesting project and it would be very informative to know how you decided to go about it.

 

So, you misunderstood my question about “other work”, I was referring to other work on this project. What I mean to say is that choosing components is a task for later in a project not at the outset. How would you know what you need?

That;s why I suggested you start with the RPi and its camera to get an idea of how it might work. How could someone advise you when you don’t even have a specification for resolution, or focal length of the lens?

I suppose if someone had already done this project they would know but if that was the case surely you could find those people on the web. I don’t know if you’ve searched for that but I expect you have.

THe sort of prototype I am suggesting is akin to making a small 3D model of something so you can visualize how the larger, actual version would work. Like are some parts in interference? Does it look the way you expected? Is it feasible when you try it?

Inb your case, successfully creating an image using your own hardware and software should give you a lot of information you currently don’t have. Starting off with a single sensor and using a static subject would give oyu a chance to develop a sense of the processing pipeline, the feasibility of using low resolution sensors, and other things I can’t predict.

Then you can try adding a second sensor and see how that works out, etc.

It is a lot easier to help someone solve problems they encounter on the way to doing something than to make suggestions without information. Who knows, maybe someone will just happen to have the information but I feel that is unlikely. You can surely, though, get plenty of help making a prototype work.

None of this is meant to be critical, I am jut relating my experience in creating systems like this. It’s the best advice OI have. If you don’t find it useful you should ignore it and I won’t take it as an insult either.

I wish you the best of luck in creating this. As you go along it would be very nice to hear updates if you are so inclined, it’s a very interesting project and it would be very informative to know how you decided to go about it.

Thanks for all that, it does help me to better understand the process and that what I'm looking for may be a little too premature. I intend to document my journey on this project on hackaday should I choose to build it. I'll link it here if I do.

I completely understand that trying to build this project from scratch (from sensor to OS) is an undertaking that even a professional would balk at. I figure if I find the right camera module and interface to an integrated system like a raspberry pi, most of the hard work will be done for me.

I found a camera multiplexer for the raspberry pi that might work, looking into the specs, but the price is a bit prohibitive. https://ivmech.com/magaza/en/develo...t-raspberry-pi-camera-module-multiplexer-p-90

 

I was thinking that it might be cheaper just to use an ESP32 module ($2-$4) for each sensor, and transfer the data to a central processor, possibly an RPi. Alternatively, an RPi Pico ($4) using its PIO capability for each sensor, and an RPi (maybe a 4, or compute module) for the processing.

With either of those arrangements you could precook the data on the MCU before shipping it off to the SBC.

 

who would have thought, in 35mm yet.

 

who would have thought, in 35mm yet.

Indeed. I think making this project digital affords one the ability to play around with camera angles and that might lead to some interesting results. Stereoscopic photography has ideal camera angle to distance ratios, I would love to be able to work that in somehow mechanically, but I can't see myself adding stepper motors at this point.

 

I was thinking that it might be cheaper just to use an ESP32 module ($2-$4) for each sensor, and transfer the data to a central processor, possibly an RPi. Alternatively, an RPi Pico ($4) using its PIO capability for each sensor, and an RPi (maybe a 4, or compute module) for the processing.

With either of those arrangements you could precook the data on the MCU before shipping it off to the SBC.

I'll need to investigate these concepts further to be able to properly comment.

With method 1 I am unsure how to snap and trigger the camera modules simultaneously and receive the images sequentially back to the SBC. I imagine there'd be a few ms delay between triggers received which may be acceptable I think. I'd have to determine if the esp32 has appropriate caching for images, which I'd expect it does with being a 32 bit MCU.

The second idea I just am not sure how to interface between the pico and RPi but I imagine it's pretty straightforward.

Thanks for the ideas, you've been a huge help!

 

This reminds me of the "Making of the Matrix" which left an impression on me when I first saw how they did it (it was pretty impressive in 1999, at least to me, I was a kid).

I've thought about recreating the effect myself, and how much cheaper it would be with the technology we have now; a hobbyist with a few thousand dollars to spare could actually afford to do this since (qty: 120) $3,000 Canon cameras wouldn't be required. Here's how I've thought of recreating the Matrix scene, which isn't exactly the same thing you're asking for, but might give you some inspiration.

You could use some camera modules like this ($25) attached to Arduinos ($23/ea). So $50 for each perspective/angle you want; maybe less, if you can do some clever things. Then tie them together with a PC or Raspberry Pi. The shutter inputs could be all tied together for a moving still-frame shot, or fired in sequence to capture motion. Each device takes one picture, stores one picture. Once everyone has taken their picture they hold it until it is retrieved by the master PC/RPi. Each device could have an address, and the master polls each in turn, retrieving its picture and stitching it into an animation. This would also be a lot more streamlined and efficient than collecting 120 CF cards from 120 Canon cameras.

 

I'll need to investigate these concepts further to be able to properly comment.

With method 1 I am unsure how to snap and trigger the camera modules simultaneously and receive the images sequentially back to the SBC. I imagine there'd be a few ms delay between triggers received which may be acceptable I think. I'd have to determine if the esp32 has appropriate caching for images, which I'd expect it does with being a 32 bit MCU.

The second idea I just am not sure how to interface between the pico and RPi but I imagine it's pretty straightforward.

Thanks for the ideas, you've been a huge help!

I don't think shutter sync would be a problem. You can communicate the shutter release using a GPIO pin, and since they are dedicated and have nothing else to do they could be "armed" so the latency would be minimal.

What might be a bit harder, though not very difficult I think, would be syncing with flash. You just have to ensure the shutter speed is sufficiently low, just like in an SLR, that it doesn't missed or truncate the flash.

It also sticks me that you could do something akin to Apple's Live Photos and take a very short 30fps video with each camera which has some interesting possibilities for post processing.

 

You could collect a suitable number of some common Canon digicam that is compatible with CHDK and trigger them remotely simultaneously. Old digicams usually sell for well under $10 at thrift stores.
CHDK is the Canon Hack Development Kit.
https://chdk.fandom.com/wiki/CHDK
https://chdk.fandom.com/wiki/USB_Remote

I've heard there are WiFi SD cards, so maybe you could access the photos without the effort of moving cards around.

 

You could collect a suitable number of some common Canon digicam that is compatible with CHDK and trigger them remotely simultaneously. Old digicams usually sell for well under $10 at thrift stores.
CHDK is the Canon Hack Development Kit.
https://chdk.fandom.com/wiki/CHDK
https://chdk.fandom.com/wiki/USB_Remote

I've heard there are WiFi SD cards, so maybe you could access the photos without the effort of moving cards around.

Thanks for that reasonable solution. I had totally considered doing this with separate cameras triggered remotely, and this would be a much, MUCH easier way to do this. This is a hobby for me and ultimately if I were to pursue the project, it would only interest me to build this into a handheld format. Perhaps anyone coming across this reply would appreciate the faster results your method would create. Thanks again for your reply!

 

If you're looking for camera PCBs for your embedded system project, this article will help you.
https://pcbtracks.com/introduction-to-camera-pcb/
It talks about the factors that should be considered before buying. Such as Frame Rate, Sensitivity, Lens etc.

 

Just cobble together the right gear - don't bother designing some fancy hardware, that is an all-consuming sinkhole.

https://www.e-consystems.com/Industrial-Camera.asp

Buy some of these and get them to trigger simultaneously.

 

@Joeyda3rd Have you made any progress on this?