Artificial Neural Networks using RC networks and ICs.

nsaspook

Joined Aug 27, 2009
16,363
I worked on old analog navy analog fire-control/ships dead reckoning equipment in the 70's. I'm extremely dubious of the claim that analog computing is the future. What can't be done, (in the realm of computation) with digital representation, of analog equations running on analog circuits?
 

Thread Starter

Aus_DIYer

Joined May 2, 2023
52
I have enjoyed the tooing and froing of this conversation. Glad I opened up a can of worms. Love it. I watched the YouTube video that was posted. I found it quite simple to understand and have downloaded the course content for later. I know that it is possible to build a quadruped machine that walks using simple components because I made one by following a circuit diagram I found on an old BEAM robotics site. Sure it is now 20 year old tech but it worked. No software required.
If a simplistic system can mimic life without programming in software, why not have that simplistic system and leave the computing power for the more difficult tasks like recognition of objects and determining the best trajectory? A hybrid if you will.
Solarbotics.net is where it all started for me. Some of the links no longer work but a majority of the information is still there. It is where I am at in my journey. It is a starting point of ANN, for me. I like the fact that there is ongoing research in ANN as I believe it has great potential. We are analogue beings after-all.
 

atferrari

Joined Jan 6, 2004
5,017
I worked on old analog navy analog fire-control/ships dead reckoning equipment in the 70's.
Add the Loadicator used in freighters to compute draft, trim & stress in merchant vessels of various types. More than precise enough for the job, they were a (warm) festival of op amps in the Deck Office.
 

ApacheKid

Joined Jan 12, 2015
1,762
I hardly need such lessons from a kid. Just another example of pointless kidsplaining.
I'm not a "kid" sir I'm 63 years old, nor did I suggest you needed a "lesson" I simply presented information that emphasizes how op-amps have their roots in analog computation.
 

ApacheKid

Joined Jan 12, 2015
1,762
I worked on old analog navy analog fire-control/ships dead reckoning equipment in the 70's. I'm extremely dubious of the claim that analog computing is the future. What can't be done, (in the realm of computation) with digital representation, of analog equations running on analog circuits?
Analog computation is attractive for some situations because power consumption can be much lower. Many digital systems must simulate continuous mathematics by discontinuous digital algorithms. Nature is not algorithmic either, there are no algorithms in physics, in Maxwell's equations.

Look at a simple op-amp integrator circuit, would you prefers to use that or use an MCU with code, memory, D/A and A/D converters, quantization noise etc?

Digital systems with computing are valuable in cases where we have no mathematics, where we can't represent behavior via continuous functions but to assume it is the be all and end all of signal processing is a huge leap of faith.

Take a look at complex dynamical systems on a scope:


Try generating that with an MCU, also have a look here too A (pure analog) Lorenz Attractor Circuit.

While the Lorenz attractor is readily simulated with iterative, discrete-type digital computation techniques on a modern desktop P.C., using software packages such as MATLAB, or even in SPICE with a little more difficulty, it is also readily simulated with simple electronics hardware conforming to a much older concept; that of continuous analog computation. A complete electrical analog of the Lorenz attractor, as described by the three differential equations above, can be implemented with an interconnection of just three distinct, basic and common circuit building blocks; namely the summing amplifier, the integrator and the analog multiplier.
 
Last edited:

ApacheKid

Joined Jan 12, 2015
1,762
Here's another example, extremely interesting stuff IMHO:


I write software for a living and have done for over forty years, it is overrated, assumed by many to be the answer to every problem but it isn't, the natural world, laws of physics are analog in nature, we often forget that.
 

nsaspook

Joined Aug 27, 2009
16,363
I have enjoyed the tooing and froing of this conversation. Glad I opened up a can of worms. Love it. I watched the YouTube video that was posted. I found it quite simple to understand and have downloaded the course content for later. I know that it is possible to build a quadruped machine that walks using simple components because I made one by following a circuit diagram I found on an old BEAM robotics site. Sure it is now 20 year old tech but it worked. No software required.
If a simplistic system can mimic life without programming in software, why not have that simplistic system and leave the computing power for the more difficult tasks like recognition of objects and determining the best trajectory? A hybrid if you will.
Solarbotics.net is where it all started for me. Some of the links no longer work but a majority of the information is still there. It is where I am at in my journey. It is a starting point of ANN, for me. I like the fact that there is ongoing research in ANN as I believe it has great potential. We are analogue beings after-all.
Energy is both quantifiable (digital) AND irreducible (analog). There is no pure state of one or the other. The analog circuit is just an implementation that produces some desired result just like the digital equivalent. IMO the actual implementation is irreverent to the end result, if, they are equivalent. Today, digital implementation of control systems and computing can be vastly superior to analog systems in precision, accuracy, actual usable resolution and scalability. A good example is SDR, where we've used digital to replace most of the analog functionality in a radio to the point we have GHz range transceivers in billions of devices. Programming software in devices designed to be electrical component replacements is just the same as building a analog circuit with transistors, resistors and reactive components.

Unfortunately modern general computer science seems to have morphed into a programming priesthood that seems to look down on the hardware electrical engineering type skills needed to program practical things at the circuit level today.
 

ApacheKid

Joined Jan 12, 2015
1,762
Energy is both quantifiable (digital) AND irreducible (analog). There is no pure state of one or the other. The analog circuit is just an implementation that produces some desired result just like the digital equivalent. IMO the actual implementation is irreverent to the end result, if, they are equivalent. Today, digital implementation of control systems and computing can be vastly superior to analog systems in precision, accuracy, actual usable resolution and scalability. A good example is SDR, where we've used digital to replace most of the analog functionality in a radio to the point we have GHz range transceivers in billions of devices. Programming software in devices designed to be electrical component replacements is just the same as building a analog circuit with transistors, resistors and reactive components.

Unfortunately modern general computer science seems to have morphed into a programming priesthood that seems to look down on the hardware electrical engineering type skills needed to program practical things at the circuit level today.
But consider this random generator discussed here. A digital implementation of this will not replicate the analog, the digital can be reduced to an algorithm and that's pseudo random not random. The analog is matter/energy, obeys, embodies laws of physics whereas the digital relies on an algorithm that approximates the physics.

A digitally generated sine wave, viewed up close suffers from quantization noise, a natural analog oscillator does not - I know you know all this of course, but wonder what your view about this is, how you reconcile that.
 
Last edited:

nsaspook

Joined Aug 27, 2009
16,363
But consider this random generator discussed here. A digital implementation of this will not replicate the analog, the digital can be reduced to an algorithm and that's pseudo random not random. The analog is matter/energy, obeys, embodies laws of physics whereas the digital relies on an algorithm that approximates the physics.

A digitally generated sine wave, viewed up close suffers from quantization noise, a natural analog oscillator does not - I know you know all this of course, but wonder what your view about this is, how you reconcile that.
https://forum.allaboutcircuits.com/threads/beauty.150513/post-1647630

Digital devices have analog properties with normally discreet levels at the chip level. We can easily use the analog nature (non-deterministic states) of the digital circuit as an random number entropy source to create random numbers sequences with an algorithm.
https://www.intrinsic-id.com/wp-content/uploads/2017/05/True-Random-Number.pdf

It's the result that matters, not the analog/digital process. I can approximate a physics result (harmonic oscillator/sine wave) with much higher fidelity with a digital representation of the equation factors instead of using actual voltages and currents connected stage to stage for an expected result of X frequency, X voltage and X phase/frequency shift/drift.
 

ApacheKid

Joined Jan 12, 2015
1,762
https://forum.allaboutcircuits.com/threads/beauty.150513/post-1647630

Digital devices have analog properties with normally discreet levels at the chip level. We can easily use the analog nature (non-deterministic states) of the digital circuit as an random number entropy source to create random numbers sequences with an algorithm.
Well that's fine but it means you are relying on analog behavior not algorithmic behavior, such a system then wouldn't be classed as an algorithmic system, the state of the system is not determined by the algorithm alone. Algorithmic systems are basically FSMs.

It's the result that matters, not the analog/digital process. I can approximate a physics result (harmonic oscillator/sine wave) with much higher fidelity with a digital representation of the equation factors instead of using actual voltages and currents connected stage to stage for an expected result of X frequency, X voltage and X phase/frequency shift/drift.
Well sometimes its the result that matters, if the system is expected to meet some specification and it does, then yes, that's one definition of what matters. But if the inherent differences themselves matter then by definition the algorithmic and analog implementations can't be regarded as the same.

As you know tiny differences in initial conditions can have huge long term consequences in a chaotic system. So the claim that these difference won't matter is only true in cases where we don't care about the differences, but if it is the differences that are relevant then...
 

ApacheKid

Joined Jan 12, 2015
1,762
Ultimately what I'm driving at here, is that there's no proof that the human mind, intelligence, consciousness can be achieved without the presence of analog/chaotic components. It is a belief, faith. Some believe and some argue that a "sufficiently complex computer" can become conscious "eventually".

As Dreyfus points out in What Computers Still Can't Do, there is no evidence that what the brain/mind can do, can be reduced to a symbol manipulator, it is a belief.
 

drjohsmith

Joined Dec 13, 2021
1,621
I'll muddy the waters a bit more,

All digits are doing is modeling how the analog system we have works,
its just that we can put millions of digital switches in a chip, and analog computers just don't seem to shrink the same.
there are even spread sheets which can be used to simulate neural networks.

https://arxiv.org/pdf/1807.00018

What is still missing is how to make a neural network want to learn, as a baby does..

as for analog
I've seen simple two layer neuron network performed on an analog computer back in the 80's
but human analog components, its just to difficult to stop cross feedback / oscilation et all.
 

nsaspook

Joined Aug 27, 2009
16,363
Well that's fine but it means you are relying on analog behavior not algorithmic behavior, such a system then wouldn't be classed as an algorithmic system, the state of the system is not determined by the algorithm alone. Algorithmic systems are basically FSMs.



Well sometimes its the result that matters, if the system is expected to meet some specification and it does, then yes, that's one definition of what matters. But if the inherent differences themselves matter then by definition the algorithmic and analog implementations can't be regarded as the same.

As you know tiny differences in initial conditions can have huge long term consequences in a chaotic system. So the claim that these difference won't matter is only true in cases where we don't care about the differences, but if it is the differences that are relevant then...
The chaotic signal of the analog Random Sequence Generator is not pure analog behavior cause and effect. The voltage/current randomness is not one formed by continuous signals and functions of voltage or current, it's caused by the corruption of the analog circuit by some external energy force like thermal, quantum etc...

There are no pure analog or digital behaviors in physics. Algorithmic behavior or analog behavior are engineering tools, not a total solution for X complex problem today.

The analog/digital divide for circuit solutions is imaginary for real world products today. You use what's best at X time for each function of X system. For the vast majority of cases, digital computing wins, when we have .10 cent 32-bit controllers.
 

drjohsmith

Joined Dec 13, 2021
1,621
Is the world analog or digital ,
discuss !

that's always a great essay for students.

obviously, we know its digital, but hay us digital people let you artists of analog pretend :->
 

BobTPH

Joined Jun 5, 2013
11,572
Until I see a chip with a million opamps, I am skeptical. I believe transistors are way smaller than resistors and capacitors on a chip, which might have something ti do with it.
 

ApacheKid

Joined Jan 12, 2015
1,762
Until I see a chip with a million opamps, I am skeptical. I believe transistors are way smaller than resistors and capacitors on a chip, which might have something ti do with it.
Speaking of transistors being small too:

1683232317472.png
 
Last edited:
Top