Analog computing is still an interest area for some researchers, the idea being that you can save time performing a calculation by doing it "instantaneously" (or nearly so, with the right setup) instead of waiting for clock cycles or whatnot. Tied in with in-memory computing, the two technologies combined lend themselves nicely to fast, efficient systems - especially with NNs and similar.

 

Very well, all you have to do now is simulate this in code and I'll retract what I said.

View attachment 293471

Your're kidding right. Far more complex things have been simulated.
https://www.ecmwf.int/en/about/media-centre/focus/2022/fact-sheet-supercomputing-ecmwf

 

Analog computing is still an interest area for some researchers, the idea being that you can save time performing a calculation by doing it "instantaneously" (or nearly so, with the right setup) instead of waiting for clock cycles or whatnot. Tied in with in-memory computing, the two technologies combined lend themselves nicely to fast, efficient systems - especially with NNs and similar.

When I see the word "nearly instantaneously" in analog computing I know my finger is being pulled.

 

"nearly instantaneously" is a relative term, and probably not the one I should have used. Compared to multiple clock cycles to do the same operation in digital, some calculations done in analog are "instantaneous," as in, within a clock cycle. Of course, this is assuming the calculation CAN be done in analog (for instance, I'm not sure if the FFT even has an analog method to perform it).
Will analog computing be a thing on its own (to compete with digital)? Probably not. However, there is merit to using it in conjunction with digital systems - digital/analog hybrid processing.

 

Your're kidding right. Far more complex things have been simulated.
https://www.ecmwf.int/en/about/media-centre/focus/2022/fact-sheet-supercomputing-ecmwf

Yes I was kidding, I was alluding to the simulation of "chaos"!

 

"nearly instantaneously" is a relative term, and probably not the one I should have used. Compared to multiple clock cycles to do the same operation in digital, some calculations done in analog are "instantaneous," as in, within a clock cycle. Of course, this is assuming the calculation CAN be done in analog (for instance, I'm not sure if the FFT even has an analog method to perform it).
Will analog computing be a thing on its own (to compete with digital)? Probably not. However, there is merit to using it in conjunction with digital systems - digital/analog hybrid processing.

I agree there are still limited places for single function (mainly filters) analog processing at the front end of digital processing systems
Yes, it's relative. We have cheap 32-bit 250MHz controllers with hardware FP that can handle IEEE single precision math operations with single-cycle throughput that can handle a Fifth-order Polynomial Calculation in 7us (from a controller chip benchmark). Relative to what it would take (current price and complexity) for an analog system to have the same performance specifications, this is outstanding IMO because it can also do a huge number of other problems with the same hardware. It's so cheap to just keep piling on specialized digital functionally with modern process methods.

 

I agree there are still limited places for single function (mainly filters) analog processing at the front end of digital processing systems
Yes, it's relative. We have cheap 32-bit 250MHz controllers with hardware FP that can handle IEEE single precision math operations with single-cycle throughput that can handle a Fifth-order Polynomial Calculation in 7us (from a controller chip benchmark). Relative to what it would take (current price and complexity) for an analog system to have the same performance specifications, this is outstanding IMO.

Fair enough. I work on R&D related to RF frontends, more or less (so amplifiers, mixers, etc) for my job, and my prior research in school involved "doing physics" with analog systems (PT-symmetric electronics, to be precise, which can be used to simulate all sorts of quantum or classical systems), so I am a bit partial to the analog world.

While I can definitely think of ways to do the fifth-order polynomial, as an example, in analog, I do agree that to have it work in the same timespan as the digital would be challenging, without having to resort to ultrafast devices (which get...expensive). 7us for such an evaluation is very impressive, given the 250 MHz clock (that's, what, 1.5 clock cycles?).

 

I agree there are still limited places for single function (mainly filters) analog processing at the front end of digital processing systems
Yes, it's relative. We have cheap 32-bit 250MHz controllers with hardware FP that can handle IEEE single precision math operations with single-cycle throughput that can handle a Fifth-order Polynomial Calculation in 7us (from a controller chip benchmark). Relative to what it would take (current price and complexity) for an analog system to have the same performance specifications, this is outstanding IMO because it can also do a huge number of other problems with the same hardware. It's so cheap to just keep piling on specialized digital functionally with modern process methods.

Well what if we ask "create me a precise digital simulation of a Chua Circuit" could that be done? I suspect it could not, for similar reasons to why the Mandelbrot set looks different as we adjust the threshold (for regarding a point as never leaving the inner circle). There's no way to deduce that some point will not escape the threshold after infinite iterations, the code just has some limit and that is used, but that limit is a limit in the implementation not the underlying math.

Yes one can approximately simulate a Chua Circuit but I doubt that a simulation left running for an hour would still be close to the actual circuit, it would drift away and eventually fail to simulate because the inherent chaotic nature of the real system is simply not present in the algorithm, in fact algorithms cannot be chaotic for the same reason that they cannot produce randomness.

Chaotic systems, where miniscule changes to inputs can have huge effects on outputs, when combined and cascaded must be near impossible to simulate digitally because the digital implementation is always just an approximation to the actual physical system.

The universe, the behavior of things in the universe, are (and have been for centuries, throughout the scientific revolution) represented by mathematics not if/then/else rules. Theories in physics are mathematical, relationships, not if/then/else step by step logic, the universe has no clock.

So because the digital/algorithmic world is so far removed from nature I do not regard it as a good way to achieve what nature does.

 

Well what if we ask "create me a precise digital simulation of a Chua Circuit" could that be done? I suspect it could not, for similar reasons to why the Mandelbrot set looks different as we adjust the threshold (for regarding a point as never leaving the inner circle). There's no way to deduce that some point will not escape the threshold after infinite iterations, the code just has some limit and that is used, but that limit is a limit in the implementation not the underlying math.

Yes one can approximately simulate a Chua Circuit but I doubt that a simulation left running for an hour would still be close to the actual circuit, it would drift away and eventually fail to simulate because the inherent chaotic nature of the real system is simply not present in the algorithm, in fact algorithms cannot be chaotic for the same reason that they cannot produce randomness.

Chaotic systems, where miniscule changes to inputs can have huge effects on outputs, when combined and cascaded must be near impossible to simulate digitally because the digital implementation is always just an approximation to the actual physical system.

The universe, the behavior of things in the universe, are (and have been for centuries, throughout the scientific revolution) represented by mathematics not if/then/else rules. Theories in physics are mathematical, relationships, not if/then/else step by step logic, the universe has no clock.

So because the digital/algorithmic world is so far removed from nature I do not regard it as a good way to achieve what nature does.

Of course algorithms can be chaotic. It's mathematically deterministic.

There are cryptographic systems that use chaotic algorithms with cryptographic keys for the initial conditions.
https://people.eecs.berkeley.edu/~chua/papers/Sobhy01.pdf

 

Of course algorithms can be chaotic. It's mathematically deterministic.

There are cryptographic systems that use chaotic algorithms with cryptographic keys for the initial conditions.
https://people.eecs.berkeley.edu/~chua/papers/Sobhy01.pdf

Thanks. The little video was thought provoking, but as you well know, there is no formal definition of "chaos" or a "chaotic" function. I totally agree that these systems (e.g. the Chua circuit) are deterministic, I do not dispute that, but they are not predictable and therein lies the subtlety.

A Turing machine is deterministic BUT it is also predictable (just run another instance of it for long enough and you get your "prediction"). The Chua circuit is not predictable whereas an algorithm is (almost by definition).

If I run some algorithm here that is the same as some algorithm you might run in your lab, then they will agree and they will always agree, they are predictable - if I run some circuit simulation here for for five hours, then freeze it, you can do the same you'll see that my prediction of your state will be correct when you freeze your simulation after five hours.

Deterministic and predictable are not the same, I can supply some source code and if we are using the same machine, the same OS, the same compiler, then I can predict what error's you'll get for some input will be EXACTLY the same as I got for that input - but the Chua circuit is not predictable this way, that's why I argue that algorithms are not and cannot be, chaotic.

 

Thanks. The little video was thought provoking, but as you well know, there is no formal definition of "chaos" or a "chaotic" function. I totally agree that these systems (e.g. the Chua circuit) are deterministic, I do not dispute that, but they are not predictable and therein lies the subtlety.

A Turing machine is deterministic BUT it is also predictable (just run another instance of it for long enough and you get your "prediction"). The Chua circuit is not predictable whereas an algorithm is (almost by definition).

If I run some algorithm here that is the same as some algorithm you might run in your lab, then they will agree and they will always agree, they are predictable - if I run some circuit simulation here for for five hours, then freeze it, you can do the same you'll see that my prediction of your state will be correct when you freeze your simulation after five hours.

Deterministic and predictable are not the same, I can supply some source code and if we are using the same machine, the same OS, the same compiler, then I can predict what error's you'll get for some input will be EXACTLY the same as I got for that input - but the Chua circuit is not predictable this way, that's why I argue that algorithms are not and cannot be, chaotic.

If you want to make up your own definitions then you can talk to yourself.

 

If you want to make up your own definitions then you can talk to yourself.

I did not "make up" any definition, what led you to write that?

 

Note:



From the encyclopedia Britannica.

 

Very well, all you have to do now is simulate this in code and I'll retract what I said.

View attachment 293471

Loving this banter. LOL. Almost crying with laughter. I suppose the obvious question is, 'Why?'.
Why waste computer power on something that will work effectively using RC networks and sensors which vary with voltage or current? The variation produced by the sensor changes the weighting of the pulses of the RC network which in turn causes a desired outcome which appears natural because that is how cells in animals work more or less? Yes, No?

 

It can be difficult to decide what is and is not chaotic. Certain systems of differential equations are, but the algorithms we use to solve them numerically are not. Is this your thesis?

 

It can be difficult to decide what is and is not chaotic. Certain systems of differential equations are, but the algorithms we use to solve them numerically are not. Is this your thesis?

I am not doing a thesis on this subject. Just very interested but also very new to electronics used for ANN. I find the idea of machines which move without software, quite intriguing and potentially under investigated. I understand that computers have an important role in complicated algorithms. However, I wonder whether simple circuits are overlooked too quickly. I am happy to be wrong about the importance of simple circuits to simulate biology. I am not there yet. I am but a novice mucking with the adults when maybe I should have been quiet until I know a bit more about what has already been achieved.

 

Fair enough. I work on R&D related to RF frontends, more or less (so amplifiers, mixers, etc) for my job, and my prior research in school involved "doing physics" with analog systems (PT-symmetric electronics, to be precise, which can be used to simulate all sorts of quantum or classical systems), so I am a bit partial to the analog world.

While I can definitely think of ways to do the fifth-order polynomial, as an example, in analog, I do agree that to have it work in the same timespan as the digital would be challenging, without having to resort to ultrafast devices (which get...expensive). 7us for such an evaluation is very impressive, given the 250 MHz clock (that's, what, 1.5 clock cycles?).

250MHz clock is a period of 4 ns, so in 7 us thats 1750 clocks

 

Very well, all you have to do now is simulate this in code and I'll retract what I said.

View attachment 293471

Looks like a flow diagram for the Twitter code base,

 

I am not doing a thesis on this subject. Just very interested but also very new to electronics used for ANN. I find the idea of machines which move without software, quite intriguing and potentially under investigated. I understand that computers have an important role in complicated algorithms. However, I wonder whether simple circuits are overlooked too quickly. I am happy to be wrong about the importance of simple circuits to simulate biology. I am not there yet. I am but a novice mucking with the adults when maybe I should have been quiet until I know a bit more about what has already been achieved.

sorry we all went off on our own track,
we do have habit of this, which can be good, but could be a comment on "engineers" in general.

Analog neural networks, been around a long time
problem that cam up , is that they dont scale to the many thousand of neurons "needed" for AI
AI has also gone off in another direction of "large data bases"( chat GPT et all ) , which is not neuron based !

at its simplest, a single levle neuron is equivilent to a virtual ground op amp mixer

the second problem analog AI hit was how to "teach" the network,
analog in real life has habit of oscilating, saturating which is much harder to control than in digits.

 

I am not doing a thesis on this subject. Just very interested but also very new to electronics used for ANN. I find the idea of machines which move without software, quite intriguing and potentially under investigated. I understand that computers have an important role in complicated algorithms. However, I wonder whether simple circuits are overlooked too quickly. I am happy to be wrong about the importance of simple circuits to simulate biology. I am not there yet. I am but a novice mucking with the adults when maybe I should have been quiet until I know a bit more about what has already been achieved.

The question is really about whether it can be done more efficiently, at the needed scale, than is possible digitally.
No, simple circuits are not overlooked too quickly. The analog designs needed for usable system today are far from simple as they require an equivalent level of programming in analog for the same effect.
https://semiengineering.com/developers-turn-to-analog-for-neural-nets/

When asked why everyone doesn’t do this, Doyle answered with a simple statement: “It’s hard.”

It’s also hard to build tools that are significantly better than what a good analog engineer can do already. “The tooling and infrastructure that digital has brought is not easy to mimic with analog.” Ashutosh Pandey, systems engineer and senior member of the technical staff at Infineon.

Ramesh Chettuvetty, senior director of applications engineering and product marketing for memory solutions in Infineon’s RAM product line, also noted that analog circuits are optimized for the application and data rate. If designed for a high rate, higher power is required, so there’s less efficiency if you want to run that circuit at a lower data rate. “If you are running the system at a lower data rate, then the analog is going to be burning that power,” he said.

Finally, there is some energy give-back when you inevitably transition back into the digital domain. “At some point, you will need power to convert the analog back to digital,” said Keith Schaub, vice president of technology and strategy at Advantest. “It’s not free.”