I have been interested in ANN using RC networks and ICs such as 74HC24x and 74HC14. I was wondering if there is still value investigating how useful a hardwired approach could be. Perhaps combined with a minimal amount of uC programming.

 

I have been interested in ANN using RC networks and ICs such as 74HC24x and 74HC14. I was wondering if there is still value investigating how useful a hardwired approach could be. Perhaps combined with a minimal amount of uC programming.

Meh. I've always thought of this as a solution searching for a problem.

 

I have been interested in ANN using RC networks and ICs such as 74HC24x and 74HC14. I was wondering if there is still value investigating how useful a hardwired approach could be. Perhaps combined with a minimal amount of uC programming.

Welcome to AAC.

I think if there is any value in a dedicated hardware approach it is most likely to come in the form of FPGAs rather than discrete logic.

 

Thank you Ya'akov. With my very basic knowledge thus far, FPGAs sound like they are worth investigating. Never heard of them until this month. Maybe I could work my way from ground zero so I get an understanding of the various neural networks as mentioned by the likes of Mark Tilden.

 

Thank you Ya'akov. With my very basic knowledge thus far, FPGAs sound like they are worth investigating. Never heard of them until this month. Maybe I could work my way from ground zero so I get an understanding of the various neural networks as mentioned by the likes of Mark Tilden.

There are some nice hobbyist options for FPGA dev boards. You could do some hands on with them. The other thing you might find interesting is the edge AI/ML boards designed to so NN-like things at the “edge”—like object, face, and voice recognition among other things.

 

Would it be worth studying human anatomy, now that I have nearly finished an associate degree in instrumentation control and automation, to help with building ANN? Does that idea make sense? That is what I am wanting to head towards but I have no idea how to get to where I want to go. I see I am very much behind the eight ball.

 

Would it be worth studying human anatomy, now that I have nearly finished an associate degree in instrumentation control and automation, to help with building ANN? Does that idea make sense? That is what I am wanting to head towards but I have no idea how to get to where I want to go. I see I am very much behind the eight ball.

Well, while NN are based on the idea of neurons, I think we are past the point that insights leasing to progress can be gleaned from there. I would say it is well worth understanding the logical structure of neurons, the biochemistry is probably not very enlightening.

We really don’t use analogues of neurotransmitters, K and Ca ions, action potential and the like. It’s very interesting in it’s own right but probably not as an aid to developing digital neural networks.

On the other hand, try watching the MIT Open Courseware lectures own YouTube from 9.40 Introduction to Neural Computation (you can download all of the course materials here, for free). MIT’s Open Courseware is marvelous. You might find some of the information from there relevant and you will certainly learn things.

 

So, analogue ANN is not worth investigating any more? Thank you for the great information. I will check it out during the week. For hobbyist sake I may do some tinkering with the BEAM robotics circuits I have on hand then progress my learning with the tools you suggested when I can find the money. Great feedback.

 

On the subject of FPGAs it may be worth mentioning that there are analog and mixed signal devices along with development tools like Verilog AMS that may be worth a look. Complex designs are more often realizable via Hardware Description Languages than ANY other method. The alternatives are not even close in terms of capability.

 

So, analogue ANN is not worth investigating any more? Thank you for the great information. I will check it out during the week. For hobbyist sake I may do some tinkering with the BEAM robotics circuits I have on hand then progress my learning with the tools you suggested when I can find the money. Great feedback.

I can’t be so definitive as to say there is not place for an analog approach to neural networks, there may well be. I guess what I am saying is, first investigate the current state of the art so you can see where such a thing has value.

That course definitely covers the material you would need to understand how that might work.

 

Much as computers were reduced from room-sized monstrosities, it is now possible to make processors that function like the human brain in a single package, as in the case of IBM's 4096 core TrueNorth, a single chip that mimics one million human neurons and 256 million synapses. That design can reduce the power requirements for neural net processing down to one-tenth of what had formerly been required.

From here

 

From here

That’s a bit confusing. ANN and CNN are very different things and the linked story is very light on detail but mentions CNN.

ANN does use brain-like networks of “neurons” with inputs and weights. It does a sort of summation of the weighted inputs to come up with the output. This is very similar to the way neurons reach a threshold and fire, acting on other neurons, etc.

CNN, on the other hand uses a set of processing “layers“ to filter the data. It is brain-like in that it does relate to the overall effect of the brain’s own neural networks but it doesn’t use the input weighting of ANN.

The details of the story seems garbled, and everything else is behind a registration wall.

In any case, ANN attempts to create a neural network mimicking the brain at the… I don’t know… morphological level. That is, it ignores the higher level abstraction that CNN is all about. The TS’ question was whether biology might be relevant to ANN (I assume) because of this. That MIT course is about modeling neurons as electrical circuits, so it seems it might be something he can use for that.

 

To me the connection between cognition and any kind of neural network appears to be tenuous at best. Progress in this area seems to go in fits and starts over the last 60 years with no "killer app" ever coming to the fore.

 

It is the part about mimicking 1M neurons and 256M synapses that I was pointing out. If they can do that on one chip it seems pointless to try to build discrete analog circuits for nets probably 4 orders if magnitude smaller.

 

I think analog computing is being overlooked, it has great potential I think, there is research ongoing today but rarely gets much press.

These machines can solve differential equations in close to real-time.



See also: WHY ALGORITHMS SUCK AND ANALOG COMPUTERS ARE THE FUTURE and MYTHIC Corporation.

 

I think analog computing is being overlooked, it has great potential I think, there is research ongoing today but rarely gets much press.

These machines can solve differential equations in close to real-time.

View attachment 293394

See also: WHY ALGORITHMS SUCK AND ANALOG COMPUTERS ARE THE FUTURE and MYTHIC Corporation.

There is a rather large difference between an analog computer solving differential equations and a neural network. Large, expensive analog computers were de rigeur in the 1950's thru the 1970's and were a well understood technology. Sadly, they are seldom used in practice nowadays.

 

There is a rather large difference between an analog computer solving differential equations and a neural network. Large, expensive analog computers were de rigeur in the 1950's thru the 1970's and were a well understood technology. Sadly, they are seldom used in practice nowadays.

Well the mathematics of a neural net can be performed by analog systems:



An analog system can readily replicate these kinds of relationships, I mean op-amps are ideally suited to that.

 

I am dubious of the claim that anybody would assemble a collection of opamps to solve a particular differential equations. I also believe that you cannot identify a manufacturer of a large-scale analog computers after the demise of ADI(Applied Dynamics Inc.) and EAI systems efforts in that field.

 

I don't know if there are any commercially-available versions, but look up memristors (passive devices which relate flux to charge). Large memristor arrays have been used with great success in the past to create NNs. Plus, a memristor can act almost exactly like the Hodgkin-Huxley model of the neuron, which is neat.

 

I am dubious of the claim that anybody would assemble a collection of opamps to solve a particular differential equations. I also believe that you cannot identify a manufacturer of a large-scale analog computers after the demise of ADI(Applied Dynamics Inc.) and EAI systems efforts in that field.

See: https://www.dummies.com/article/tec...-differential-equations-using-op-amps-166161/

The term "operational" in their name originates from their initial intended use case, mathematical operations.