I don't quite understand this part, I'll study a little before I say something wrong.

 

I don't quite understand this part, I'll study a little before I say something wrong.

Hi,

Take your time, no rush. Once you see how simple the concept is though you'll see why i keep this technique in mind. The full version is probably found on the web, and most likely would be written using derivatives like dF/dKp, dF/dKi (F being the function here) which forms the gradient but the version i showed you was for numerical derivatives, so it's the same thing but the calculation is done with small increments instead of the actual derivatives.

In either case though it's like being blind and standing on a hill side where you can imagine looking up toward the top of the mountain and down toward the valley. You want to go down toward the valley (where the solution is) so you take a test step in four to eight directions and move in the directions that take you down the hill the fastest. Once you get there, you do the same thing again. Eventually, any test step you take does not lead you down the hill anymore. You either accept that you are at the lowest point now, or take bigger steps (giant perhaps) to try to discover if you are really at the lowest point or should try some other starting points also.

BTW, i think i may have explained the time to decrease the increment incorrectly. You only decrease the increment by 2 once you cant find a better solution with the current increment.
On the hillside example, this would mean that once you take steps in all directions and dont find a lower point, you decrease your step size by 1/2 and try again. You get better and better accuracy that way.

There will also be times when you get the same sum of squares solution for different variable sizes. This happens when there is a region that gives the same results. In that case you start looking at which errors are the most important and try to pick a solution that satisfies that as close as possible as well as the sum of squares.