undetermined coefficients approaches and variation of parameters

Thread Starter

PG1995

Joined Apr 15, 2011
832
Hi

Linear non-homogeneous differential equations are solved using the following two techniques in the book: (1) Undetermined Coefficients (superposition and annihilator approaches), (2) Variation of Parameters.

The book says undetermined coefficients approaches do not apply when the forcing function (e.g. g(x) in this equation ay'' + by' + cy = g(x)) is: In(x), 1/x, tan(x), arcsin(x) and so on. And in such cases variation of parameters technique is used.

Q1: Does this mean variation of parameters (VP) method is superior to undetermined coefficients (UC) method in that it can applied to cases where UC method fails?

Q2: Please have a look on the attachment. You can see the formula for solving linear non-homogeneous diff. eq. there. Can I apply that formula to the 2nd order equations with getting the same results as with methods UC and VP?

Q3: Is there an easy way to extend the formula in the attachment to apply to higher order equations (4th order is the highest I'm interested in)?

Please help me with the queries above. Thank you very much for all the help.

Regards
PG
 

Attachments

Last edited:

Zazoo

Joined Jul 27, 2011
114
The book says undetermined coefficients approaches do
Q1: Does this mean variation of parameters (VP) method is superior to undetermined coefficients (UC) method in that it can applied to cases where UC method fails?
It depends on what is meant by superior.
UC works because the allowable forcing functions (sinusoids, exponentials, constants) have patterns of differentiation that can be leveraged to "guess" the form of the particular solution.

UC is easier and much faster than VP, so it is superior when applicable. VP, on the other hand, does have the advantage of working with forcing functions that can't be handled by UC.

Q2: Please have a look on the attachment. You can see the formula for solving linear non-homogeneous diff. eq. there. Can I apply that formula to the 2nd order equations with getting the same results as with methods UC and VP?
If they match the form shown, then yes. Although, as noted above, UC will be much faster in cases where it can be applied.

Q3: Is there an easy way to extend the formula in the attachment to apply to higher order equations (4th order is the highest I'm interested in)?
Not that I know of (but I'm far from an expert.) Adding higher orders will change the nature of both the homogeneous and particular solutions.
 

steveb

Joined Jul 3, 2008
2,436
Q1: Does this mean variation of parameters (VP) method is superior to undetermined coefficients (UC) method in that it can applied to cases where UC method fails?
I don't know the answer to this. I'm sure this information can be tracked down or figured out. (EDIT: I see Zazoo gave you some information about this while I was writing this response.) It is a good question and worthy of an answer, but I tend not to think in these terms. I view all of the methods as complementary and as long as a method works, it is good and interesting.

Q2: Please have a look on the attachment. You can see the formula for solving linear non-homogeneous diff. eq. there. Can I apply that formula to the 2nd order equations with getting the same results as with methods UC and VP?
I don't see why not. There is only one general solution to equations of this type. No matter how you arrive at the answer, the answer should be the same if the method is valid and if it is used correctly.

Q3: Is there an easy way to extend the formula in the attachment to apply to higher order equations (4th order is the highest I'm interested in)?
The word "easy" is a relative term. Everything is "simple" and "easy" in hindsight, once you figure it out and absorb it. Usually, things look complex and difficult when you are on the other side of it looking forward. I'm sure you are seeing this as you learn these various methods of solving equations.

To answer your question, there are many different techniques that can be applied to higher order equations of this type. The ones I know are easy :p. The ones I don't know are very very difficult. :D

The two methods that come to mind are the State-Transition-Matrix method (which I prefer myself) and the Laplace Transform method. I don't think you should rush into learning these yet, but it is good to know about them in general terms. These methods are usually formulated in the time/frequency domain because they are used extensively in electrical engineering (particularly control theory), where functions tend to be signals in the time domain.

The following reference discusses the state transition matrix approach in the context of state space analysis. This is a very general and powerful method that can be applied to systems with any number of differential equations of the type you mentioned, with any number of input signals and any number of output signals. This is quite a bit more general than the higher order equation you are asking about, which has only one input signal and one output signal. (note that the Laplace method is also discussed in this book)

http://web1.johnshopkins.edu/csl/ac...m_Theory_Chap_6_State_Variable_Techniques.pdf

The following video discusses the Laplace method which generalizes to any order. This video is discussing the simpler case of one input and one output, but the method can be generalized to multiple inputs and multiple outputs.

http://www.youtube.com/watch?v=HadqGVH0v6U
 
Last edited:

Zazoo

Joined Jul 27, 2011
114
Thanks for that first link steveb, this will really help me. I struggled with state matrices in Controls, and I still don't feel totally comfortable with this method. In scanning the first few pages this looks to be a much better presentation of the method than the one given in my textbook.
 

Thread Starter

PG1995

Joined Apr 15, 2011
832
Thank you very much, Zazoo, Steve. I really appreciate your help.

@Steve: I really like your reply to Q3. Sadly, I don't understand those methods you mentioned but it was good to know that those techniques exist. Perhaps someday in the hindsight the methods you mentioned would seem really simple! But I really doubt this because I know myself...! :)

Please have a look on the attachment. Please help me with those queries. Thank you.

Best regards
PG
 

Attachments

studiot

Joined Nov 9, 2007
4,998
If you are being taught variation of parameters it is probably because this method leads naturally on to numerical methods such as finite element analysis.
 

steveb

Joined Jul 3, 2008
2,436
... variation of parameters ... this method leads naturally on to numerical methods such as finite element analysis.
That's a good point. There can be more than one reason for a person to study various methods, and for a school making them part of a curriculum.

Perhaps someday in the hindsight the methods you mentioned would seem really simple! But I really doubt this because I know myself...!
I have no doubt that, at the proper time, and if your curriculum includes the appropriate subjects, you would be able to learn this. That particular material looks very complex on paper, but is relatively simple to apply. Also, you have demonstrated the ability to learn and the ability to apply yourself, which is all that is required. All of this stuff is just a matter of learning step by step and accumulating the background needed to make it seem simple, which it is (always in hindsight though).

I have to admit that particular method looks absolutely horrendous on paper, and I remember when I first learned it, the idea of taking an exponential of a matrix seemed preposterous. Notice how Zazoo also had an initial struggle with the concepts, but now sees it clearer with a different presentation. Learning is often partly about finding the correct vantage point to look from. Every time a concept or subject seems difficult, I keep working until I find that proper vantage point. Ten dense text books on a subject sometimes shed no light, and then one day you read that one critical sentence that makes it all clear. It's important to know that there is a very fine line between being average and being smart. Often it as more to do with confidence, positive attitude and curiosity that drives one to keep searching for the insight. In other words, keep doing what you have been doing and nothing will be beyond your grasp.
 

steveb

Joined Jul 3, 2008
2,436
Please have a look on the attachment. Please help me with those queries. Thank you.
Q1 is one of those questions that can get the answerer in trouble, due to the word "always". :p I think the simple answer is yes, but every rule seems to have exceptions, and saying something is "always" true is likely to be an overstatement. I don't know what the exceptions might be here, but I always advise caution when making an absolute statement like this.

Q2 is difficult to answer. It does seem that the author is just imposing a condition to get the two equations, but he may have done this after a very long process of thinking that seemed to reveal that the condition is necessary to get at the one allowed solution, or at least he found that the method was valid for obtaining one of many possible solutions. (I wouldn't know which of the two cases applies here, unless I thought about it more.)

In the end, the author uses a nice handy trick and it works. Like you say, it's hard to define "natural". Rather than using the word natural, I would try to insert the word insight, or intuition. Mathematical intuition is a very powerful thing, but mathematicians will not rely on it solely. A formal proof could be (and should be) obtained, once the insight leads to a possible answer. In a very real sense, use of intuition is "unnatural" in mathematics, because even the most intuitive ideas can be found to be wrong when subjected to rigorous mathematical scrutiny. But, this business of solving differential equations is difficult and I like to think of the subject as mathematicians going to war. Anything that gets the job done is going to be considered as a useful weapon.
 

Thread Starter

PG1995

Joined Apr 15, 2011
832
Thank you very much, Steve. I don't know how to thank you enough!

A general solution of a nth order homogeneous diff. equation contains all the n linearly independent solutions. This would mean that there is only one general solution. Please correct me if I'm wrong.(cont'd) Now Please have a look on the attachment. Please help me with it. Thanks.

Best regards
PG
 

Attachments

steveb

Joined Jul 3, 2008
2,436
This would mean that there is only one general solution. Please correct me if I'm wrong.

Now Please have a look on the attachment. Please help me with it.
First, I agree there is only one general solution. That is kind of the definition of "general" in this case. If there was another solution, then the general solution wouldn't be general, and so, we would add it in to make the solution more general. It's kind of like defining the "universe". If you find something that you think is outside the universe, then just add it in to the definition of "universe".

For your attached question, there can be more than one particular solution, in general. Specifically, we can always force another particular solution once we have one, since we can make a particular solution which includes that particular solution plus any fundamental piece of the complementary solution. Indeed, we might do this accidentally without even realizing it. You'd be surprised how functions can "hide" themselves. Remember our previous discussion where we found that very different looking functions might be the same.

This allowance of "complementary solutions" added in to the particular solution does not interfere with the method because the final general solution would still be a valid solution, and forcing the method to require complete elimination of any piece of the complementary solution is a needless complication since the complimentary piece contributes zero always, by definition. Actually, I'm not even sure if it is possible to make a clear definition of yp that is independent of of yc, in general. However, even if you could do it, it would be a moot point as far as the method is concerned.
 

Thread Starter

PG1995

Joined Apr 15, 2011
832
Thank you very much.

In the equation below, D stands for differential operator, Yc is complementary solution, and Yp is particular solution. Could you please tell me other form of Yp we can use which equals the expression on the right side of the equation? Many thanks.

\($D^{2}y+4Dy-2y=2x^{2}-3x+6,\hspace{0.06in}y_{p}=-x^{2}-\frac{5}{2}x-9,\hspace{0.06in}y_{c}=c_{1}e^{-(2+\sqrt{6})x}+c_{2}e^{(-2+\sqrt{6})x}$\)

Best wishes
PG
 

steveb

Joined Jul 3, 2008
2,436
Thank you very much.

In the equation below, D stands for differential operator, Yc is complementary solution, and Yp is particular solution. Could you please tell me other form of Yp we can use which equals the expression on the right side of the equation? Many thanks.

\($D^{2}y+4Dy-2y=2x^{2}-3x+6,\hspace{0.06in}y_{p}=-x^{2}-\frac{5}{2}x-9,\hspace{0.06in}y_{c}=c_{1}e^{-(2+\sqrt{6})x}+c_{2}e^{(-2+\sqrt{6})x}$\)

Best wishes
PG
One possibility is as follows.

\(y_p=-(x+3)^2+3.5x+\cosh(2+\sqrt{6})x-\sinh(2+\sqrt{6})x\)

There are infinitely many others.
 

Thread Starter

PG1995

Joined Apr 15, 2011
832
Thanks a lot. I think I get it somewhat now.

The Yp you gave and the one I mentioned are the same thing inside but looks different from the outside. So, to me, it makes more sense to say that Yp can take on many different forms. In other words, if we have found one Yp, then using algebraic techniques, trigonometric identities etc. we can transform the Yp into completely different form(s).

Please help me with Q1 and Q2. Once again, I offer my thanks.

Best wishes
PG
 

Attachments

Last edited:

steveb

Joined Jul 3, 2008
2,436
The Yp you gave and the one I mentioned are the same thing inside but looks different from the outside. So, to me, it makes more sense to say that Yp can take on many different forms. In other words, if we have found one Yp, then using algebraic techniques, trigonometric identities etc. we can transform the Yp into completely different form(s).
Well, you almost have it, but this is not quite right. It is important to clarify that the yp you gave and the one that I gave are not equal functions at all. If you plot them you will see the are different.

There are two separate things going on when we make another valid yp. For one, we can have an algebraic manipulation, as you say. But, the other thing is that we can add in additional pieces from the compementary solution. These pieces will make the final function (yp) different, but when the new yp is substituted in to the differential equation it still works because the complementary part of it contributes zero.

In this example you gave, it is quite easy to spot the separate pieces and to see how I constructed the new valid yp. However, I think one could find other examples that are much harder to see.

The beauty of the given method is that we can find general solutions to the nonhomogeneous equation, if we know the general solution to the homogeneous equation and if we can find ANY other solution to the non-homogeneous equation. Since no constraints are placed on what form yp must take, our job becomes much easier. Rather than having to find a yp and subject it to various tests and manipulations until we are satisfied that our yp is the one true yp to be placed on a pedestal, we just validate the one we stumbled across and use it.
 

steveb

Joined Jul 3, 2008
2,436
Please help me with Q1 and Q2. Once again, I offer my thanks.
For Q1, the function is defined at zero. It just so happens than the value becomes negative infinity at x=0.

For comparison, lets say I have the funtion y=1/x. In this case I can say the function is valid over the span of all real numbers and i dont' have to exclude x=0 from the definition of the funciton. It just so happens that y goes to infinity as z goes to zero.

There are even stranger functions that might raise some red flags in your mind. Check out the Dirac delta function. It has very strange behavior at x=0, but it is still a valid function over the entire range of real numbers.

http://en.wikipedia.org/wiki/Dirac_delta_function

For Q2, you can't use the normal transcendental functions you are familiar with. What you can do is determine integral relations, or convert to series expansions. Functions like this that turn up again and again are sometimes given a name like erf(x).

http://en.wikipedia.org/wiki/Error_function
 

Thread Starter

PG1995

Joined Apr 15, 2011
832
Thank you very much.

In this example you gave, it is quite easy to spot the separate pieces and to see how I constructed the new valid yp. However, I think one could find other examples that are much harder to see.
Sorry, I can't spot how you got it! :( Well, there are some common terms between Yp and Yc such as "2+sqrt(6)".

For Q1, the function is defined at zero. It just so happens than the value becomes negative infinity at x=0.

For comparison, lets say I have the funtion y=1/x. In this case I can say the function is valid over the span of all real numbers and i dont' have to exclude x=0 from the definition of the funciton. It just so happens that y goes to infinity as z goes to zero.
I know you have said the similar thing once before and at that time I couldn't really understand it and thought would ask you later. Perhaps, now it's the time.

Lim (x->0) sin(x)/x = 1

Here one might say that "Lim" is an operator just like sqrt(). This means as x tends to zero, the expression sin(x)/x gets closer to "1". This is because of the operator "Lim" that we can say that sin(0)/0 = 1; otherwise it's totally wrong, at least this is what I think. So, this expression "7x + sin(x)/x" is undefined at x=0 but its limit at x=0 exists and is "1". In my view, saying that a function is defined at a certain point and its limit exits at some point are two totally different things. When we say the function is defined at a point, then we simply means that it takes on a certain value. But saying its limit exists at some point, simply means that we can know how that function behaves when it gets closer that point. You said that the expression "1/x" is defined at x=0. I don't get this. To me, it seems it isn't defined at x=0 and even its limit doesn't exist at x=0. I'm asking from a beginner's point of view.

Thank you for the help.

Best wishes
PG
 

steveb

Joined Jul 3, 2008
2,436
Sorry, I can't spot how you got it! :( Well, there are some common terms between Yp and Yc such as "2+sqrt(6)".
Don't be sorry. It just means that I'm a better magician than I thought I was. I was actually trying to make a function that might be hard to see through. I basically used the fact that cosh and sinh functions are sums of exponential functions (similar to the Euler relations like exp(ix)=cos(x)+i*sin(x)). But, don't waste time trying to figure it out. The fact that it isn't obvious helps to make my point, and it is the main reason why allowing ANY yp is a useful trick.


I know you have said the similar thing once before and at that time I couldn't really understand it and thought would ask you later. Perhaps, now it's the time.

Lim (x->0) sin(x)/x = 1

Here one might say that "Lim" is an operator just like sqrt(). This means as x tends to zero, the expression sin(x)/x gets closer to "1". This is because of the operator "Lim" that we can say that sin(0)/0 = 1; otherwise it's totally wrong, at least this is what I think. So, this expression "7x + sin(x)/x" is undefined at x=0 but its limit at x=0 exists and is "1". In my view, saying that a function is defined at a certain point and its limit exits at some point are two totally different things. When we say the function is defined at a point, then we simply means that it takes on a certain value. But saying its limit exists at some point, simply means that we can know how that function behaves when it gets closer that point. You said that the expression "1/x" is defined at x=0. I don't get this. To me, it seems it isn't defined at x=0 and even its limit doesn't exist at x=0. I'm asking from a beginner's point of view.
Well, limits are definitely important, but I wasn't trying to draw too heavily on the concept of limits when I answered this particular question. I probably should not have use that phrasing because it just adds confusion to a relatively simple idea.

You are basically questioning what it means for a function to be defined at a point, and you have some good comments.

First of all, I should not use the 1/x as an example, because the definition of the function is not clear at x=0, because the limit from the left is different than the limit from the right. So, that's a bad example on my part.

Also, the author defined the interval of the function, but did not say if the endpoints were inclusive or exclusive. You can define the range and say the the range does not include the endpoint. So, the author may or may not be conflicting with your opinion. I don't think there is any problem saying that the function is not defined at x=0. I also don't think there is a problem trying to define the value there either, although it is clearly going to negative infinity, so any definitions should be made with caution. So, basically, I don't know what opinion a strict mathematician would give on your question.
 
Last edited:

Thread Starter

PG1995

Joined Apr 15, 2011
832
Thank you.

I think the author did mean an open interval (0, pi/6) here and I was wrong earlier where I was saying that the function is defined on interval (-inf, +inf) excluding "0". The function isn't defined for negative values, in my opinion even at "0".

Also, the author defined the interval of the function, but did not say if the endpoints were inclusive or exclusive. You can define the range and say the the range does not include the endpoint. So, the author may or may not be conflicting with your opinion. I don't think there is any problem saying that the function is not defined at x=0. I also don't think there is a problem trying to define the value there either, although it is clearly going to negative infinity, so any definitions should be made with caution. So, basically, I don't know what opinion a strict mathematician would give on your question.
Okay. We are talking about f(x)=In(x) and you are saying that at x=0 f(x)=-inf. Is the [ limit(x->0) In(x) ] defined? I don't think it is. But one sided limit does exist, [ limit(x->+0) In(x)=-inf ]. In my view, the f(x)=In(x) isn't defined at x=0 but it's one sided limit does exist which lets us predict how the function behaves as it gets near to "+0". What do you say on this? I wanted to ask similar question(s) to you in this thread but then thought it'd be better to leave the discussion for sometime and then come back later.

As an addition example, consider this function:
g(x) = (x^2 - 4) / (x - 2)

The g(x) isn't defined at x=2 but the limit does exist at x=2 which is 4.

Now please help me to come out of this confusion. Thank you.

Best wishes
PG
 

Thread Starter

PG1995

Joined Apr 15, 2011
832
Thank you for agreeing with me! :)

But I still think I'm missing something here because even my calculator tells In(0)=-inf. I was expecting to say "Undef".

Now I want to return to this thread but not sure to start the discussion there or here. I think it would be better to start the related discussion there. So, I will post there.

Thanks.
 
Top