OK, I've been studying ODEs (ordinary differential equations) all night, and I've just had a thought.

Say we know apriori that a system of ODEs has a asymptotic steady state solution. This means that the solution is no longer changing, so we have y'=0. If we can write our differential equation in the form y'=f(y), then this means in turn that we only need to examine points where f(y)=0. This can be done with any old root finder, then the candidate "steady-state" solutions can be examined for stability. This we know.

What if, instead, we know apriori that a system of ODEs has a asymptotic periodic solution? In particular, this should be true for the ionic concentrations of heart cells. We can write y(t) = y(t+P), and y'(t)=f(y). How can we solve for all possible periodic solutions? Do people do this sort of thing? I've never seen anyone examine a system this way.

You could rewrite the equations in an integral form and take the Fourier transform of both sides, but what that get you? You still can't compute the Fourier transform because you don't know what y looks like.

Tags: ideas, math

This person build a wooden adding machine using gravity and marbles. Your computer does the same thing as this computer, but uses groups of electrons instead of marbles and the AC outlet instead of gravity.*




* Note: This is an extreme simplification.

Tags: compsci, math

Alright, inspired by a conversation with my dear sister castlerock, I've decided to publish something a little different today. I'm going to be talking about infinity, and the different types of infinity that occur in higher mathematics.

Most people deal with infinity every day:

• "I will love you forever"
  
• "When you die, you are dead forever."
  
• "You go to heaven forever."

We also play games as children:

• "I'm better than you times infinity, plus 1!"

In one way, the concept of "infinity" is easy to understand. I believe the brain has a special internal language for the words "this pattern holds," also known as "and so on", "yadda yadda yadda," "forever". Forever encompasses the idea of closure. It means that if you understand a finite number of things, you can extrapolate and deal with real world problems. Mentally, we perform this powerful reduction all the time. Without it, we wouldn't be able to cope. In this way, infinity in comforting.

However, if you look at infinity from a size standpoint, infinity is dizzyingly terrifying. Most people don't have a proper concept of what "large" means, but mathematicians know better:

• The universe is 13.7 billion years old, or 4.3*10^-17 seconds. Current quantum theory has trouble measuring time below Planck's time constant, or 1.855×^-43 s. If we treat the idea of time as a discrete number of Planck's constants, then the age of the universe is 8*10^60 of these constants. Infinity is bigger than this.
  
• The number of atoms in the universe is estimated to be around 10^80.
  
• The biggest number ever seriously used in mathematics is Graham's number. The number is an upper bound for some mathematical property. The number is so huge that it cannot be written with scientific notation (the exponent would have more digits than atoms in the universe). Still, infinity is bigger than this number.
  
• The Ackermann function is a strange non primitive recursive function:
  
  This function generates huge finite numbers. For example, A(5,2) is so large that it cannot be describing it with common math notation would take more letters than there are atoms in the universe (this includes 9^9^9^9^9^9... etc.). Still, infinity is bigger than this number.

In fact, infinity is more than just a really huge number. Infinity breaks our notion of numbers altogether.

• Say we have a set of all integers, and we split that up into a sets of all even integers and all odd integers. Obviously, any integer is going to be even or odd, and no integer can be both. Therefore, the set of all even integers is smaller than the set of all integers, right? Wrong. The size of the set of even integers is the same size as integers.
  
• A line goes to infinity in two directions, but a ray only goes to infinity in one direction. Still, lines are the same size as rays, even though if you divide a line in half, you get two rays.

The reason our intuition breaks is that infinity isn't a number. x/2 is less than x if x is finite, but infinity/2 is an abuse of notation, and technically gives you infinity again. The same goes for infinity+1, or infinity+infinity. All our intuition about the "measure" of things is broken by infinity, so we need a new idea of what it means to measure something infinite.

Richard Dedekind and George Cantor came up with a clever solution to compare sets with infinite size. If you have two sets A and B, and you can construct a mapping (a->b) so that every b is covered, then size(A) >= size(B). For finite sets this can be seen by the following diagram. As you can see, there is no mapping from B to A that covers all of A.



This idea is useful because it allows us to compare the size of sets without having to say what size(A) is. Also this definition corresponds to infinite sets as well: For example, positive integers = even integers because we can construct the mappings (n -> 2n) and (2n -> n). Therefore, A <= B and B >= A, so A=B.

So what infinite sets are equal to each other? The following sets all have the same size:

• N, the set of positive integers
  
• The set of all integers
  
• All subsets of integers
  
• The 2d natural number plane, (N, N)
  
• Q, the set of all numbers that can be written as n/m where n and m are integers.

However, you can prove that the set of real numbers between 0 and 1 (which includes things like sqrt(2)-1) is BIGGER than the set of all positive integers. Wikipedia does a good job of explaining the proof, I'll write it out in detail if people are interested.

Higher mathematics has two different words for types of infinity. Countable infinity means the size of the set is equal to the size of positive integers, and uncountable infinity means that it's not. We distinguish between these two cases because working with countably infinite things is nice and working with uncountably infinite things is messy.

In numerical analysis, this is really important for solving equations. Let's say that we're trying to find a unique function from 0 to 1 to real numbers with a special property. There are an uncountably infinite number of these functions. If we write a computer program to test all these functions and let it run forever, that computer program can only do a countably infinite number of function tests. We can prove this by numbering the tests that the computer does them. Therefore, no computer can test all possible functions, even if it runs forever.

Does this mean that trying to solve a differential equation on a computer is hopeless? The answer is yes and no. We cannot solve a differential equation exactly, but by allowing some error, we can transform the uncountably infinite space into some thing that is countably infinite or even finite. This is the heart of all numerical analysis.

(
Side note, if you want to win the "I'm better times infinity plus 1" game, here are a couple of guidelines:
* ack(n,n) > n^n for large n
* n^n > n! for large n
* n! > x^n, x finite, for large n

When someone tells you they are better than you by "infinity plus 1", use some of these huge functions to blow them away.
)

Tags: math, whatrobdoes

Last time, I covered the differential equation for Newton's Law of Cooling. The example I used was modeling the temperature of a cup of coffee sitting on your desk. The temperature of the coffee is described by



where C(t) is the temperature of the coffee at a given point in time (starting at 90 degrees F), S is the temperature of it's surroundings (70 degrees F), and p is some constant given by experimental measurement (1/20).

By itself, this equation isn't all that useful. We'd really like to find an equation for C(t) that only has constants and t. When we find an explicit formula for C(t) that only involves constants and time, we say that this formula "solves" or "satisfies" the differential equation. Note that solving a differential equation is not like solving an algebra equation. You can't just simplify and get C(t) to one side because that pesky C'(t) gets in the way.

For now, I'll skip the solution process and just give you the answer. Those of you in the audience who know calculus can verify that this C(t) solves the differential equation by plugging it in and verifying that it solves the equation.



Here, C(0) means the starting temperature of the coffee, 90 degrees in our last example. You can verify this equation in your head: when t=0, the exponential part equals 1, and we get C(0)=C(0). When t is really really huge, the exponential part goes to 0, and we get C(t)=S, or in other words the coffee has cooled.

The graph of this function is shown here.



Now that we have an explicit formula for C(t), we can find the temperature of the coffee at any moment in time. Lets assume that you like your coffee at 80 degrees F. This graph tells you that you'll have to wait about 14 minutes in order for the coffee to get that cool.

---

I just used a differential equation to describe how a cup of coffee cools over time. I showed you how we find all the parameters in the equation. I pulled a solution to the equation out of thin air, and showed you how we can use that solution to find exactly how long we should let our coffee cool. At this point, you should have some questions. I'll try to cover the concept of differential equation modeling from a few different viewpoints.

Practical Viewpoint - Why do we need all this complicated math to model how a cup of coffee cools? Can't we just stick our pinky finger in the coffee and be done with it?

Yes, we could, and using a differential equation to model the cooling of a cup of coffee is probably overkill. I just wanted to pick an example that was common to most people. I wanted you thinking of me the next time you are in Starbucks.

However, the same equation that works for coffee works for any heating situation. Let's take an example that is closer to home. During hurricane Katrina in New Orleans, lots of people lost power to their homes for a long time. If the power goes out for more than a day or so, you need to worry about the contents of your freezer. If the power goes out long enough, you need to find a way to cook the meat and other perishables you have so they don't spoil. However, you don't want to open the fridge too often to check the temperature, because every time you open the door you let out the cold air. How long do you have until you need to cook all your meat?

The normal temperature for the contents of your freezer is about 0 degrees. In a New Orleans summer, the temperature of your outside surroundings is about 90 degrees. Meat begins to spoil above 37 degrees. As soon as the power goes out, your fridge's temperature is governed by Newton's Law of Cooling.

By opening your fridge one hour after the power goes out, you can get an estimate of p. The exact value of p will depend on the insulation of your freezer and how much meat you have stored in there. After you've measured p, you can use the solution to Newton's Law of Cooling to estimate up to the hour when you'll have to start eating the meat inside your freezer.

Physics viewpoint - You said before that Newton's Law of Cooling made a whole bunch of simplifications. Can we change our differential equation to account for those things?

Yes. You can add extra terms to the differential equations so that it can account for the following things:
1.) The fact that the top of the coffee looses heat faster than the side and bottom of the coffee.
2.) The fact that the air around the coffee is warmer than the rest of the room
3.) The fact that the temperature in the coffee itself is non-uniform
4.) The fact that the temperature differences in the coffee causes the hot parts to rise, creating movement within the liquid.
5.) Etc.

You can do all these things, but as a result the equation gets harder and harder to solve. With (1), you no longer have a single variable for the coffee, so you need to take the coffee's 3D shape into account. With (2), the variable S becomes a function in time, S(t), and you need more differential equations that describe how S(t) is related to it's derivative. With (3) and (4), you need differential equations that describe how fluids move due to temperature differences. Note that even if we took (1-4) into account, we still would only have about 10 differential equations we would need to solve simultaneously. This is the great thing about differential equations-- a few simple equations can model even the most complex physical systems.

Math viewpoint - How do we solve an arbitrary differential equation?

This is what makes differential equations so interesting. Most of the time, we CAN'T solve a differential equation. Mathematicians have been working on this for 300 years, but even after all this time, we don't have a way to solve an arbitrary differential equation. Take the following two equations, but instance:



The top equation can be solved by y=e^(-t), but the bottom equation is much trickier. We know that a solution exists, but we also know that this mystery solution cannot be described by any finite number of math symbols!

We only know how to exactly solve a few easy equations like Newton's Law of Cooling, and we know how to find the solution to some special cases. For most of the interesting physics equations though, we can't find a solution to the differential equation even though we can prove that a solution exists. Solving these equations gets so complicated that some researchers spend their whole life trying to come up with approximations to the solutions to one differential equation.

What I do

Now that we have all this background, I can finally describe what I do. These differential equations are too hard to solve exactly, but often we can find approximate answers that are close enough for practical use. I work on designing computer calculators that solve approximations to these differential equations.

I want to post here more often, so over the next couple of weeks I'll be spending one post per day on one differential equation. For now, I don't want this journal to get too mathy-- instead, I want to try and develop an intuition as to what kinds of things you can model with differential equations. After I've convinced you that solving differential equations is important, I'll show you how we can cheat and find a "good enough" answer to a differential equation even if we can't solve it exactly. I'll show you what goes into making these advanced calculators.

Tags: calculus, diffeq, math, whatido

I covered calculus in such detail before because I wanted to move on to the cool stuff-- differential equations. A differential equation is an equation between a function and it's derivatives. These equations occur all the time in physics-- in fact, most Newtonian physical laws can be written as differential equations. This is why I kept on re-iterating that calculus is the language of physics.

For example, assume that you have a cup of hot coffee that is sitting on your desk. The temperature of the coffee over time can be determined by Newton's Law of Cooling. The following website has an excellent explanation of Newton's Law of Cooling: "The rate of change of the temperature of an object is proportional to the difference between its own temperature and the ambient temperature (i.e. the temperature of its surroundings). " This statement makes a whole bunch of simplifications. It assumes that the heat coming from the coffee won't affect the heat of the surroundings. Anyone who has used a cup of coffee to warm their hands in the winter knows that this is false. Also, it assumes that the top of the coffee is the same temperature as the rest of the coffee, even though everyone knows you can cool down the surface of the coffee by blowing across its top. Still, even with all the simplifications, Newton's Law of Cooling is a pretty good description of how a hot cup of coffee cools over time.

When Newton says "rate of change of the temperature," he means the instantaneous rate of change of the temperature. At a single moment in time, let C(t) be the temperature of the Coffee at time t, and let S be the temperature of the Surroundings. We can express this equation as



"Proportional to" means that one is a multiple of the other, so we can make this an exact equation by introducing a Proportion p:



I need to explain two things in this equation: the negative sign and the constant p. The p constant here is a positive number determined by the exact physical situation we are modeling. If our coffee is in a thermos, then it will stay hot longer. We can model this by making p smaller, so that the rate of temperature change is smaller. Likewise, if we have less coffee or if our coffee is in a paper cup, then p is going to be a large number. In reality, we can estimate p by measuring the temperature of the coffee after 1 minute of cooling. Lets say our coffee is 90 degrees F, and the surroundings are 70 degrees F. If the coffee cools to 89 degrees in 1 minute, we say that C'(t) is approximately 1 degree per minute, and C-S is 20 degrees. Therefore, we can solve for p:



The negative sign in the differential equation is just an artifact of the math. The negative sign says that "If the coffee is hotter than the surroundings, then the temperature of the coffee should decrease." For our hot coffee example, if our coffee is at 90 degrees F and our surroundings are at 70 degrees F, then the temperature of our coffee should be decreasing. Without the negative sign, for p=1/20 we would have that the rate of our temperature change would be p(C-S)=(90-70)/20=1, a positive number. This would imply that the temperature would be increasing, which is the opposite of what we are describing! The negative sign fixes this problem.

Next time, I'll give you the solution to this differential equation and talk a bit about why differential equations are so hard to solve.

Tags: calculus, diffeq, math, whatido

Brock, I'd like to thank you for the links and sending attention my way.

Again, if you're confused by any of this, please respond in the comments. I'll edit the post to make things even clearer. I'd like to keep these posts as a reference that I can refer people to later.

Today, we're going to talk about the other half of calculus: finding the area under a curve. You occasionally need this in engineering. Let's say you are an aircraft engineer, and you've figured out an equation g(x) that describes the perfect cross section for an airplane wing.

( airplane wing )

Assume for the moment that the wing doesn't taper, and that the wing is 10 meters long. The total volume of metal you need to build the wing equals (base area of cross section)*(length of the wing). Therefore, you want to know what is the area of the cross section under the curve so you can tell your boss how much metal he needs to buy. How do you find the area under this curve?

In calculus, this process is called "integration." Much like computing derivatives, the idea behind integration is simple: To measure the area under any curve, divide it up into a whole bunch of tall and thin rectangles, then add up the area of those rectangles.

( Estimate the area )

As your rectangles get thinner and thinner, you get a better and better estimate until you get exactly the right answer. Also, notice that the left side of each rectangle touches the function. This is just a convention. I could have chosen the right side, the midpoint, or pretty much any point. As the rectangles get thinner, you get closer to the correct answer regardless of your choice.

Like the derivative operation eventually produces a new function that describes the slope, you can use math to define a function that computes the area under a curve. Given any function g(x), you can write a math equation for these thin rectangles, factor out the small "h" (where h is the width of the rectangle), and get another function G(x) so that the area between "a" and "b" is G(b)-G(a). As an engineer, this helps you because you can now you have a simple expression that gives you the exact area under the curve. Unlike the derivatives, I'm not going to show you this because it's a bit mathy and doesn't give you any intuition. The important thing to remember is that like a derivative, integration is a process you apply to a function g(x) that produces another function G(x), and G(x) allows you to measure the area under a curve.

Here's the surprising part: Derivatives and integration 'undo' each other. In other words, (area of f'(x) from a to b) = f(b)-f(a). Do you remember that we showed yesterday that if f(x)=x^2, then f'(x)=2x? If you wanted to find the area under the graph of f'(x)=2x from 0 to 1, all you need to compute is f(1)-f(0) = 1-0 = 1. This matches our geometric formula: (area of a triangle)=0.5*(base)*(height)=0.5*(1)*(2)=1. The amazing thing is that this works for any function and it's derivative.

If you understood all that, you should be experiencing utter disbelief. We said that derivative meant "slope of" and that integration meant "area under the curve". How can these two operations 'undo' each other? Why are they even related? What's going on? You don't find out the reason why until your junior or senior year of college as a Math major. Most people never find out. This brings us to our third take home point: Derivatives and integration undo each other because they are Magic.

There you go, that is everything that you learn in a year of calculus. The rest of the time is spent doing endless drills and problems, which you forget in 5 years anyway. If you've gotten this far, you have as good a conceptual understanding of calculus as anyone who has taken 1 year of study in the subject. The ideas behind calculus are simple.

Here's a summary of everything you need to know about calculus to understand the rest of my posts.

• Calculus is the language of physics.
  
• The "slope of a graph at a point" is called the derivative at that point.
  
• If you zoom into any function, it becomes a straight line with a measurable slope.
  
• Whenever you are at a maximum or minimum, the slope of the graph is zero.
  
• To find the area under a curve, we say you "integrate" that curve
  
• To measure the area under any curve, add up the area of tall thin rectangles.
  
• Derivatives and integration undo each other because they are Magic.
  

Tags: calculus, math, whatido

In my last post, I talked about how I want to do research on how to build better calculators. However, I was being a bit misleading. Your basic hand-held calculator can solve all the math problems that most people encounter in their daily lives, so that problem has already been well solved. When I say I want to build better calculators, I mean that I want to build computers that solve harder math problems.

Here's where I come to a dilemma. My research usually tries to solve problems written in the language of calculus, and just the word "calculus" scares most people. Calculus seems to have this mystique of difficulty. For 85% of the population, calculus means "hard math that you never had because you chose a non-calc path in high school." For the other 14.9% of the population, calculus means "The highest (and hardest) math I know." My high school physics teacher never took calculus. Whenever he talked about it, his voice was filled with equal parts fear and awe, as if calculus were some black art that could rip the physical planes asunder if used improperly.

I want to dispel this myth. Today, I'll try to explain the notation of calculus and how it helps us. Then, I'll explain half of the results of calculus in 5 minutes. The target for this posts is the absolute lay person-- an amateur. I sometimes expect simple geometry, simple high school algebra, and that's about it. So if you don't understand these posts, please let me know in the comments.


( Continue reading )

This, in a nutshell, is the first 3 months of calculus. You spend all your time calculating "slopes", or "derivatives" of many different functions. You learn some tricks that allow you to quickly calculate the "slope" of arbitrary functions. Then, you drill solving 100 maximization problems until you can do it in your sleep. The reason that you had to learn all those complicated simplification procedures in Algebra II was because calculating the "slope" can sometimes involve some tricky simplifications. From a conceptual standpoint though, you can sum up the first three months of calculus with the following ideas:

• The "slope of a graph at a point" is called the derivative at that point.
  
• If you zoom into any function, it becomes a straight line.
  
• Once you zoom in, you can just measure slope at that point.
  
• Whenever you are at a maximum or minimum, the slope of the graph is zero.
  
• Calculus is the language of physics. If there is a Heaven where you can learn the mysteries and wonders of the universe, then you can be sure that everyone in Heaven knows calculus.
  

Next post, I'll talk about the other half of calculus: integration. I'll show you why the "slope of" aka derivative operation is magically related to the area under a curve.

(Edit, fixed the statistics as per this this graph from the NSF.)

Tags: calculus, math, whatido