Least Squares - UMD - find the least squares solution calculator

---

• (1) Least square line calculator
• (2) Least squares solution calculator matrix
• (3) Method of least squares calculator
• (4) Least squares regression equation calculator

Least Squares
Justin Wyss-Gallifent
July 21, 2021
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
3.2 Reminder - Solutions and Column Space . . . . . . . . . . . . . . 2
3.3 The Intuition and Theory . . . . . . . . . . . . . . . . . . . . . . 3
3.4 Theory: Least-Squares Solution . . . . . . . . . . . . . . . . . . . 4
3.5 Practical: Least-Squares Solution . . . . . . . . . . . . . . . . . . 5
3.6 Picture of a Simple Case . . . . . . . . . . . . . . . . . . . . . . . 8
3.7 Matlab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1
3.1 Introduction
Let's go back to the matrix equation
Ax? = ?b
We know that a unique solution exists if A is invertible and if A is not invertible
then there are either no solutions or infinitely many solutions.
Specifically the question that we'd like to address here is what can we do if there
are no solutions at all? One answer might be to just stop, however maybe we
could ask the question - what's the nearest solution we could find?
In other words if we can't find x? so that Ax? = ?b, can we find some x? so that Ax?
is as close as possible to ?b?
More rigorously can we find some x^ such that:
For all x? we have ||Ax^ - ?b|| ||Ax? - ?b||
3.2 Reminder - Solutions and Column Space
First let's recall:
Definition 3.2.0.1. Given an n ? m matrix A the column space of A is the
subspace of Rn given by:
Col(A) = span {Columns of A}
Fact 3.2.0.1. The column space of A is exactly the vectors ?b such that Ax? = ?b
has at least one solution.
To reinforce this, a simple example will do:
Example 3.1. The equation

12 4
0 3 x? = 5
-1 0 6
2
has a solution if and only if there are x1, x2 with

12 4
0 3 x? = 5
-1 0 6

4
1 2 x1 = 5
0 3
-1 0 x2 6

1x1 + 2x2 4
0x1 + 3x2 = 5
-1x1 + 0x2 6

1 24
x1 0 + x2 3 = 5
-1 0 6

4
1 2
5 col 0 , 3
6 -1 0
3.3 The Intuition and Theory
So the situation we're in is that ?b is not in Col(A) and we wish to find x^ so that
Ax^, which is in Col(A), is as close as possible to ?b.
Here's a picture of the situation:
?b
Close As Possible!
o? Col(A)
This picture suggests that we can obtain a solution by projecting ?b onto Col(A)
to get PrCol(A)?b and then finding x^ to solve the equation:
Ax^ = PrCol(A)?b
Assuming this is correct, the problem with this approach, practically, is that
3


How to calculate least squares line?Method of Least Squares. In Correlation we study the linear correlation between two random variables x and y. We now look at the line in the xy plane that best fits the data (x1, y1), …, (xn, yn). Recall that the equation for a straight line is y = bx + a, where. b = the slope of the line.

• LEAST SQUARES SOLUTIONS - Mathematics
• Least Squares Solutions - College of Arts and Sciences
• Least Squares - UMD
• Solving Linear Least Squares with SVD - UIUC
• Least Squares Fitting of Data