This page is intended to be a part of the Numerical Analysis section of Math Online. Similar topics can also be found in the Linear Algebra section of the site.

Linear Equations and Systems of Linear Equations

In many applications of mathematics, often times we need to find a solution that satisfies a set of linear equations in order to solve a problem in the physical sciences, economics, business, etc… Sometimes there can be thousands of equations and thousands of knowns to solve for. Being able to solve such equations is thus a tedious but very important task. We will formally define a system of linear equations below.

Definition: A Linear Equation of $n$ Variables, $x_1, x_2, ..., x_n$ is one that can be written in the form $a_1 x_1 + a_2x_2 + ... + a_nx_n = b_1$ where $a_1, a_2, ..., a_n, b_1$ are constants. A System of $m$ Linear Equations and $n$ Variables, $x_1, x_2, ..., x_n$ is a set of linear equations that is often written in the form, $\left\{\begin{matrix} a_{11}x_1 + a_{12}x_2 + ... + a_{1n}x_n = b_1 \\a_{21}x_1 + a_{22}x_2 + ... + a_{2n}x_n = b_2 \\ \vdots \quad \quad \quad \vdots \quad \quad \: \: \quad \vdots \quad \quad \vdots \: \: \: \\ a_{m1}x_1 + a_{m2}x_2 + ... + a_{mn}x_n = b_m \end{matrix}\right.$ for constants $a_{ij}$ from $i = 1, 2, ..., m$ and $j = 1, 2, ..., n$

For example, a relatively simple system of equations to solve is the following system of two equations of the two variables $x$ and $y$:

(1)

\begin{align} \left\{\begin{matrix} 2x + 3y = 4 \\ x + y = 1 \end{matrix}\right. \end{align}

We can easily solve this linear system by hand. Take the second equation and rewrite it as $y = 1 - x$. Then substitute this into the first equation and solve for $x$:

(2)

\begin{align} \quad 2x + 3(1 - x) = 4 \Leftrightarrow 2x + 3 - 3x = 4 \Leftrightarrow x = -1 \end{align}

Then take the value $x = -1$ and plug it back into either of the two equations. In this case, plugging $x = -1$ into the second equation is much simpler and we get that $y = 2$. Therefore $(x, y) = (-1, 2)$ is a solution to our system of equations. Geometrically, a system of $m$ linear equations of $n$ unknowns is a point in $\mathbb{R}^n$ for which the graphs of all $m$ linear equations intersect each other.

Of course, a system of linear equations need not have a solution. Furthermore, a system of linear equations may have infinitely many solutions. We will look more into the solutions of linear equations later on.

One useful way to represent a linear system of equations is by constructing a coefficient matrix $A$ with $m$ columns and $n$ rows that contain the coefficients of the linear system. For example, if we have the following linear system of equations:

(3)

\begin{align} \left\{\begin{matrix} a_{11}x_1 + a_{12}x_2 + ... + a_{1n}x_n = b_1 \\a_{21}x_1 + a_{22}x_2 + ... + a_{2n}x_n = b_2 \\ \vdots \quad \quad \quad \vdots \quad \quad \: \: \quad \vdots \quad \quad \vdots \: \: \: \\ a_{m1}x_1 + a_{m2}x_2 + ... + a_{mn}x_n = b_m \end{matrix}\right. \end{align}

Then the corresponding coefficient matrix that represents this system is:

(4)

\begin{align} A = \begin{bmatrix} a_{11} & a_{12} & \cdots & a_{1n}\\ a_{21} & a_{22} & \cdots & a_{2n}\\ \vdots & \vdots & \ddots & \vdots\\ a_{m1} & a_{m2} & \cdots & a_{mn} \end{bmatrix} \end{align}

Furthermore, we can represent the set of variables $x_1, x_2, ..., x_n$ as an $n \times 1$ column matrix (or column vector) $x = \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix}$ and the set of constants $b_1, b_2, ..., b_n$ as an $n \times 1$ column matrix $b = \begin{bmatrix} b_1 \\ b_2 \\ \vdots \\ b_n \end{bmatrix}$ and represent the system of equations with respect to the variables $x_1, x_2, ..., x_n$ and the constants $b_1, b_2, ..., b_n$ as $Ax = b$, or in matrix form:

(5)

\begin{align} \quad \begin{bmatrix} a_{11} & a_{12} & \cdots & a_{1n}\\ a_{21} & a_{22} & \cdots & a_{2n}\\ \vdots & \vdots & \ddots & \vdots\\ a_{m1} & a_{m2} & \cdots & a_{mn} \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix} = \begin{bmatrix} b_1 \\ b_2 \\ \vdots \\ b_n \end{bmatrix} \end{align}

We will see later that this form is particularly convenient for representing systems, especially when it comes to using computers to solve them where solutions exist.