Linear Equations and Systems of Linear Equations

This page is intended to be a part of the Numerical Analysis section of Math Online. Similar topics can also be found in the Linear Algebra section of the site.

# Linear Equations and Systems of Linear Equations

In many applications of mathematics, often times we need to find a solution that satisfies a set of linear equations in order to solve a problem in the physical sciences, economics, business, etc… Sometimes there can be thousands of equations and thousands of knowns to solve for. Being able to solve such equations is thus a tedious but very important task. We will formally define a system of linear equations below.

 Definition: A Linear Equation of $n$ Variables, $x_1, x_2, ..., x_n$ is one that can be written in the form $a_1 x_1 + a_2x_2 + ... + a_nx_n = b_1$ where $a_1, a_2, ..., a_n, b_1$ are constants. A System of $m$ Linear Equations and $n$ Variables, $x_1, x_2, ..., x_n$ is a set of linear equations that is often written in the form, $\left\{\begin{matrix} a_{11}x_1 + a_{12}x_2 + ... + a_{1n}x_n = b_1 \\a_{21}x_1 + a_{22}x_2 + ... + a_{2n}x_n = b_2 \\ \vdots \quad \quad \quad \vdots \quad \quad \: \: \quad \vdots \quad \quad \vdots \: \: \: \\ a_{m1}x_1 + a_{m2}x_2 + ... + a_{mn}x_n = b_m \end{matrix}\right.$ for constants $a_{ij}$ from $i = 1, 2, ..., m$ and $j = 1, 2, ..., n$

For example, a relatively simple system of equations to solve is the following system of two equations of the two variables $x$ and $y$:

(1)
\begin{align} \left\{\begin{matrix} 2x + 3y = 4 \\ x + y = 1 \end{matrix}\right. \end{align}

We can easily solve this linear system by hand. Take the second equation and rewrite it as $y = 1 - x$. Then substitute this into the first equation and solve for $x$:

(2)
\begin{align} \quad 2x + 3(1 - x) = 4 \Leftrightarrow 2x + 3 - 3x = 4 \Leftrightarrow x = -1 \end{align}

Then take the value $x = -1$ and plug it back into either of the two equations. In this case, plugging $x = -1$ into the second equation is much simpler and we get that $y = 2$. Therefore $(x, y) = (-1, 2)$ is a solution to our system of equations. Geometrically, a system of $m$ linear equations of $n$ unknowns is a point in $\mathbb{R}^n$ for which the graphs of all $m$ linear equations intersect each other.

Of course, a system of linear equations need not have a solution. Furthermore, a system of linear equations may have infinitely many solutions. We will look more into the solutions of linear equations later on.

One useful way to represent a linear system of equations is by constructing a coefficient matrix $A$ with $m$ columns and $n$ rows that contain the coefficients of the linear system. For example, if we have the following linear system of equations:

(3)
\begin{align} \left\{\begin{matrix} a_{11}x_1 + a_{12}x_2 + ... + a_{1n}x_n = b_1 \\a_{21}x_1 + a_{22}x_2 + ... + a_{2n}x_n = b_2 \\ \vdots \quad \quad \quad \vdots \quad \quad \: \: \quad \vdots \quad \quad \vdots \: \: \: \\ a_{m1}x_1 + a_{m2}x_2 + ... + a_{mn}x_n = b_m \end{matrix}\right. \end{align}

Then the corresponding coefficient matrix that represents this system is:

(4)
\begin{align} A = \begin{bmatrix} a_{11} & a_{12} & \cdots & a_{1n}\\ a_{21} & a_{22} & \cdots & a_{2n}\\ \vdots & \vdots & \ddots & \vdots\\ a_{m1} & a_{m2} & \cdots & a_{mn} \end{bmatrix} \end{align}

Furthermore, we can represent the set of variables $x_1, x_2, ..., x_n$ as an $n \times 1$ column matrix (or column vector) $x = \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix}$ and the set of constants $b_1, b_2, ..., b_n$ as an $n \times 1$ column matrix $b = \begin{bmatrix} b_1 \\ b_2 \\ \vdots \\ b_n \end{bmatrix}$ and represent the system of equations with respect to the variables $x_1, x_2, ..., x_n$ and the constants $b_1, b_2, ..., b_n$ as $Ax = b$, or in matrix form:

(5)
\begin{align} \quad \begin{bmatrix} a_{11} & a_{12} & \cdots & a_{1n}\\ a_{21} & a_{22} & \cdots & a_{2n}\\ \vdots & \vdots & \ddots & \vdots\\ a_{m1} & a_{m2} & \cdots & a_{mn} \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix} = \begin{bmatrix} b_1 \\ b_2 \\ \vdots \\ b_n \end{bmatrix} \end{align}

We will see later that this form is particularly convenient for representing systems, especially when it comes to using computers to solve them where solutions exist.