Definition: A positive definite binary quadratic form $f(x, y) = ax^2 + bxy + cy^2$ is said to be Reduced if one of the following inequalities hold: 1) $-a < b \leq a < c$. 2) $0 < b \leq a = c$.

Let:

(1)
\begin{align} \quad M_1 = \begin{bmatrix} 0 & -1\\ 1 & 0 \end{bmatrix} \quad , \quad M_2 = \begin{bmatrix} 1 & m\\ 0 & 1 \end{bmatrix} \end{align}

Observe that $\det (M_1) = 1$ and $\det (M_2) = 1$. If $f(x, y) = ax^2 + bxy + cy^2$ then under $M_1$ we have that:

(2)
\begin{align} \quad f(0x -1y, 1x + 0y) &= f(-y, x) \\ &= a(-y)^2 + b(-y)(x) + c(x)^2 \\ &= ay^2 -bxy + cx^2 \\ &= cx^2 -bxy + ay^2 \\ &= a'x^2 + b'xy + c'y^2 \end{align}

where $a' = c$, $b' = -b$, and $c' = a$. So if $a > c$ then $a' = c < a = c'$. So if $f(x, y)$ is such that the coefficient of $x^2$ is greater than the coefficient of $y^2$ then applying $M_1$ to $f(x, y)$ gives us an equivalent binary quadratic form where the coefficient of $x^2$ is less than the coefficient of $y^2$.

Again, if $f(x, y) = ax^2 + bxy + cy^2$ then under $M_2$ we have that:

(3)
\begin{align} \quad f(1x + my, 0x + 1y) &= f(x + my, y) \\ &= a(x + my)^2 + b(x + my)(y) + c(y)^2 \\ &= a(x^2 + 2mxy + m^2y^2) + b(xy + my^2) + cy^2 \\ &= ax^2 + 2amxy + am^2y^2 + bxy + bmy^2 + cy^2 \\ &= ax^2 + (2am + b)xy + (am^2 + bm + c)y^2 \\ &= a'x^2 + b'xy + c'y^2 \end{align}

where $a' = a$, $b' = 2am + b$, and $c' = am^2 + bm + c$. If $b > a$ then $m$ can be chosen such that:

(4)
\begin{align} \quad -a' = -a < 2am + b = b' < a = a' \end{align}

So if $f(x, y)$ is such that the coefficient of $xy$ is greater than the coefficient of $x^2$ then applying $M_2$ to $f(x, y)$ gives us an equivalent binary quadratic form such that the coefficient of $xy$ is less than the coefficient of $x^2$.

 Theorem 1: Let $f(x, y) = ax^2 + bxy + cy^2$ be a reduced binary quadratic form with discriminant $d \in \mathbb{Z}$ where $d$ is not a perfect square. a) If $f$ is indefinite then $0 < |a| \leq \frac{1}{2} \sqrt{d}$. b) If $f$ is positive definite then $0 < a \leq \sqrt{\frac{-d}{3}}$.
• Proof of a): Suppose that $f$ is indefinite and is a reduced binary quadratic form. Then $d > 0$ and $-a < b \leq a < c$. If $a$ and $c$ have the same signs then $ac > 0$. So $ac > a^2$. Hence:
(5)
\begin{align} \quad d = b^2 - 4ac \leq a^2 - 4ac \leq a^2 - 4a^2 = -3a^2 < 0 \end{align}
• Which is a contradiction. So $a$ and $c$ must have different signs, and hence $-4ac = 4|ac|$. Thus:
(6)
\begin{align} \quad d = b^2 - 4ac = b^2 + 4|ac| \geq 4|ac| \geq 4a^2 \end{align}
• Dividing both sides by $4$ and taking the squareroot of both sides of the inequality gives us:
(7)
\begin{align} \quad |a| \leq \frac{1}{2} \sqrt{d} \end{align}
• If $a = 0$ then the inequality $0 < b \leq a = c$ cannot be satisfied, and so:
(8)
• Proof of b) Suppose that $f$ is positive definite and is a reduced binary quadratic form. Then $d < 0$ and $a, c > 0$, and $-a < b \leq a < c$. So:
• Dividing both sides by $-3$ and taking the squareroot of both sides gives us that:
• The absolute value bars on $a$ are removed since $a > 0$, and so: