Our aim is to show that the group of orientation-preserving isometries of \mathbb{R}^3, that is

SO(3) = \{ M \in GL(3,\mathbb{R}) \mid MM^T= \mathrm{Id}, \det(M) =1 \},

is a simple group. Here we use a geometric point of view, mixing the proofs that can be found in Stillwell’s book, Naive Lie theory, and Perrin’s book, Cours d’algèbre. We begin with an easy structure lemma:

Lemma 1: If M \in SO(3), there exists an orthonormal basis in which M may be written as

\left( \begin{matrix} 1 & 0 & 0 \\ 0 & \cos(\theta) & -\sin(\theta) \\ 0 & \sin(\theta) & \cos(\theta) \end{matrix} \right)

for some \theta \in [0,2 \pi).

Proof. First, notice that

\det(M- \mathrm{Id})= \det( M- \ MM^T)= \det(M) \cdot \det( \mathrm{Id}- M) = (-1)^3 \det( M- \mathrm{Id}),

hence \det(M- \mathrm{Id})=0 ie. there exists v \in \mathbb{R}^3 \backslash \{ 0 \} such that Mv=v. Let W=v^{\bot}. Then for all w \in W,

\langle Mw,v \rangle = \langle w, Mv \rangle = \langle w,v \rangle=0,

so W is stable under M. Therefore, M may be written as \left( \begin{matrix} 1 & 0 \\ 0 & S \end{matrix} \right) in an orthonormal basis; in particular, because MM^T= \mathrm{Id} and \det(M)=1, S \in SO(2).

If S= \left( \begin{matrix} a & b \\ c & d \end{matrix} \right), the equality S^T S = \mathrm{Id} becomes \left\{ \begin{array}{l} a^2+c^2=1 \\ b^2+d^2=1 \\ ab+cd=0 \\ ad-bc=1 \end{array} \right.. The two first equations allow us to set \left\{ \begin{array}{l} a= \cos(\theta), \ c= \sin(\theta) \\ d = \cos( \varphi), \ b= \sin(\theta) \end{array} \right. for some \theta, \varphi \in [0,2\pi). Then, the two last equations give

\left\{ \begin{array}{l} \cos(\theta) \sin(\varphi) + \sin(\theta) \cos(\varphi)=0 \\ \cos(\theta) \cos(\varphi)- \sin(\varphi) \sin(\theta) = 1 \end{array} \right., that is \left\{ \begin{array}{l} \sin(\theta+ \varphi)=0 \\ \cos( \theta+ \varphi)=1 \end{array} \right..

Therefore, \varphi=- \theta ~ \text{mod} ~ 2\pi and M may be written as the mentionned matrix. \square

Definition: If M is as in the lemma, we say that M is a rotation of axis v and angle \theta.

Lemma 2: Let M \in SO(3) and P \in SO(3). Suppose that M is a rotation of axis v. Then PMP^{-1} is a rotation of axis Pv.

Proof. Let W = v^{\bot}. Then PMP^{-1}(Pv)=Pv and PW=(Pv)^{\bot} is stable under PMP^{-1}. Therefore, according to lemma 1, PMP^{-1} is a rotation of axis Pv. \square

Lemma 3: Following the notations given on the figure below, the rotation around P of angle \theta composed with the rotation around Q of angle \varphi is a rotation around R whose angle equals to twice the angle at R of the spherical triangle PQR.


Proof. Let H_1,H_2 be two planes intersecting along a line D. Let r_1,r_2 be the reflections with respect to H_1,H_2 respectively.  Then r_1r_2 fixes D and it is not difficult to see that the restriction of r_1r_2 to the plane normal to D is a rotation of angle twice the angle \theta between H_1 and H_2. Therefore, according to lemma 1, r_1r_2 is a rotation of axis D and of angle 2\theta.

Thus, if R_{\varphi}, R_{\theta} denotes the rotations around Q, P of angles \varphi,\theta respectively, and if r_{\mathcal{L}}, r_{\mathcal{M}}, r_{\mathcal{N}} denotes the reflections with respect to \mathcal{L},\mathcal{M}, \mathcal{N} respectively, we can write

R_{\varphi}=r_{\mathcal{N}} r_{\mathcal{M}} and R_{\theta}= r_{\mathcal{M}} r_{\mathcal{L}},

hence R_{\varphi} R_{\theta}=r_{\mathcal{N}} r_{\mathcal{L}}, that is R_{\varphi} R_{\theta} is a rotation around R of angle twice the angle at R of the spherical triangle PQR. \square

Now, we give two lemmas to characterize the natural action of SO(3) on the sphere \mathbb{S}^2.

Lemma 4: The action SO(3) \curvearrowright \mathbb{S}^2 is transitive, ie. for every x,y \in \mathbb{S}^2 there exists g \in SO(3) such that g \cdot x =y.

In fact, it is a special case of the following lemma, which states that the action SO(3) \curvearrowright \mathbb{S}^2 is “as 2-transitive as possible”.

Lemma 5: The action SO(3) \curvearrowright \mathbb{S}^2 is isometrically 2-transitive, ie. for every x_1,x_2,y_1,y_2 \in \mathbb{S}^2 satisfying \| x_1 -y_1 \| = \| x_2 - y_2 \| there exists g \in SO(3) such that g \cdot x_1=x_2 and g \cdot y_1 = y_2.

Proof. Let R_1 be a rotation sending x_1 to x_2, and R_2 be a rotation of axis x_2=Rx_1 sending R_1y_1 to y_2. Then R := R_2 \circ R_1 \in SO(3) and \left\{ \begin{array}{l} Rx_1= R_2(R_1x_1)=R_2x_2=x_2 \\ Ry_1= R_2(R_1y_1)=y_2 \end{array} \right. . \square

Theorem: SO(3) is a simple group.

Proof. Let N \lhd SO(3) be a non-trivial normal subgroup. We first want to show that the action N \curvearrowright \mathbb{S}^2 is transitive. Let u \in N \backslash \{ \mathrm{Id} \} be a rotation of axis \mathbb{R}a for some a \in \mathbb{S}^2 and let x \in a^{ \bot} \cap \mathbb{S}^2. Because u \neq \mathrm{Id}, u(x) \neq x, so d:= \| x-u(x) \| >0.


Notice that u sends the meridian L passing through x to the meridian passing through u(x), so \|z-u(z) \| runs over [0,d] when z runs over L: \|z -u(z) \| tends to 0 (resp. d) when z tends to the north pole (resp. to the equator). More precisely, if 0 \leq m \leq d, it is possible to find so \lambda \geq 0 such that

\displaystyle \frac{ \| u(x+ \lambda a) - (x+ \lambda a) \| }{ \| x+ \lambda a \| } =m.

Because u is linear and x \in a^{\bot}, the above equation is equivalent to

\displaystyle d=m \| x+ \lambda a \| = \sqrt{ \|x\|^2 + \lambda^2 \| a \|^2} = \sqrt{1+ \lambda^2}, hence \displaystyle \lambda = \sqrt{ \frac{d^2}{m^2} -1 }.

Let v,w \in \mathbb{S}^2 such that \| v - w \| \leq d. We just saw that there exists y \in \mathbb{S}^2 such that \| u(y)-y \| = \|v-w \|. According to lemma 5, there exists g \in SO(3) such that g(v)=y and g(w)=u(y). Then gug^{-1} \in N because N is a normal subgroup and gug^{-1}(v)=w.

Therefore, we proved that the action N \curvearrowright \mathbb{S}^2 is transitive “for the small distances”. Now, if v,w \in \mathbb{S}^2 are any points, clearly there exists a sequence of points v=p_0,p_1,\dots,p_n=w \in \mathbb{S}^2 satisfying \| p_i - p_{i+1} \| \leq d for all 0 \leq i \leq n-1. In particular, for all 0 \leq i \leq n-1, there exists u_i \in N such that u_i(p_i)=p_{i+1}, and finally u_{n-1} \cdots u_0 is an element of N sending v to w. Thus, the action N \curvearrowright \mathbb{S}^2 is transitive.

In particular, there exist x \in \mathbb{S}^2 and u \in N such that u(x)=-x; such an element of SO(3), with -1 as an eigenvalue, is called a codimension-two reflection. Clearly, using lemmas 1 and 2, two codimension-two reflections are always conjugated. Therefore, since N is a normal subgroup, any codimension-two reflection belongs to N.  We deduce, according to lemma 3 and using the notations of the figure below, that the rotation of angle \theta around R belongs to N.



Clearly, when P runs over the equator, \theta runs over [0, \pi). Therefore, N contains rotations of any angle. Because two rotations of the same angle are conjugated and N is a normal subgroup, we deduce that any rotation belongs to N, that is N= SO(3). \square

Nota Bene: The conclusion of our proof, based on lemma 3, may be replaced with an algebraic argument, stating that the codimension-two reflections generate SO(3). Therefore, in order to show that a normal subgroup is SO(3), it is sufficient to find only one codimension-two reflection belonging to it.

Another classical proof is to find a codimension-two reflection in a nontrivial normal subgroup H by introducing the map

\varphi : \left\{ \begin{array}{ccc} SO(3) & \to & \mathbb{R} \\ g & \mapsto & \mathrm{tr}(ghg^{-1}h^{-1}) \end{array} \right.

for some fixed h \in H \backslash \{ \mathrm{Id} \}. By connectedness, \mathrm{Im}(\varphi)= [a,3] for some a \leq 3, because for all g \in SO(3) there exists \theta \in [0,2\pi] such that \mathrm{tr}(g)=1+2\cos(\theta) (lemma 1).  If a=3, then [g,h]= \mathrm{Id} for all g \in SO(3), that is h is in the center of SO(3), which is trivial. Therfore, a<3 and there exist n \geq 1 and g_n \in SO(3) such that

a \leq \varphi(g_n) = 1+2 \cos \left( \frac{\pi}{n} \right) < 3.

Then h_n:= g_nhg_n^{-1}h^{-1} \in H is a rotation of angle \pi/n and h_n^n is a codimension-two reflection.

Finally, a proof based on the isomorphism SU(2)/ \mathbb{Z}_2 \simeq SO(3) exhibit in Fundamental group of SO(3) and Quaternions can be found in Artin’s book, Algebra.