Eigenvalues and eigenvectors
Given an operator A, an eigenvector is a non-zero vector |v\rangle such that
A|v\rangle = \lambda|v\rangle
for some \lambda\in\mathbb{C} (which is called the corresponding eigenvalue).
We call the pair (\lambda,|v\rangle) an eigenpair, and we call the set of eigenvalues the spectrum of A, denoted by \sigma(A).
It is a surprising (but incredibly useful) fact that every operator has at least one eigenpair.
Geometrically, an eigenvector of an operator A is a vector upon which A simply acts by “stretching”.
Note that eigenvectors can be scaled by an arbitrary non-zero length: if A|v\rangle=\lambda|v\rangle then
\begin{aligned}
A(\mu|v\rangle)
&= \mu(A|v\rangle)
\\&= \mu\lambda|v\rangle
\\&= \lambda(\mu|v\rangle)
\end{aligned}
for any \mu\neq0.
Because of this, we usually assume all eigenvectors to be of length 1.
Rewriting the defining property of an eigenpair (\lambda,|v\rangle), we see that
(A-\lambda\mathbf{1})|v\rangle = 0
which tells us that the operator A-\lambda\mathbf{1} has a non-zero kernel, and is thus non-invertible.
This gives a useful characterisation of the spectrum in terms of a determinant:
\sigma(A) = \{\lambda\in\mathbb{C} \mid \det(A-\lambda\mathbf{1})=0\}.
The spectrum \sigma(A) allows us to recover information about the operator A.
For example, the trace of A is equal to the sum of all its eigenvalues, and the determinant of A is equal to the product of all its eigenvalues.
We can show this easily for normal operators using the fact that their eigenvectors are orthogonal: they satisfy \langle v|w\rangle=0 for v\neq w.
Because eigenvectors can always be scaled, this means that we can assume the eigenvectors of a normal operator to be orthonormal.
If we write the eigenpairs as (\lambda_i,|v_i\rangle), we can define
U
= \sum_i |i\rangle\langle v_i|
for an orthonormal basis \{|1\rangle,\ldots,|n\rangle\}, which is an orthogonal matrix (since the eigenvectors are also assumed to be orthonormal).
Then we see that
\begin{aligned}
UAU^\dagger
&= \sum_{i,j} |i\rangle\langle v_i| A |v_j\rangle\langle j|
\\&= \sum_{i,j} |i\rangle\langle v_i| \lambda_j|v_j\rangle\langle j|
\\&= \sum_{i,j} \lambda_j|i\rangle(\langle v_i|v_j\rangle)\langle j|
\\&= \sum_i \lambda_i |i\rangle\langle i|
\end{aligned}
(where we again use this hypothesis that the eigenvectors of A are all orthonormal) which is the diagonal matrix D consisting of the eigenvalues \lambda_i of A along its diagonal.
Then, since UAU^\dagger=D, we can equally write A=U^\dagger DU, which gives us
A
= \sum_i \lambda_i |v_i\rangle\langle v_i|.
We call each \lambda_i|v_i\rangle\langle v_i| the eigenspace projector, since a projector is defined to be any operator P that satisfies P=P^\dagger and P^2=P.
Note that projectors can only have eigenvalues equal to 0 or 1, since if |v\rangle is an eigenvector of P then, using the fact that P^2=P,
\begin{aligned}
0
&= (P^2-P)|v\rangle
\\&= (\lambda^2-\lambda)|v\rangle
\\\implies \lambda(\lambda-1)=0
\end{aligned}
and so \lambda must be equal to either 0 or 1.
Finally we can now return to the relationship between eigenvalues and the trace and determinant, using this fact that any normal operator A gives a unitary operator U such that A=U^\dagger DU for the diagonal matrix D of eigenvalues of A.
Indeed,
\begin{aligned}
\operatorname{tr}(A)
&= \operatorname{tr}(U^\dagger DU)
\\&= \operatorname{tr}(DUU^\dagger)
\\&= \operatorname{tr}(D)
\\&= \sum_i \lambda_i
\end{aligned}
which proves that the trace is equal to the sum of the eigenvalues, and
\begin{aligned}
\det(A)
&= \det(U^\dagger DU)
\\&= \det(U^\dagger)\det(D)\det(U)
\\&= \det(D)\det(U^\dagger)\det(U)
\\&= \det(D)\det(U^\dagger U)
\\&= \det(D)
\\&= \prod_i \lambda_i
\end{aligned}
which proves that the determinant is equal to the product of the eigenvalues.
The eigenspace projectors give us the spectral decomposition of A, which is where we write
A
= \sum_i \lambda_i|v_i\rangle\langle v_i|.
The spectral decomposition of a normal operator gives an effective way of calculating the action of a function on a matrix.
If f\colon\mathbb{C}\to\mathbb{C} is a function, then we can define
f(A)
= \sum_i f(\lambda_i)|v_i\rangle\langle v_i|.
For example, if f(x)=x^2, then, by this definition, f(A)=\sum_i \lambda_i^2|v_i\rangle\langle v_i|.
But this is consistent with the definition of A^2 that you expect:
\begin{aligned}
A^2
&= \left(\sum_i \lambda_i|v_i\rangle\langle v_i|\right)\left(\sum_i \lambda_i|v_i\rangle\langle v_i|\right)
\\&= \sum_{i,j} \lambda_i\lambda_j |v_i\rangle\langle v_i||v_j\rangle\langle v_j|
\\&= \sum_i \lambda_i^2 |v_i\rangle\langle v_i|
\end{aligned}
using the fact that the eigenvectors |v_i\rangle are orthonormal and that projectors P=|v_i\rangle\langle v_i| satisfy P^2=P.