I heard my math teacher mentioning that a matrix is an operator. This confused me, so I looked up what an operator was. In simple terms, it is a function that maps from one space to another space. Examples I can think up are as follows:
\begin{equation} f(x) = x^{2} \end{equation}This function takes some number in the set of all complex numbers, and outputs another number in the set of all complex numbers, and is thus an operator.
\begin{equation} \frac{\mathrm{d}}{\mathrm{d}x} f(x) = \frac{\mathrm{d}}{\mathrm{d}x} (x^{2} + 2x + 3) \end{equation}The $\mathrm{d}/\mathrm{d}x$ here is an operator as it takes a function from a set of functions that may have complex coefficients, and outputs another function that may have complex coefficients.
So I understand why both of these are "operators". But let's take a look at a matrix:
\begin{equation} \mathbb{M} = \begin{pmatrix} 3 & 2 \\ 4 & 1 \end{pmatrix} \end{equation}
How can this be called an operator? It's not taking any element from a set and then outputting an element from another set. I think of it as similar to a scalar, like $3$—is it not an operator, but you can use an operator, such as addition or multiplication on scalars, and the addition or multiplication can be considered an operator.
Why would a matrix such as $\mathbb{M}$ be called an operator? Can any $n$-rank tensor then be called an operator as well if a matrix is an operator?
$\endgroup$ 62 Answers
$\begingroup$In some sense you are correct that a matrix is not an operator on its own in the sense that you define it. Indeed, a matrix is nothing more than an array of numbers. However, we typically identify a matrix $A\in \mathcal{M}_{n\times m}(\Bbb{R})$ with the associated mapping $\Bbb{R}^m\to \Bbb{R}^n$ it defines by left multiplication. In this way it becomes an operator in the sense you have defined in a canonical fashion. For instance, your matrix $M$ defines a transformation $\Bbb{R}^2\to \Bbb{R}^2$ by$$ \begin{bmatrix} 3&2\\ 4&1 \end{bmatrix} \begin{bmatrix} x_1\\ x_2 \end{bmatrix}= \begin{bmatrix} 3x_1+2x_2\\ 4x_1+x_2 \end{bmatrix} .$$That is $(x_1,x_2)\mapsto (3x_1+2x_2, 4x_1+x_2)$.
$\endgroup$ 3 $\begingroup$I would like to come straight to the essence.
The term "operator" is usually used for a mapping between vectors, but the most common meaning is mapping from one space of functions to another space of functions (note that functions often form a vector space). I say "usually" because it is the most common meaning, and what really differentiates an operator from a multi-variable function.
Then there are linear operators and non-linear operators. A linear operator $A$ has the properties $A[cf]=cA[f]$ and $A[f+g]=A[f]+A[g]$.
Example of a linear operator:
$A[f](x)=\int_0^x f(t)dt$
(area under the curve)
Example of a non-linear operator:
$A[f](x)=\int _{a}^{x}{\sqrt {1+f'(t)^{2}}}\,dt$
(length of curve).
Now, we can consider an (analytic) function as a vector, generally of infinite dimension (Hilbert space) with components of the vector being the coefficients of its Taylor series expansion.
It turned out that many linear operators on functions can be represented as infinite matrices, and their action as multiplying the matrix on the Taylor series of the function.
For instance, consider the horizontal scaling operator:
$$Uf(x)=f(k x)= f(0)+\frac {f'(0)}{1!} k x+ \frac{f''(0)}{2!} k^2 x^2+\frac{f^{(3)}(0)}{3!} k^3 x^3+ \cdots+\frac{f^{(n)}(0)}{n!} k^n x^n +\cdots.$$
Its action on the Taylor series of the function is similar to multiplication by the following matrix according to the matrix multiplication rules:
$$U=\left( \begin{array}{cccccc} 1 & 0 & 0 & . & 0 & . \\ 0 & k^1 & 0 & . & 0 & . \\ 0 & 0 & k^2 & . & 0 & . \\ . & . & . & . & . & . \\ 0 & 0 & 0 & . & k^n & . \\ . & . & . & . & . & . \end{array} \right)$$
The vertical columns of the matrix can be obtained by applying the operator to the basis vectors (e.g., functions that have only one term in their Taylor series expansion). These columns are called eigenvectors of the operator.
Here are matrices of some other operators acting on analytic functions:
Derivative:
$D=f'(x)=\left( \begin{array}{cccccc} 0 & 1 & 0 & 0 & 0 & . \\ 0 & 0 & 2 & 0 & 0 & . \\ 0 & 0 & 0 & 3 & 0 & . \\ 0 & 0 & 0 & 0 & 4 & . \\ 0 & 0 & 0 & 0 & 0 & . \\ . & . & . & . & . & . \\ \end{array} \right) f(x)$
Integral:
$\int_0^xf(t)dt=\left( \begin{array}{cccccc} 0 & 0 & 0 & 0 & 0 & . \\ 1 & 0 & 0 & 0 & 0 & . \\ 0 & 2 & 0 & 0 & 0 & . \\ 0 & 0 & 3 & 0 & 0 & . \\ 0 & 0 & 0 & 4 & 0 & . \\ . & . & . & . & . & . \\ \end{array} \right)f(x)$
Shift:
$f(x+1)=\left( \begin{array}{cccccc} 1 & 1 & 1 & 1 & 1 & . \\ 0 & 1 & 2 & 3 & 4 & . \\ 0 & 0 & 1 & 3 & 6 & . \\ 0 & 0 & 0 & 1 & 4 & . \\ 0 & 0 & 0 & 0 & 1 & . \\ . & . & . & . & . & . \\ \end{array} \right)f(x),$
and so on.
Moreover, it turned out that one can combine the operators using matrix algebra, so to construct new operators. For instance,
$f(x+a)=e^{aD} f(x)$
$\Delta f(x) = (e^{D}-1) f(x)$
$\sum f(x) = \frac1{e^{D}-1} f(x)$
$\int f(x) dx= \frac1D$
$f(ax)=a^{xD}f(x)$
${\hat {f}}(x )={\frac {1}{\sqrt {2\pi }}}\int _{-\infty }^{\infty }f(t)e^{-ix t}\,dt=e^{\frac{1}{4} \pi i \left(D^2-x^2+1\right)} f(x)$
$\endgroup$
