1 Jacobian Matrix
1.1 Jacobian Matrix
In vector calculus the Jacobian matrix of a vector function \(f:\mathbb{R}^n\to \mathbb{R}^m\) is the matrix of all the first-order partial derivatives. If this matrix is square, that is \(n=m\), then the matrix determinant is called the Jacobian determinant. Intuitively, the Jacobian matrix is the natural generalization of the derivative / differential of functions to vector valued functions.
Let \(\boldsymbol{f}:\mathbb{R}^n\to \mathbb{R}^m\) be a vector function such that all first-order partial derivatives exist on \(\mathbb{R}^n\). Then the Jacobian matrix of \(\boldsymbol{f}\) denoted \(J_{\boldsymbol{f}}\) is the \(m \times n\) matrix whose \((i,j)\)-th entry is \(\frac{{\partial f_{i}}}{\partial x_{j}}\) \[ J_{\boldsymbol{f}}=\begin{bmatrix} \frac{{\partial \boldsymbol{f}}}{\partial x_{1}} & \cdots & \frac{{\partial \boldsymbol{f}}}{\partial x_{n}} \end{bmatrix}=\begin{bmatrix} \nabla ^Tf_{1} \\ \vdots \\ \nabla^T{f}_{m} \end{bmatrix}=\begin{bmatrix} \frac{{\partial f_{1}}}{\partial x_{1}} & \cdots & \frac{{\partial f_{1}}}{\partial x_{n}} \\ \vdots & \ddots & \vdots \\ \frac{{\partial f_{m}}}{\partial x_{1}} & \cdots & \frac{{\partial f_{m}}}{\partial x_{n}} \end{bmatrix}, \] where \(\nabla^Tf_{i}\) is the transpose (row vector) of the gradient of the \(i\)-th component.
Example (Jacobian in \(\mathbb{R}^2)\): Consider the function \(f(x,y)=(2x^2+3xy, 4y^2x+2x)\). We can compute the Jacobian as \[ J_{f}=\begin{bmatrix} 4x+3y & 3x \\ 4y^2+2 & 8xy \end{bmatrix}. \] Further, the Jacobian determinant is given by \[ \det(J_{f})=8xy(4x+3y)-3x(4y^2+2)=32x^2y+24xy^2-12xy^2-6x=32x^2y+12xy^2-6x. \]
The Jacobian is giving the change of the new \(x\) and \(y\) coordinates with respect to the old \(x\) and \(y\) coordinates.
The Jacobian of the gradient of a scalar function of several variables is known as the Hessian Matrix which is intuitively the second derivative of the function in question.
1.2 Jacobian and Best Linear Approximations
If the function \(\boldsymbol{f}\) is differentiable at a point \(\boldsymbol{x}^*\in \mathbb{R}^n\) then its differential is represented by \(J_{\boldsymbol{f}}(\boldsymbol{x}^*)\). In this case, the [[linear-transformation]] represented by \(J_{\boldsymbol{f}}(\boldsymbol{x})\) is the best linear approximation of \(\boldsymbol{f}\) near the point \(\boldsymbol{x}^*\) in the sense that \[ \boldsymbol{f}(\boldsymbol{x})-\boldsymbol{f}(\boldsymbol{x}^*)=J_{\boldsymbol{f}}(\boldsymbol{x}^*)(\boldsymbol{x}-\boldsymbol{x^*})+o(\lVert \boldsymbol{x}-\boldsymbol{x}^* \rVert );\quad \boldsymbol{x}\to \boldsymbol{x}^*, \] where \(o(\lVert \boldsymbol{x}-\boldsymbol{x}^* \rVert)\) denotes some quantity that is of inferior order to \(\lVert \boldsymbol{x}-\boldsymbol{x}^* \rVert\) and thus approaches zero much faster.
This approximation specializes to the approximation of a scalar function of a single variable by [[memory-calculus-taylor-series]] of degree one, namely \[ f(x)-f(x^*) = f'(x^*)(x-x^*)+o(x-x^*);\quad x \to x^*. \] Thus our statement that a Jacobian may be regarded as a kind of first-order derivative of a vector function.
1.3 Jacobian Chain Rule
Composable functions \(f:\mathbb{R}^n\to \mathbb{R}^m\) and \(g:\mathbb{R}^m\to \mathbb{R}^k\) satisfy the Chain Rule in that for \(x \in \mathbb{R}^n\) we have \[ J_{g\circ f}(x)=J_{g}(f(x))J_{f(x)}. \]
PROOF: ## Jacobian Inverse From the [[inverse-function-theorem]], the matrix inverse of the Jacobian of an [[bijective-function|invertible function]] \(f:\mathbb{R}^n\to \mathbb{R}^n\) is the Jacobian of the inverse. That is, the Jacobian matrix of the inverse function at some point \(x \in \mathbb{R}^n\) is given by \[ J_{f^{-1}}(x)=J_{f}^{-1}(f^{-1}(x)), \] and the Jacobian determinant is \[ \det(J_{f^{-1}}(x))= \frac{1}{\det(J_{f}(f^{-1}(x)))}. \] If the Jacobian is [[continuous-functions|continuous]] and nonsingular at the point \(x \in\mathbb{R}^n\) then \(f\) is invertible when restricted to some [[neighborhoods|neighborhood]] of \(x\). In other words, if the Jacobian determinant is non-zero at a point, then the function is locally invertible near this point.
1.4 Critical Points
If \(f : \mathbb{R}^n → \mathbb{R}^m\) is a differentiable function, a critical point of \(f\) is a point where the [[matrix-rank|rank]] of the Jacobian matrix is not maximal. This means that the rank at the critical point is lower than the rank at some neighbor point. In other words, let \(k\) be the maximal dimension of the open balls contained in the image of \(f\); then a point is critical if all minors of rank \(k\) of \(f\) are zero.