tags math matrix algebra

matrix algebra

multiplication

$$AB=\left[\begin{array}{cc}1& 0\\ 2& 3\end{array}\right]\left[\begin{array}{cc}a& b\\ c& d\end{array}\right]$$

dot product

$$AB=\left[\begin{array}{cc}1& 0\\ 2& 3\end{array}\right]\left[\begin{array}{cc}a& b\\ c& d\end{array}\right]=\left[\begin{array}{cc}1a+0c& 1b+0d\\ 2a+3c& 2b+3d\end{array}\right]$$

by columns

$$AB=\left[\begin{array}{cc}1& 0\\ 2& 3\end{array}\right]\left[\begin{array}{cc}a& b\\ c& d\end{array}\right]=\left[\begin{array}{ccc}a\left[\begin{array}{c}1\\ 2\end{array}\right]+c\left[\begin{array}{c}0\\ 3\end{array}\right]& ,& b\left[\begin{array}{c}1\\ 2\end{array}\right]+d\left[\begin{array}{c}0\\ 3\end{array}\right]\end{array}\right]$$

by row

$$AB=\left[\begin{array}{cc}1& 0\\ 2& 3\end{array}\right]\left[\begin{array}{cc}a& b\\ c& d\end{array}\right]=\left[\begin{array}{c}1\left[\begin{array}{cc}a& b\end{array}\right]+0\left[\begin{array}{cc}c& d\end{array}\right]\\ 2\left[\begin{array}{cc}a& b\end{array}\right]+3\left[\begin{array}{cc}c& d\end{array}\right]\end{array}\right]$$

by block

$$AB=\left[\begin{array}{cc}1& 0\\ 2& 3\end{array}\right]\left[\begin{array}{cc}a& b\\ c& d\end{array}\right]=\left[\begin{array}{c}1\\ 2\end{array}\right]\left[\begin{array}{cc}a& b\end{array}\right]+\left[\begin{array}{c}0\\ 3\end{array}\right]\left[\begin{array}{cc}c& d\end{array}\right]$$

matrix form of gaussian elimination

actually the row reduce from the there are matrix multiplication as well

Subtracting 2 times the $1^{st}$ row of A from the $2^{nd}$ row of A

$$\begin{gathered} \{EA\}\to A^{+} \\ \left[\begin{array}{ccc}1& 0& 0\\ -2& 1& 0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{ccc}{x}_{11}& x_{12}& x_{13}\\ x_{21}& x_{22}& x_{23}\\ x_{31}& x_{32}& x_{33}\end{array}\right]=\left[\begin{array}{ccc}{x}_{11}& x_{12}& x_{13}\\ x_{21}-2x_{11}& x_{22}-2x_{21}& x_{23}-2x_{13}\\ x_{31}& x_{32}& x_{33}\end{array}\right] \end{gathered}$$

for example

$$\begin{array}{rl}& \\ \{EA\}& \to A^{+}\\ \left[\begin{array}{ccc}1& 0& 0\\ -2& 1& 0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{ccc}2& 1& 1\\ 4& -6& 0\\ -2& 7& 2\end{array}\right]& \to \left[\begin{array}{ccc}2& 1& 1\\ 0& -8& -2\\ -2& 7& 2\end{array}\right]\end{array}$$

revert U to A

$G,F,E$ are elimination operation matrix

$$\begin{array}{rl}& \\ E^{-1}{F}^{-1}{G}^{-1}& =L\\ \left[\begin{array}{ccc}1& 0& 0\\ l\ast 21& 1& 0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ l\ast 31& 0& 1\end{array}\right]\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& l\ast 32& 1\end{array}\right]& =\left[\begin{array}{ccc}1& 0& 0\\ l\ast 21& 1& 0\\ l\ast 31& l\ast 32& 1\end{array}\right]\\ \left[\begin{array}{ccc}1& 0& 0\\ l\ast 21& 1& 0\\ l\ast 31& l\_32& 1\end{array}\right]& =L\end{array}$$

$$\begin{gathered} U=\left[\begin{array}{cccc}d\ast 1& & & \\ & d_2& & \\ & & \ddots & \\ & & & d_n\end{array}\right]\left[\begin{array}{cccc}1& \frac{u\ast 12}{d\ast 1}& \frac{u\ast 13}{d\ast 1}& \\ & 1& \frac{u\ast 23}{d_2}& \\ & & \ddots & \\ & & & 1\end{array}\right] \\ U=D{U}^{\ast } \end{gathered}$$

$L$ is Lower triangular matrix

$U$ is Upper triangular matrix

$$\begin{gathered} GFEA=U \\ {E}^{-1}{F}^{-1}{G}^{-1}U=A \\ \text{let }L=E^{-1}{F}^{-1}{G}^{-1} \\ LU=A \\ LD{U}^{\ast }=A \end{gathered}$$

$$\begin{array}{rl}Ax& =b\\ L^{-1}Ax& =L^{-1}b\\ Ux& =L^{-1}b\\ x& =U^{-1}{L}^{-1}b\end{array}$$

Symmetric matrix

$$\begin{gathered} A=A^T \\ A=LD{U}^{\ast }=LD{L}^{-1} \end{gathered}$$

projection matrix(project vector to columns space)

projection vector on vector

$$\begin{gathered} \text{project }\stackrel{\rightharpoonup }{b}\text{ on }\stackrel{\rightharpoonup }{a}\text{ as }\hat{a} \\ \stackrel{\rightharpoonup }{(b-\hat{a})}\perp \stackrel{\rightharpoonup }{a}\to a^T(b-\hat{a})=0 \\ \hat{a}=x\times \stackrel{\rightharpoonup }{a} \\ \begin{array}{rl}{a}^T(b-\hat{a})& =0\\ a^{T}b-a^T\hat{a}& =0\\ a^T(\hat{a})& =a^{T}b\\ a^T(x\times a)& =a^{T}b\\ x\times (a^{T}a)& =a^{T}b\\ x& =\frac{a^{T}b}{a^{T}a}\end{array} \\ \begin{array}{rl}\hat{a}& =x\times \stackrel{\rightharpoonup }{a}\\ & =\frac{a^{T}b}{a^{T}a}\times \stackrel{\rightharpoonup }{a}\\ & =\underset{P}{\underbrace{(\frac{a{a}^T}{a^{T}a})}}\stackrel{\rightharpoonup }{b}\end{array} \end{gathered}$$

projection matrix

$$\begin{gathered} \hat{a}=Pa \\ a-Pa=(I-P)a \end{gathered}$$

property

• symmetric $$\begin{gathered} P=P^T \\ {P}^n=P \end{gathered}$$

Pseudoinverse matrix

$$\begin{array}{rl}& A^{+}(Ax)=x_r\\ & A^{+}=A^T(A{A}^T{)}^{-1}\\ & A{A}^{+}=I\end{array}$$

determinant

$$\begin{array}{rl}det(ab)& =det(a)det(b)\\ det(a^T)& =\frac{1}{det(a)}\end{array}$$

SVD

$$\begin{gathered} A=U\Sigma V^T \\ det(A^{T}A-\lambda I)=0 \\ (A^{T}A-\lambda I)V=0 \\ \Sigma =\text{diagonal}(\sqrt{\lambda })\text{sort by big to small} \\ V=\text{unit-length}(V) \\ {V}^{T}V=I \\ U=AV{\Sigma }^{-1} \\ {u}_i=\frac{1}{\sqrt{{\lambda }_i}}A{v}_i \end{gathered}$$

polar decomposition

$$\begin{array}{rl}A& =QS\\ A& =U\Sigma V^T\\ & =(U{V}^T)(V\Sigma V^T)\\ & =QS\end{array}$$