machine-learing

resnet

residual block(skip-connection)

transformer

attention

self attention

transformer encoder

cross attention

transformer decoder

causal attention

CLIP

class Clip(nn.Module):
    def __init__(self,motion_dim=75,music_dim=438,feature_dim=256):
        super(Clip, self).__init__()

        self.motion_encoder = MotionEncoder(input_channels=motion_dim,feature_dim=feature_dim)
        self.music_encoder = MusicEncoder(input_channels=music_dim,feature_dim=feature_dim)

        self.motion_project = nn.Linear(feature_dim, feature_dim)
        self.music_project = nn.Linear(feature_dim, feature_dim)

        self.temperature = nn.Parameter(torch.tensor(1.0))

        self.criterion = nn.CrossEntropyLoss()

    def forward(self, motion:Tensor, music:Tensor):
        assert motion.shape[1] == music.shape[1]

        b,s,c= motion.shape

        motion_features = self.motion_encoder(motion)
        music_features = self.music_encoder(music)

        motion_features =F.normalize( self.motion_project(motion_features),p=2,dim=-1)
        music_features = F.normalize( self.music_project(music_features),p=2,dim=-1)


        # relation=(motion_features@music_features.T)*(1.0 / math.sqrt(c))

        # batch matrix multiplication and .mT is batch transpose matrix
        logits=torch.bmm(motion_features,music_features.mT)*self.temperature

        labels=torch.arange(s).repeat(b,1).to(motion.device)

        loss_motion = self.criterion(logits, labels)
        loss_music = self.criterion(logits.mT, labels)

        loss=(loss_motion+loss_music)/2

        return (motion_features,music_features),(loss,loss_motion,loss_music)

CAN(GAN base)

AE

VAE

reparameterization trick

Assuming the distribution of the output is a Gaussian distribution, the model only predicts the mean ($\mu$) and std ($\sigma$). We then sample the latent variable from this Gaussian distribution. The sample latent distribution parameters should match the true distribution, which is enforced using the KL divergence.

VAE loss

Evidence Lower Bound (ELBO)

We want replace $p_{\theta }(z|x)$ with $q_{\varphi }(z)$ since we dont have GT of $p_{\theta }(z|x)$

$$\begin{array}{rl}\log p_{\theta }(x)& =E_{(q_{\varphi })}[\log p_{\theta }(x)]\\ & =E_{q_{\varphi }(z)}[\log (\frac{p_{\theta }(x,z)}{p_{\theta }(z|x)})]\\ & =E_{q_{\varphi }(z)}[\log (\frac{p_{\theta }(x,z)}{p_{\varphi }(z)}\frac{p_{\varphi }(z)}{p_{\theta }(z|x)})]\\ & =E_{q_{\varphi }(z)}[\log (\frac{p_{\theta }(x,z)}{p_{\varphi }(z)})]+E_{q_{\varphi }(z)}[\log (\frac{p_{\varphi }(z)}{p_{\theta }(z|x)})]\\ & =E_{q_{\varphi }(z)}[\log (\frac{p_{\theta }(x,z)}{p_{\varphi }(z)})]+KL(q_{\varphi }(z)||p_{\theta }(z|x))\\ \log p_{\theta }(x)& =E_{q_{\varphi }(z)}[\log (\frac{p_{\theta }(x,z)}{p_{\varphi }(z)})]+KL(q_{\varphi }(z)||p_{\theta }(z|x))\\ \log p_{\theta }(x)& ={\mathcal{L}}_{ELBO}+KL(q_{\varphi }(z)||p_{\theta }(z|x))\\ {\mathcal{L}}_{ELBO}& =\log p_{\theta }(x)-KL(q_{\varphi }(z)||p_{\theta }(z|x))\end{array}$$

base on difference condition KL divergence can simplify to difference term

• Variational Inference
• Importance Sampling to ELBO
• Variational EM

refs

• kvfrans
• odie2630463
• bobondemon

VQVAE(d-vae)

Variational Autoencoder (Kingma & Welling, 2014)

quantise bottleneck

• random init centroids
• find the nearest centroids of each unquantise vector
  • if quantise have low usage then random init a new centroids
• calculate average center of unquantise vector
• Exponential Moving Average Update between new centroid and old centroid

loss

• VQ loss: The L2 error between the embedding space and the encoder outputs.
• Commitment loss: A measure to encourage the encoder output to stay close to the embedding space and to prevent it from fluctuating too frequently from one code vector to another.
• where $\text{sg}[.]$ is the stop_gradient operator.

$$L=\underset{\text{reconstruction loss}}{\underbrace{\Vert \mathbf{x}-D({\mathbf{e}}_k){\Vert }_2^2}}+\underset{\text{VQ loss}}{\underbrace{\Vert \text{sg}[E(\mathbf{x})]-{\mathbf{e}}_k{\Vert }_2^2}}+\underset{\text{commitment loss}}{\underbrace{\beta \Vert E(\mathbf{x})-\text{sg}[{\mathbf{e}}_k]{\Vert }_2^2}}$$

CVAE

reinforce learning

actor-critic

origin

• $s_t$: state at time t
• $V(s_t)$: value function predict reward with $s_t$
• $A(s_t)$: action function return action probability base on $s_t$
• top1: select the action with max probability
• $R(a_{[0:t]})$: reward function input a sequence of action out float
• advantages value: if the can get more reward then positive value else negative value

$$\begin{array}{rl}\text{action probability}& =A(s_{[0:t-1]})\\ a_{[0:t-1]}& =\text{top1}(\text{action probability})\\ \text{reward}& =R(a_{[0:t-1]})\\ L_{\text{critic}}& =MSE(reward,V(s_{t-1}))\\ \text{advantages value}& =\text{reward}-V(s_{t-1})\\ L_{\text{actor}}& =\text{CrossEntropy}(\text{action probability},a_{[0:t-1]})\times \text{advantages value}\end{array}$$

with Temporal Difference error(TD-error)

• Hope $V(s_t)$ can predict reward that may get in future with proportion $\gamma$
• note: you should add stop gradient to $T{D}_{\text{target}}$ (aka detach in pytorch)

$$\begin{array}{rl}T{D}_{\text{target}}& =\text{reward}+\gamma V(s_t)\\ T{D}_{\text{error}}& =T{D}_{\text{target}}-V(s_{t-1})\\ L_{\text{critic}}& =MSE(T{D}_{\text{error}},0)=MSE(T{D}_{\text{target}},V(s_{t-1}))\\ \text{advantages value}& =T{D}_{\text{error}}\\ L_{\text{actor}}& =\text{CrossEntropy}(\text{action probability},a_{[0:t-1]})\times \text{advantages value}\end{array}$$