Commit 04d75700 by Ying-Qiu Zheng

### Merge branch 'new-branch' into 'master'

Update 2021JUL21.md

See merge request !2
parents 5e771513 772cc0c3
 ... ... @@ -7,16 +7,32 @@ Suppose $\mathbf{X}^{H}, \mathbf{X}^{L}$ are $N \times V$ feature matrices ( For a single voxel, suppose $\mathbf{y}_{n} \sim \text{multinomial}(\mathcal{\pi})$, and $p(\mathbf{x}^{L}_{n}|y_{nk}=1) = \mathcal{N}(\mu_{k}, \Sigma_{k}^{L})$. To use high-quality data to inform the inference on low-quality data, we assume $p(\mathbf{x}^{H}_{n}|y_{nk}=1, \mathbf{U}) = \mathcal{N}(\mathbf{U}\mathbf{x}^{H}_{n}|\mu_{k}, \Sigma_{k}^{H})$ where $\mathbf{U}^{T}\mathbf{U} = \mathbf{I}$. The complete log-likelihood can be written as math \log p(\mathbf{x}^{H}_{n}, \mathbf{x}^{L}_{n}, \mathbf{y}_{n}|\mathbf{U},...\mathbf{\pi}, \mathbf{\mu}_{k},...\mathbf{\Sigma}^{H}_{k},...\mathbf{\Sigma}^{L}_{k},...) = \prod_{k=1}^{K}(\mathcal{N}(\mathbf{x}^{L}_{n}|\mu_{k},\Sigma_{k}^{L})\mathcal{N}(\mathbf{Ux}^{H}_{n}|\mu_{k},\Sigma_{k}^{H}))^{y_{nk}} p(\mathbf{x}^{H}_{n}, \mathbf{x}^{L}_{n}, \mathbf{y}_{n}|\mathbf{U},...\mathbf{\pi}, \mathbf{\mu}_{k},...\mathbf{\Sigma}^{H}_{k},...\mathbf{\Sigma}^{L}_{k},...) = \prod_{k=1}^{K}(\mathcal{N}(\mathbf{x}^{L}_{n}|\mu_{k},\Sigma_{k}^{L})\mathcal{N}(\mathbf{Ux}^{H}_{n}|\mu_{k},\Sigma_{k}^{H}))^{y_{nk}}  The marginal distribution of $\mathbf{x}_{n}^{L}, \mathbf{x}_{n}^{H}$ is math \log p(\mathbf{x}^{H}_{n}, \mathbf{x}^{L}_{n} | \mathbf{U},...\mathbf{\pi}, \mathbf{\mu}_{k},...\mathbf{\Sigma}^{H}_{k},...\mathbf{\Sigma}^{L}_{k},...)=\sum_{k=1}^{K}\pi_{k}\mathcal{N}(\mathbf{x}^{L}_{n}|\mu_{k},\Sigma_{k}^{L})\mathcal{N}(\mathbf{Ux}^{H}_{n}|\mu_{k},\Sigma_{k}^{H}) p(\mathbf{x}^{H}_{n}, \mathbf{x}^{L}_{n} | \mathbf{U},...\mathbf{\pi}, \mathbf{\mu}_{k},...\mathbf{\Sigma}^{H}_{k},...\mathbf{\Sigma}^{L}_{k},...)=\sum_{k=1}^{K}\pi_{k}\mathcal{N}(\mathbf{x}^{L}_{n}|\mu_{k},\Sigma_{k}^{L})\mathcal{N}(\mathbf{Ux}^{H}_{n}|\mu_{k},\Sigma_{k}^{H})  In summary, in addition to finding the the hyper-parameters $\pi, \mu, \Sigma_{k}^{H}, \Sigma^{L}_{k}$, we want to estimate a transformation matrix $\mathbf{U}$ such that $\mathbf{UX}^{H}$ is as close to $\mathbf{X}^{L}$ as possible (or vice versa). ### Pseudo code - Algorithm 1. EM for the Fusion of GMMs 1. Run K-means clustering on the high-quality data to generate the assignment of the voxels $R^{(0)}$. 2. Initialise the means $\mu_{k}^{L}$, $\mu_{k}^{H}$, covariances $\Sigma_{k}^{L}$, $\Sigma_{k}^{H}$, and mixing coefficients $\pi_k$ using the K-means assignment $R^{(0)}$, and evaluate the initial likelihood. 3. Initialise the transformation matrix $\mathbf{U} = \mathbf{MN}^{T}$, where $\mathbf{MDN}^{T}$ is the SVD of $\sum_{k=1}^{K}\mu_{k}^{H}(\mu_{k}^{L})^{T}$. 4. For iteration = $1, 2, ...$, do - **E-step.** Evaluate the responsibilities using the current parameter values - $\gamma(y_{nk}) = \frac{\pi_{k}\mathcal{N}(\mathbf{x}^{L}_{n} | \mu_{k}^{L}, \Sigma_{k}^{L})\mathcal{N}(\mathbf{Ux}^{H}_{n} | \mu_{k}^{L}, \Sigma_{k}^{H})}{\sum_{j=1}^{K}\pi_{j}\mathcal{N}(\mathbf{x}^{L}_{n} | \mu_{k}^{L}, \Sigma_{k}^{L})\mathcal{N}(\mathbf{Ux}^{H}_{n} | \mu_{k}^{L}, \Sigma_{k}^{H})}$ - **M-step.** Re-estimate the parameters using the current responsibilities by setting the derivatives of log likelihood to zero - $\mu_{k}^{L} = \frac{1}{N_{k}}((\Sigma^{H}_{k})^{-1} + (\Sigma^{L}_{k})^{-1} )^{-1}\sum_{n=1}^{N}\gamma(y_{nk})((\Sigma_{k}^{H})^{-1}\mathbf{Ux}^{H}_{n} + (\Sigma_{k}^{L})^{-1}\mathbf{x}_{n}^{L} )$ - $\Sigma_{k}^{L} = \frac{1}{N_{k}}\sum_{n=1}^{N}\gamma(y_{nk})(\mathbf{x}^{L}_{n} - \mathbf{\mu}_{k}^{L})(\mathbf{x}^{L}_{n} - \mathbf{\mu}_{k}^{L})^{T}$ - $\Sigma_{k}^{H} = \frac{1}{N_{k}}\sum_{n=1}^{N}\gamma(y_{nk})(\mathbf{Ux}^{H}_{n} - \mathbf{\mu}_{k}^{L})(\mathbf{Ux}^{H}_{n} - \mathbf{\mu}_{k}^{L})^{T}$ - $\pi_k = \frac{N_{k}}{N}$ - $\mathbf{U}=\mathbf{MN}^{T}$ where $\mathbf{MDN}^{T}$ is the svd of $\sum_{k=1}^{K}\gamma(y_nk)\mu_{k}^{L}(\mathbf{x}_{n}^{H})^{T}(\sum_{n=1}^{N}\mathbf{x}_{n}^{H}(\mathbf{x}_{n}^{H})^{T})$ - Evaluate the likelihood and check for convergence. 5. Using $\mu_{k}^{L}, \Sigma_{k}^{L}, \pi_{k}$ to assignment unseen low-quality data points. ### Simulation results #### We considered three scenarios ##### I. Low-quality data noisier than the high-quality data ... ...
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!