t-SNE: a Machine-Learning Method for Data Dimensionality Reduction and Visualization

Sobia Iqbal

Master's thesis

Abstract:

In the thesis, t-SNE is introduced, discussed , and applied on various synthetic and non synthetic datasets to outline the inner working of t-SNE. The objective of the thesis is to investigate all t-SNE parameters in general, perplexity in particular. This study is intended to find a way of choosing optimal perplexity value. We use two different approaches to find optimal perplexity value. First, we plot t-SNE with set of perplexity values and choose the best plot. Than we plot Kullback-Leibler divergence against perplexity for each batch of data and choose the perplexity suggested by Kullback-Leibler divergence as the author of the t-SNE Laurens van der Maaten said in his original paper "The kullback-Liebler divergence between the joint probability distribution of high-dimensional similarities and low-dimension similarities is minimised by using gradient descent ". Therefor choosing the best perplexity by minimizing the Kullback-Leibler divergence appeared fair. In last we compared the performance from both methods to check which method provides the best results. The research questions investigated throughout the thesis are as follows: Difference between Implementation of Laurens Van der Maaten t-SNE code and t-SNE Package. Second, Comparison of manually and KL chosen perplexity and to conclude which one is more reliable.

Abstract:

In the thesis, t-SNE is introduced, discussed , and applied on various synthetic and non synthetic datasets to outline the inner working of t-SNE. The objective of the thesis is to investigate all t-SNE parameters in general, perplexity in particular. This study is intended to find a way of choosing optimal perplexity value. We use two different approaches to find optimal perplexity value. First, we plot t-SNE with set of perplexity values and choose the best plot. Than we plot Kullback-Leibler divergence against perplexity for each batch of data and choose the perplexity suggested by Kullback-Leibler divergence as the author of the t-SNE Laurens van der Maaten said in his original paper "The kullback-Liebler divergence between the joint probability distribution of high-dimensional similarities and low-dimension similarities is minimised by using gradient descent ". Therefor choosing the best perplexity by minimizing the Kullback-Leibler divergence appeared fair. In last we compared the performance from both methods to check which method provides the best results. The research questions investigated throughout the thesis are as follows: Difference between Implementation of Laurens Van der Maaten t-SNE code and t-SNE Package. Second, Comparison of manually and KL chosen perplexity and to conclude which one is more reliable.

Keywords

SNE, t-SNE, Kullback–Leibler divergence, perplexity

Keywords

SNE, t-SNE, Kullback–Leibler divergence, Perplexity

Language used: English

Date on which the thesis was submitted / produced: 29. 6. 2023

Thesis defence

• Date of defence: 23. 8. 2023
• Supervisor: Miroslav Plašil
• Reader: Jan Fojtík

Vysoká škola ekonomická v Praze

Master programme / field:
Economic Data Analysis / Data Analysis and Modeling