cv
Basics
| Name | Karthik Viswanathan |
| Label | Researcher |
| vkarthik095@gmail.com | |
| Phone | (31) 684103282 |
| Url | https://karthikviswanathn.github.io/ |
| Summary | Topological Data Analysis for Cosmology \( \to \) Interpretability in LLMs |
Education
-
2021.09 - Present Amsterdam, NL
-
2021.08 - 2019.09 Amsterdam, NL
-
2013.09 - 2017.06 Chennai, India
Work
-
2017.08 - 2019.01 Surveillance Analyst
Goldman Sachs, Bangalore, India
Developed ML models for anomaly detection in financial data.
Publications
-
2025.01.17 The Geometry of Tokens in Internal Representations of Large Language Models
arXiv
We investigate the relationship between the geometry of token embeddings and their role in next token prediction within transformer models. Our analysis reveals a correlation between the geometric properties of token embeddings and the cross-entropy loss of next token predictions, implying that prompts with higher loss values have tokens represented in higher-dimensional spaces.
-
2024.10.14 Persistent Topological Features in Large Language Models
arXiv
We present a novel framework based on zigzag persistence, a method in topological data analysis (TDA) to describe data undergoing dynamic transformations across layers of a large language models. As a practical application, we leverage persistence similarity to identify and prune redundant layers, demonstrating comparable performance to state-of-the-art methods across several benchmark datasets.
-
2024.09.18 Cosmology with Persistent Homology: a Fisher Forecast
Journal of Cosmology and Astroparticle Physics
We apply persistent homology to the dark matter halo catalogs, and build a summary statistic for comparison with the joint power spectrum and bispectrum statistic regarding their information content on cosmological parameters and primordial non-Gaussianity. Through a Fisher analysis, we find that constraints from persistent homology are tighter for 8 out of the 10 parameters by margins of 13–50%.
Awards
- 2020.09.01
Sander Bais Prize
Institute for Theoretical Physics, Amsterdam
Awarded by Institute for Theoretical Physics Amsterdam for exceptional academic performance in the master’s program