CSpace: a concept embedding space for biomedical applications.

Abstract

<h4>Motivation</h4>The rise of transformer-based architectures has dramatically improved our ability to analyze natural language. However, the power and flexibility of these general-purpose models come at the cost of highly complex model architectures with billions of parameters that are not always needed.<h4>Results</h4>In this work, we present CSpace: a concise word embedding of biomedical concepts that outperforms all alternatives in terms of out-of-vocabulary ratio and semantic textual similarity task, and has comparable performance with respect to transformer-based alternatives in the sentence similarity task. This ability can serve as the foundation for semantic search by enabling efficient retrieval of conceptually related terms. Additionally, CSpace incorporates ontological identifiers (MeSH, NCBI gene and taxonomy IDs), enabling computationally efficient disease, gene or condition relatedness measurement, potentially unlocking previously unknown disease-condition associations.<h4>Availability and implementation</h4>Full and compressed models are available on Zenodo at https://doi.org/10.5281/zenodo.14781672, while training code, examples, interactive visualizations and experiments are available at https://doi.org/10.5281/zenodo.15125706 and on the GitHub repository.