Generative Modeling in Discrete Spaces
Building Order from Noise in Discrete Worlds
Abstract
The rise of deep generative models has revolutionized machine learning, extending data synthesis from continuous domains like images and audio to challenging discrete spaces such as text, molecules, and genomes. The first part of this study builds the mathematical and conceptual foundation for discrete generative modeling, connecting diffusion processes with Markovian dynamics and offering a proof-oriented analysis of advanced frameworks. The second part surveys practical algorithms, optimization techniques, and real-world applications, providing a structured overview of current capabilities, trade-offs, and frontiers in discrete generative modeling.