Language models are probability distributions over sequences of words, used to generate probabilities for valid sentences. They are used in a variety of applications such as speech recognition, machine translation, natural language generation, and information retrieval. N-gram language models use the Markov assumption to assign probabilities to sequences of words, and smoothing techniques are used to address sparsity issues.