Despite brain functions decline with age in the elderly, the brain maintains a certain degree of plasticity which can delay this process through cognitive training. Various cognitive trainings were tested in studies, including strategy training, cognitive processes based training, and multidimensional cognitive training. Recently, computerized cognitive training is becoming the focus of this area because of its universal potential of application. Previous studies have identified the effectiveness of cognitive trainings, but some vital issues remain unclear. For example, to what extent cognitive trainings can help benefit cognition of the elderly and which kind of training is the most beneficial to the specific cognitive ability. To better understand the effectiveness of cognitive trainings and the way they work, this paper reviewed the studies of neural mechanisms of cognitive trainings and the related theoretical models. SMRI studies find that cognitive trainings can alter the structure of the brain, thus delaying or resisting the cognitive decline with age. Functional MRI studies also find that cognitive trainings help improve cognition functionally in both rest state and task-related state. Based on the perspective of compensatory and magnification, several theory models were established to interpret these findings, including HAROLD model, CRUNCH model, Lovden’s model, STAC model and Belleville’s Interactive model. Compensatory perspective focuses the individual differences within the same age range and proposes that cognitive trainings benefit the elderly with lower cognitive ability better, while magnification perspective emphasizes the differences between the youth and the elderly and puts forward that cognitive trainings magnify these differences (cognitive trainings benefit individuals with higher cognitive ability better). At present, there is no consistent conclusion about the two perspectives, and more studies are needed to reconcile the contradiction. In addition, it is beneficial for the application of cognitive trainings in the future to use brain image techniques to examine the effectiveness of cognitive trainings, to carry more studies on computerized cognitive trainings and to adopt more rigorous experimental design is beneficial to the application of cognitive trainings in the future.