Neurodegenerative diseases give rise to irreversible neural damage in the brain. By the time it is diagnosed, the disease may have progressed. Although there is no complete treatment for many types of neurodegenerative diseases, by detecting the disease in its early stages, treatments can be applied to relieve some symptoms or prevent disease progression. Many invasive and non-invasive methods are employed for the diagnosis of dementia. Computer-assisted diagnostic systems make the diagnosis based on volumetric features (structural or functional) or some two-dimensional brain perspectives obtained from a single image modality. This chapter firstly introduces a broad review of multi-modal imaging approaches proposed for dementia diagnosis. Then it presents deep neural networks, which extract structural and functional features from multi-modal imaging data, are employed to diagnose Alzheimer’s and mild cognitive impairments. While MRI scans are safer than most types of scans and provide structural information about the human body, PET scans provide information about functional activities in the brain. Thus, the setup has been designed to make experiments using both MRI and FDG-PET scans. Performances of multi-modal models were compared with single-modal solutions. The multi-modal solution showed superiority over single-modals due to the advantage of focusing on assorted features.