Choroid plexus (CP), best known for producing CSF, also regulate inflammation and clear metabolic waste to maintain brain homeostasis. CP dysfunction is implicated in Alzheimer's Disease (AD), with MRI studies showing CP enlargement in AD. The basis for CP enlargement is unknown. We hypothesized that calcium deposition within CP, which increases with aging and in certain neurodegenerative conditions, might underlie pathologic CP enlargement and be linked to neurodegeneration. In 166 cognitively normal participants, we used multimodal imaging to examine CP structure (MRI-measured overall volume, CT-measured calcium volume), PET-measured Aβ, age, and APOE genotype as predictors of neurodegeneration, indexed as hippocampal volume. CP enlargement was associated with reduced hippocampal volume, particularly in APOE4 carriers. CP calcium was not independently associated with hippocampal volume. However, a significant interaction revealed APOE4 genotype-specific associations between CP calcium and neurodegeneration, with APOE4 carriers showing greater hippocampal volumes in association with greater CP calcium-opposite to our hypothesis. Results suggest that a factor other than calcium drives pathologic CP enlargement associated with neurodegeneration, with this factor especially important in APOE4 carriers. Candidate factors include lipids and inflammatory cells, which are known to accumulate in CP and be regulated by APOE. Our findings highlight CP as a critical locus for studying AD pathogenesis and the mechanisms by which APOE4 promotes AD.