Too little sleep, too much sleep, or frequent changes in sleep patterns can all lead to an increased risk of memory problems in women, especially in middle age.