Some people have very poor facial recognition ability. Actually a lot of people, to various degrees. Here's some info:

"Prosopagnosia, also called face blindness, is an impairment in the recognition of faces. It is often accompanied by other types of recognition impairments (place recognition, car recognition, facial expression of emotion, etc.) though sometimes it appears to be restricted to facial identity. Not surprisingly, prosopagnosia can create serious social problems. Prosopagnosics often have difficulty recognizing family members, close friends, and even themselves."