Your contact list also has a maximum number of spots for social network friends. If you have many Facebook friends playing the game on the same platform as you, some of them may not be shown due to your having exceeded that maximum number.
Additionally, sometimes your contact list may take some time to update and show new friends. If some of your contacts do not appear right away, check back again later. Communication between the game and Game Center/Google Play servers can take longer than expected, so please be patient.