One contact may be one file of contacts.
If you have downloaded a contact file (vCard) that contains all of your contacts and are attempting to add that to Outlook (2007 or prior), you will only see one contact import. This is because Outlook doesn't support the importing of multiple contacts. To verify that the downloaded file contains all your contacts within it, try opening the file in Excel or Google Contacts. You should now see the list of all your contacts. To work around this quirk in Outlook, try adding the contacts to another contacts manger and syncing that with Outlook. You can also try the CSV file format. (What is CSV?)