Your date's impression of you relies on your Romantic Bonus score (located on the left side of the screen in your Wardrobe). You can purchase clothing and objects within your homes in order to increase this score. Look for clothing and furniture with gold hearts!
When you have a high Romantic Bonus, you'll get a bonus gold heart at the beginning of your date and will only need to fill out 4 of the 5 hearts to complete your date.
Please note that you do not need to be wearing specific clothing in order to get the bonus rewards; simply owning the items will contribute to your Romantic Bonus points!