VestaCP will have dealt with the most parts of the initial setup, but you also need to go through the following steps.
In its documentation, VestaCP recommends to do the following:
Install Vesta Control Panel
Get a domain name
Link domain name with your new server. see how to set up vanity nameservers
Make sure port 25 is not firewalled you can use SMTP diagnostic tool
Make sure server hostname is FQDN compliant
root@localhost:~# hostname
localhost
root@localhost:~# v-change-sys-hostname mail.vestacp.com
root@localhost:~# hostname
mail.vestacp.com
Set PTR record. Please contact your ISP to find out how to do it. Your ISP is most likely the company you are hosting your server with.
Check server IP reputation. Blacklist Check
Enable DKIM support. This is optional step but highly recommended. DKIM can be enabled in the Vesta under MAIL tab.
Add email users and you are all set
After having set the PTR records, you should also enter your domain name in the DNS Whitelist. That way, your server enters a global whitelist that some mail servers and clients may respect.