If you installed SmarterMail on your VPS or Dedicated Server, and are experiencing email bounces that contain the error "Invalid HELO name", you need to enable SmarterMail to correctly announce itself by assigning it a Fully Qualified Domain Name (FQDN) for the hostname setting.
You will need to enter any hostname which points back to an IP address on your mail server:
Log into SmarterMail as the System Administrator
Click on ‘Settings’ and ‘General Settings’.
Click on the ‘Server Info’ tab.
Change the Host name from whatever is there now, to an actual domain which points to your server.
Content retrieved from: https://support.appliedi.net/kb/a310/smartermail-invalid-helo-name-error.aspx.