Okay, there are two different issues here. First, the mail delivery.
You have
mydomain = domain.com
myhostname = mail.domain.com
and getting
Relay access denied (in reply to RCPT TO command)
This means that received mail is addressed to a domain that is not configured for local delivery, and the mail server is not accepting it to be relayed to the actual target server. This is a good thing, you do not want to have a public relay under any circumstances because it would mean people could make your server launch spam anywhere.
As for why it's not configured to accept that domain for local delivery, you need to look at the mydestination setting:
mydestination (default: $myhostname, localhost.$mydomain, localhost)
The list of domains that are delivered via the $local_transport mail delivery transport. [...]
(from postconf(5).)
You left it at the default value, so it will accept mail addressed to mail.domain.com, localhost.domain.com, and localhost. You'll probably want to set that to additionally contain $mydomain (at least that is how mine is configured).
Also, something else:
My server’s hostname is domain.com not mail.domain.com (mail.domain.com is what my MX record points to), but this shouldn’t really matter as I configured postfix with:
You'll want those to match up, system hostname and postfix's myhostname, since you'll need to set the PTR record of your IP to match the hostname your SMTP server identifies itself as, and otherwise your server's IP resolves to mail.domain.com while the canonical hostname is domain.com. It will work for mail, it'll just not be nice when your server's IP resolves to mail.domain.com for stuff that isn't mail and that isn't the canonical hostname. I recommend giving it some other hostname (or just setting both to mail.domain.com if the system just handles mail).
For the IMAP login issue, I'm pretty sure this is the cause looking at the "unknown user" error:
Have you set up the users in that file (/etc/dovecot/users) if you even want to do that instead of just using passwd? Also note %u is the full user string including domain. Not sure how that plays together with auth_username_format=%n which is just the user name.
Personally I just have
so I don't have anything further to go off of.