In particular, do not rely on the $USER environment variable always existing.
tmux for example seems to clear it, causing lots of invalid messages:
"previous known host file not found"

This broke in commit 80fd22dc, but instead of reverting that commit, we now
fall back to expanding just ~ when $USER is not set.