How to deal with GMAIL Undelivered Mail Returned to Sender

If you run your own mail server, you might end up quite frustrated because the Google relays do reject your mails.

Maybe your mails get rejected by the Google servers with the following message:

Our system has detected that this  message does not meet IPv6 sending guidelines regarding PTR records and authentication. Please review  https://support.google.com/mail/?p=IPv6AuthError for more information.

If you start reading your frustration level might even increase, due to the difficulty to deal with many of the requirements to get your mails to a Google server delivered. If you start digging into the various topics, you might end up even more frustrated as you have more questions as before.

At the very end, you just have to set up a few things to make your mails fully compliant, so even Google’s server do accept them. In addition you will get high quality mails – at least regarding the tech, content is entirely up to you.

I do run a setup of mail, Dovecot, Postfix and SpamAssasin on a Ubuntu server. Therefore, this article will cover the topic on these examples but should be able to be applied to almost any other system – as long as you replace tech X by tech Y.

Testing you Mail

The first issue is how to learn about your mails issues at all. To evaluate your mails, http://www.mail-tester.com turned out to be a perfect platform for me. As you only can evaluate three mails a day for free, you might throw in a few bucks if you in a hurry, or it might simply take some days to you to finish the tests.

How does it work? You get an unique mail address where you send your mail to. Once the mail was received any flaws in the mail will be displayed and explained (which definitely is a huge added value).

10/10 Score at mail-test.com

Believe me or not, I definitely haven’t had a score of 10/10 before.

SPF Record

First make sure, your DNS provides an SPF record. With no or little knowledge, I tried to create one by myself based on Google’s help page about SPF records. Did not work well at all.

I finally ended up with the following SPF entry for my DNS:

v=spf1 mx ip4:92.51.166.153 -all
  • a is just to indicate that there might be scripts sending mails
  • ip4:92.51.166.153 depicts the mails server we are talking about in the entry
  • -all means there only this mail server while the mx indicates that only domain’s MXs are allowed to send mail for the domain while all others are prohibited.

It took me a while to figure out the right (or at least something working) based in the SFP record syntax. So why do we do this? The idea behind SPF is simple, once you have published your policy, the receiving server can check if the mail is compliant with your policy. In my case, mails sent from scripts might be fine, as long as they come from my server. If someone sends a mail from server 154.354.32.2 this mail will be probably not sent by me, as it is not compliant with my policy. Google’s server do these checks, and that’s on of the reasons your mails are rejected by them.

Finally the SPF entries doe look like the following at my DNS:

DKIM – DomainKeys Identified Mail

Next, I started with DKIM. Again, if you start reading at dkim.org, you probably end up reading several IETF RFCs and a lot of question. Reading RFCs is always good and highly recommended by me, but actually did not solve my issue.

The good thing about Ubuntu is, there is an package called OpenDKIM which can be installed via

sudo apt-get install opendkim opendkim-tools

I got relatively lost, when i came to configure OpenDKIM and Postfix until I found the Postfix/DKIM site at the Ubuntu documentation.

First of all you need to add the domain to the /etc/opendkim.conf. You might want to exchange my domain aheil.de to your corresponding one.

Domain    aheil.de
KeyFile   /etc/postfix/dkim.key 
Selector   dkim

Now, I had to generate the key file.

opendkim-genkey -t -s dkim -d aheil.de

If you did it the same way as me, the file is probably located in the wrong directory. So move the file to the location provided above.

mv dkim.private /etc/postfix/dkim.key

Now I went to /etc/default/opendkim. Unlike as described in the Ubuntu documentation, I had to add the listening socket – which seems to be a standard one at Ubuntu.

SOCKET="inet:8891@localhost"

I then went back to /etc/opendkim.conf and added the same port right below the Selector entry.

Domain    aheil.de 
KeyFile   /etc/postfix/dkim.key 
Selector  dkim
Socket    inet:8891@localhost

Now, I went to the /etc/postfix/main.cf file and added the following entries:

milter_default_action = accept
milter_protocol = 2
smtpd_milters = inet:localhost:8891
non_smtpd_milters = inet:localhost:8891

For some reason, I now ended up with two DKIM entries in my outgoing mails. I had to add no_milters to the receive_override_options line in my /etc/postfix/master.cf. I had to fiddle a little bit with the master.cf until everything worked again.

-o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters

Wherever you have created the keyfile, there will be an additional dkim.txt file. It will contain something like

dkim._domainkey IN TXT ( "v=DKIM1; k=rsa; t=y; "
 "p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCZFYWrNHupXvZHvWJAo6aGB6sXYf+kpUIJv+BcP6NFiF9GBy26oYZKt6/ngCXmNAIY9+yReY8UxG5GIm/QnInbXWxwDXyD0MHD8HrhHyVa6JVqTncexm29M5Bbp/u0JI4ToOQwIQqpgTr0t9ADP8i76pbelYQ5IQDOwJRJQ2N1iQIDAQAB" ) ; ----- DKIM key dkim for aheil.de

Creating the corresponding DNS entry out of this seemed to be the most challenging part. You need to create an TXT entry similr to dkim._domainkey.aheil.de, of course using your domain. and you simply need the follwowing text in there

v=DKIM1;
p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCZFYWrNHupXvZHvWJAo6aGB6sXYf+kpUIJv+BcP6NFiF9GBy26oYZKt6/ngCXmNAIY9+yReY8UxG5GIm/QnInbXWxwDXyD0MHD8HrhHyVa6JVqTncexm29M5Bbp/u0JI4ToOQwIQqpgTr0t9ADP8i76pbelYQ5IQDOwJRJQ2N1iQIDAQAB

No quotes, no brackets, no nothing beside the above. This took me quite a while especially as DNS replication is nothing that works in real time.

For me the entry does now look like the following.

Once this is done, start OpenDKIM and restart Postfix so the changes will apply.

sudo service opendkim start
sudo service postfix restat

So far what does this actually mean? When a mail server receives your mail, it now will be signed with a DKIM key. If you check your mail headers, the will look similar to the following one.

By checking the dkim._domainkey DNS entry created before, the mail receiver now can check if the mail signature is authentic. Any mail server not in possession of the private key will probably not be able to sign the mail though, and could be thought of a spam server.

DMARC – Domain-based Message Authentication, Reporting & Conformance

Finally, it is necessary to create a DMARC record. Once SPF and DKIM are setup as above, you proceed similar as with the DKIM DNS entry.

Again you create a TXT revord for your DNS. This time the record is named _dmarc.domain.TDL, e.g. _dmarc.aheil.de in my case.

I created a relatively simple record:

v=DMARC1; p=none

I.e. p=none: the I do not requests any specific action be taken on mail that fails DMARC authentication and alignment.

At the very end, it looks like this for my DNS:

Summary

To get Google accepting mails from your own mail server, only a few steps are necessary. Setting up SPF, DCIM and DKIM are basically all which might be needed if Google rejects mails from your handcrafted mail server. You might want polish your SPF and DMARC records to align your policies with your very personal goals. If you are interested further in e-mail architecture, the book from Kevin Thomas called Email Architecture, Design, and Implementations might be something for your.

RDP from Mac to Ubuntu

I am currently in the progress of moving my entire home automation server from Windows 2003 to a Ubuntu LTS. However, connecting to the new server via SSH is quite painful. As I am using openHAB including the Eclipse-based editor, I would prefer to connect to the server from Windows and Mac via RDP.

I tried quite a while using XRDP, but almost gave up as I always had trouble, either not being able to connect or having no desktop at all.

Therefore, kudos to Mike Rehner, who came up with a step by step guide how to install and cofigure XRDP on Ubuntu.

20 minutes absolutely worth watching, especially as he comes up with two or three hints, I haven’t foundon any other tutorial so far such as

  • you are going to need a 2D desktop such as MATE Desktop Environment
  • you need to connect several times using the RDP as the first or second attempt might fail
  • how to change the XRDP settings to be able to connect to previous session

The proof, I was able to connect to the Ubuntu via my Mac, though:

XRDP Mac OS X Ubuntu

 

Ubuntu: Setting up Oracle JDK

Since Oracle’s Java SDK is not part of Ubuntu, there are some steps required to get Java set up on a Ubuntu system. Especially running a 64-bit version of Ubuntu requires one tiny additional step driving you crazy in case do not speak Ubuntu fluently.

There are quite some scripts and tutorials out there how to get Oracle’s Java working on a Ubuntu system, however, there are only a few steps necessary to do so:

Pick the bits from Oracle’s download site and extract the archive using tar. In my case we use the 32-bit version of the JDK

tar -xvf ~/Downloads/jdk-7u15-linux-i586.tar.g

Create a folder and move all the stuff over there

sudo mkdir -p /usr/lib/jvm/jdk1.7.0 
sudo mv jdk1.7.0_15/* /usr/lib/jvm/jdk1.7.0/

Now set up the symbolic links for java, javac and javaws.

sudo update-alternatives --install "/usr/bin/java" "java" "/usr/lib/jvm/jdk1.7.0/bin/java" 1
sudo update-alternatives --install "/usr/bin/javac" "javac" "/usr/lib/jvm/jdk1.7.0/bin/javac" 1
sudo update-alternatives --install "/usr/bin/javaws" "javaws" "/usr/lib/jvm/jdk1.7.0/bin/javaws" 1

Now, create a Bourne script to set the JAVA_HOME variable and update the PATH variable. In case you haven’t used Vim yet, get it using

sudo apt-get install vim

and then create the file

sudo vim /etc/profile.d/java7.sh

edit the file file and add the following export statements

export JAVA_HOME=/usr/lib/jvm/jdk1.7.0
export PATH=$PATH:$JAVA_HOME/bin

Finally execute the script

source /etc/profile

and try to call java. If you run a freshly set up Ubuntu such as mine, you probably will end up the system telling you

bash: /usr/bin/java: No such file or directory

What happened? Actually, we installed a 32-bit version of the JDK, however as mentioned at the beginning running a 64-bit version of Ubuntu and therefore we come back to this very tiny bit missing. One additional package is required to run the 32-bit version: libc6-i386.

“This package includes shared versions of the standard C library and the standard math library, as well as many others. This is the 32bit version of the library, meant for AMD64 systems.”

Said that, get it

sudo apt-get install libc6-i386

and try to run java one more time. It should work now fine.