After making these changes, I noticed that font rendering had changed in Iceweasel and other applications too. The only thing in my .fonts.conf file not in the XFCE fonts setting GUI is a setting for lcdfilter. It seems this enables LCD hinting. There is a way to do this without a .fonts.conf file.
LCD HintingXFCE Docs.
On some systems LCD hinting results in better fonts. But because this is not working on all system, this setting is not configurable through the interface. You can set it with a simple query.
xfconf-query -c xsettings -p /Xft/Lcdfilter -n -t string -s lcddefault
Other possible values are lcdnone, lcddefault, lcdlight and lcdlegacy. You can check if the property is set property, run xrdb -query in a terminal. To fully apply the LCD hinting, it is advised to logout and in again (for restarting the X server).
It's subjective of course, but I think I prefer the slightly softer rendering of fonts with LCD hinting enabled. Here are some before and after screenshots that illustrate the difference.