For some reason PulseAudio wants to send sounds from my computer to the HDMI monitor, with rather crappy speakers, rather than to my headphones.
To stop it I set up a default output by editing /etc/pulse/default.pa. In my case the right configuration was:
set-default-sink alsa_output.pci-0000_38_00.6.analog-stereo
