I always understood a bad case of scarlet fever to affect mainly women who find the allure of a gentleman wearing scarlet to be irresistible. As far as I can see it bears no relation to how good looking the wearer of the scarlet is, as even the most unattractive chap with a complexion that clashes with his coat can be a target of someone suffering from scarlet fever.