'delete this address' still triggers validation

I have a form validate checking postal codes. If I do not complete the 'standard address' in editing a contact, there is no issue. If I click 'Another address' and leave it blank, no problem. I enter an illegal postcode in the second address, then click 'delete this address'. Upon submit, I get a validation error on that address, while it should have been deleted.