Is the Supervised Rule actually working?

Changing the Supervised rule to something else appears to have zero effect on the popup. For instance I have created a supervised rule that checks only for first name and email. The pop still shows up for matching last names.