"Edit All Contacts" required to remove contact from group from within contact record

When user has ACL permission to edit a contact, and tries to remove them from a group (or to rejoin a past group) from within the groups tab on the contact record, they get this error:

"API permission check failed for GroupContact/delete call; insufficient permission: require access CiviCRM and edit all contacts"

They are however able to go to the group listing, e.g. through Manage Groups or by clicking on the group name, and they are able to remove contacts from there.