Mailings does not have an ACL restriction in place

Summary
In a CiviCRM installation with ACL in places the mailing could be potentially send to all the contacts in the database

Explanation
At a membership organisation people are a member of a local chapter. Local chapter users may login and see their local chapter members. The local chapter users are not allowed to see other contacts in the system only their local members.
This is solved by implementing ACL restrictions and that works perfectly! Except for CiviMail. It is possible to create a smart group as a local chapter user without any conditions and use this smart group in CiviMail. Both as recipients and as test group. What happens then is that the smart group will contain all contacts from the system and CiviMail will use all those contacts to send the mail to. There is no ACL restriction in this process and one would assume that the local chapter user when using this smart group for CiviMail should only send the email to all local chapter members.

Cause
When a local chapter user sends a email or a test email from CiviMail then a background job is created. This background job is run from the cron. And the the ACL restrictions for the cron user applies and not the ACL of the user who created the mail.

Zie o.a. https://github.com/civicrm/civicrm-core/blob/master/CRM/Mailing/BAO/Mailing.php#L918

How to solve this?
I am not sure how to solve this. One way to do this is to implement ACL restrictions in getRecipients and getTestRecipients who are based on the creator of the mail.
But maybe others have better ideas.