Uploaded image for project: 'CiviCRM'
  1. CiviCRM
  2. CRM-17629

Improving locks and mail job logging for CiviMail

    Details

    • Type: Improvement
    • Status: Done/Fixed
    • Priority: Minor
    • Resolution: Fixed/Completed
    • Affects Version/s: 4.6.10
    • Fix Version/s: 4.7.9
    • Component/s: CiviMail
    • Labels:
    • Documentation Required?:
      None
    • Funding Source:
      Contributed Code

      Description

      I've been trying to get my head around the mail sending process in Civi as we've been seeing some anomalies. From the code, it looks like the mailer batch limits are not respected. The description says 'Throttle email delivery by setting the maximum number of emails sent during each CiviMail run'. Eg: 6000

      But I actually think that a single cron job will try to deliver 6000 emails from each available job in sequence, rather than delivering 6000 and then stopping and waiting for the next cron run?

      To dig deeper...

      Jobs can be split in to sub-jobs of x emails according to the 'Mailer Batch Size' setting.

      When cron runs, if there is an available space (according to the 'Mailer CRON job limit') then MailingJob::runJobs() is called.

      This gets a list of all un-started and in-progress jobs, and loops through _each _of them. For each job it calls deliver(), which organises the delivery of emails for the job, up to the 'Mailer Batch Limit' (eg: 6000 from the job)

      But then, it looks like the loop in runJobs() (https://github.com/civicrm/civicrm-core/blob/1c46e7a2d674b55e005e05227e1d07ee8ec6ec46/CRM/Mailing/BAO/MailingJob.php#L133) will iterate on to the next available job, and call deliver() again - sending another 6000 emails from the next job. Etc.

      I suppose this could be 'by design'. The name of the setting suggests this, but the description of the setting implies that no more than 6000 will be sent by the cron run. So this could be made clearer.

      If multiple cron runs are started then they will all cycle through all the jobs, looking for one that isn't locked and delivering x emails before moving on. So each cron job moves through each of the available mail jobs in sequence. This seems over-complicated if it is by design.

      It would be great if someone could help clarify what the intended behaviour is here, and what the actual behaviour is too.

      Thanks.

        Attachments

          Activity

            People

            • Assignee:
              monish.deb Monish Deb
              Reporter:
              john John K.
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: