CiviCRM

Increasing the odds of long-running procedures completing successfully

Details

  • Type: Improvement Improvement
  • Status: Open Open
  • Priority: Major Major
  • Resolution: Unresolved
  • Affects Version/s: 3.0.2
  • Fix Version/s: 4.3
  • Component/s: Core CiviCRM
  • Labels:
    None

Description

On a tunned server, the MySQL Global variable for wait_timeout will likely be tunned rather low. These variables tell MySQL to terminate any open connections that haven't done anything in a while. Since there may be as many as 4 connections open to the database at one time (The Drupal database and the CiviCRM database by both Drupal and CiviCRM) this causes a problem during long running operations. CiviCRM may for example be synchronizing users to contacts for several minutes, finish, and then return control to the CRM. The CRM tries to run a query, only to discover that "The MySQL database has gone away".

Luckily enough you can override this global variable for the current session. I propose something like the following that could be called for things like import, syncing users to contacts, or anything else that is expected to take a long time:

function increaseWaitTimeout() {
  // Increase the timeout on the CiviCRM core DB
  civicrm_initialize();
  require_once('CRM/Core/DAO.php');
  CRM_Core_DAO::executeQuery('SET SESSION wait_timeout = 900', CRM_Core_DAO::$_nullArray);
 
  // Do the same for the CRM Host DB.
  // Not sure how CiviCRM switches between DBs.

  // Do the same for the Host's DBs. Drupal version below.
}

function increaseWaitTimeoutDrupal() {
  global db_url;
  if (is_array($db_url)) {
    foreach ($db_url as $db_key => $db) {
      db_set_active($db_key);
      db_query('SET SESSION wait_timeout = 900');
    }
    db_set_active();
  }
  else {
    db_query('SET SESSION wait_timeout = 900');
  }
}

Activity

Hide
dave hansen-lange added a comment -
This should be done before the upgrade & install scripts as well.
Show
dave hansen-lange added a comment - This should be done before the upgrade & install scripts as well.
Hide
xavier dutoit added a comment -
Why 4 ?

I get the two, but why should re-open a connection to drupal db, and why whould drupal open the civicrm db ?

X+
Show
xavier dutoit added a comment - Why 4 ? I get the two, but why should re-open a connection to drupal db, and why whould drupal open the civicrm db ? X+
Hide
dave hansen-lange added a comment -
There's place in the CiviCRM config file to set a connection string for the host DB, so I assume it's used somewhere, possibly for synching users to contacts. It's also common practice for Drupal modules that need direct access to the CiviCRM DB to do so via a connection string in $db_url. Something like:

$db_url = array(
  'default' => 'mysqli://user:pass@dbserver/drupal',
  'civicrm' => 'mysqli://user:pass@dbserver/civicrm',
  'other' => 'mysqli://user:pass@other_server/other',
);
Show
dave hansen-lange added a comment - There's place in the CiviCRM config file to set a connection string for the host DB, so I assume it's used somewhere, possibly for synching users to contacts. It's also common practice for Drupal modules that need direct access to the CiviCRM DB to do so via a connection string in $db_url. Something like: $db_url = array(   'default' => 'mysqli://user:pass@dbserver/drupal',   'civicrm' => 'mysqli://user:pass@dbserver/civicrm',   'other' => 'mysqli://user:pass@other_server/other', );
Hide
dave hansen-lange added a comment -
This patch adds a generic function that can be called before long-running processes start, and does so for upgrades.
Show
dave hansen-lange added a comment - This patch adds a generic function that can be called before long-running processes start, and does so for upgrades.

People

Vote (0)
Watch (0)

Dates

  • Created:
    Updated: