Using non-transactional cache backends in Drupal

Using a backend other than the DrupalDatabaseCache backend can cause cache consistency issues (read: lead to corrupted cache) especially during high load.

For example, when using memcache as a cache backend, memcache does not know about the database transactions currently in progress. This will lead to premature cache invalidation if cache_clear_all() is used inside a transaction. In fact, all other cache backends than the DrupalDatabaseCache backend suffers from this.

Consider the following:

1. Start transaction
2. Invoke hook_node_update()
3. Modules do their stuff, e.g. field cache is invalidated, etc.
4. Node is saved
5. EntityCache is cleared
6. Commit transaction

Whatever data is saved during the entire node_save() operation isn't committed to the database (and therefore available to concurrent requests) until step 6.

If concurrent requests are made to the node in question between #3 and #6, the path cache, field cache, and god knows what else, will be updated with old/wrong data. Only a cache clear or a new node save without any concurrent requests, will fix this.

Same with EntityCache. There may not be much of a time window between #5 and #6, but it's there. If concurrent requests manage to populate the entity cache in that time window, old/wrong data will be used. If you've ever experienced that you were unable to save a node because "it has already been altered", you may be the victim of this effect.

The Cache Consistent module contains a cache wrapper that addresses this issue, by buffering cache operations until the transaction is committed. Hooking in to the transaction requires a core patch, which is bundled with the module.