Question regarding memcached

[joepie91]

In the commandDelegator, a comment states that memcached is needed to prevent jobs from getting queued twice. Similarly, the jobQueue speaks of "locking".

It's not entirely clear to me, however, what exactly these locks are meant to prevent - as I understand it, the child_added event from Firebase only fires once for any given job (ie. no duplicates can occur there). Similarly, presumably beanstalkd will only hand out a job once, so no locking should be necessary there either.

Could you clarify on the purpose of memcached here, and whether it still plays a role in the current version of the codebase? I've been looking at the code for a while now, and I can't find any explanations or hints anywhere of the concrete issues that this is meant to prevent.

[LtSquigs]

The locking system is for two situations:

1. If you have multiple processes of anything that uses the "child_added" listener. These arent unique across processes.

2. If you have multiple distinct jobs for the same site (i.e. two separate builds for the same site in succession). The memcached locks make sure the site is only being built by at most one worker.

In the most simple setup these are just no-ops because you typically will only have one job queue or build worker, but in more complex setups you may have more of these and need to avoid conflicts.