Load balancing on an engine can be determined by its available memory.

This is the RAM that is installed in the computer. This is where the software is executed from and all data is stored and accessed from.

Applications running on a computer typically require more storage space than there is available physical memory. In this situation, a file on disk (called a swap file or page file) is used to simulate additional physical memory.

This is not to be confused with “virtual memory”. Each process typically has 2GB of addressable memory. Some of this address space corresponds to physical memory, and some to virtual memory. When the system tries to access something that is stored in virtual memory, it is automatically transferred to physical memory.

If there is not enough physical memory, any data that has not been used recently is transferred to virtual memory.

This is the memory that is currently being used by a process. It can consist of both physical memory and virtual memory. It might also include memory that was recently used by the process but is no longer required; this unused memory is removed from the working-set when the operating system needs it.

An application uses a structure called a heap to organize memory that is temporarily used by the application. As the application requires memory, it is allocated from its memory heap. As the heap runs out of memory to allocate, it grows and more memory is allocated to the working-set for the heap to use. When a block of memory is no longer required it is returned to the heap so it can be reused. This can leave part of the address space of the heap unused. The part of the working-set associated with that part of the heap can be removed, but the address-space is marked as reserved to it can be reused later. Over time the heap becomes “fragmented” which means that it contains both used and unused blocks. When the application requires a block of memory, the heap searches for an unused block that is large enough; if it cannot find one, the heap grows in size.

The virtual memory address space used by a process typically grows, but not decrease again, as the heap grows but leaves blocks of memory marked as reserved but not used.

The working-set expands and contracts as required as memory corresponding to unused heap blocks can be removed from the working-set.

At some point, the application might request a block of memory from the heap that is larger than any of its unused blocks, and if there is insufficient virtual memory space for the heap to expand, the application fails. The solution to this is to stop new interviews from starting on the engine if the probability of the heap being unable to allocate sufficient memory is too high. Testing has shown a conservative limit is a working-set of 1.2GB if the virtual memory address space is limited to 2GB (the typical limit).

This generally allows all interviews to run successfully. The working-set limit could be raised to 1.5GB, but at this level the engine might try to allocate a larger block than can be found amongst the unused block, which causes it to fail.

The relationship between working set and virtual address space depends on what is being loaded into memory, and an increase in the size of the working set does not necessarily mean an increase in the size of the virtual address space. For example:

Small projects lead to higher memory fragmentation, as illustrated in the following diagram that shows a representation of memory mapped into virtual address space:

Some performance counters in the load balancing script correspond to information available in Task Manager and Performance Monitor.