Although most estimates focus on calculations per second, the relevant parameters are:

• Calculations per second
• Memory size
• Memory bandwidth per node
• Inter-node communication bandwidth

Here I assume the WBE detail level to be somewhere between level 4 (spiking neural model) and level 5 (electrophysiological model) from the Whole Brain Emulation Roadmap.  The computational capacity required would be around 100 Exa-FLOPS (1e20) and the memory capacity 10 Peta-bytes (1e13).  Plugging these into the CPU and memory timeline calculators, gives an expected arrival time of 2026 and 2020 respectively for $1M.

Based on my models, it appears that memory bandwidth will be a significant bottleneck, while inter-CPU node bandwidth will not be.

I’ve created a spreadsheet where the two bandwidths are estimated.

The architecture I envision is 1 million compute nodes arranged in a 2-D cluster, 1000 on the side.  Each node will have 100 TFLOPS CPU capacity and 10 GB of memory.  Nodes are connected to the nearest 4 neighbors using a high speed bus.  Longer distances are covered by multiple hops.

The inter-node bandwidth is dependent on the distance that voltage information has to travel, which is related to the axon length.  Although some axons are very long, axon lengths probably follow a power law distribution, and long ones are rare.  Based on a 1KHz update frequency and  ~60 nodes within average axon distance, the required communication bandwidth is 250 Gb/s, which is close to the capabilities of current technology.

The local node memory bandwidth is dependent on the amount of synapse state memory and required refresh rate.  Based on a 1KHz update rate, this works out to 14 TB/s.  This figure is about 3 orders of magnitude higher than current technology.

The high memory bandwidth required makes it is likely that some kind of CPU/memory on-die integration will be required.  The memristor seems a good candidate.

It is interesting to think of the brain as a 2.x dimension structure, due to the non-zero depth and also due to the folds.  The question has been asked if this provides a barrier to implementation.  The depth aspect is not an issue, since dividing the brain on a 2-D grid will put most vertical columns on the same node.  Folds effectively reduce the length that an axon has to travel.  It seems that in the worst case two points that “should” be a meter apart (brain diameter when laid flat) are only 10 cm apart (brain actual diameter).   This may increase the inter-node bandwidth by a factor of 100, if most axons take advantage of the added dimensionality for routing.  Since the inter-node bandwidth is relatively modest, it is doubtful that even a 100 fold increase will make it the bottleneck instead of the memory bandwidth.

In sum, here are the performance requirements:

• 1 million compute nodes in a 2-D topology
• 100 TFLOPS per node
• 10GB memory per node
• 10TB/s memory bandwidth
• 4 links to neighboring nodes at 250 Gb/s

It seems likely that this kind of machine will be available by 2030 for $1M, and possibly as early as 2025.  It would be useful to evaluate the increase in memory bandwidth over time and extrapolate to this time frame.

It is useful to think about the simulation’s level of detail (LOD) in terms of certain thresholds, from most detailed to least detailed:

• Noise level: at this level (call it LODn) the lack of precision is on the order of the noise present in a biological brain.  It is not possible to distinguish the functioning of such a simulated brain from the functioning of a biological brain.
• Functional level: at this level (call it LODf) the lack of precision is greater, but the result is functionally similar.  There may be some behavior changes, but the overall capabilities (e.g. “intelligence”) are similar.
• Equivalence level: at this level (call it LODe) the precision is even lower, but this is compensated with  tweaks to the simulated physiology.  The result is equivalent in capabilities, although some characteristics may be very different.  For example, the retina can be replaced with a non-biological equivalent.

The computation power required for LODn > LODf > LODe.  There are likely order of magnitude differences between the levels.

If we consider a non-biological example – a digital computer, what does it take to simulate it?  It is obviously enough to simulate the logic function.  The Markham’s line of reasoning would seem to argue that we have to simulate the voltage gradients and charge movements in each transistor!

In the transistor case, LODn would involve simulating each transistor’s logic function.  LODf would involve an instruction set simulation (e.g. the QEMU emulator).  LODe would involve using the most convenient instruction set (e.g. x86) and recompiling any software.  Clearly, an LODe simulation is several orders of magnitude more efficient.

Markham fails to convince that his preferred level of simulation is required for LODn, never mind the other levels.

One way to find out what levels require is to actually run simulations and compare to physical neural matter.  The Blue Brain project aims to do that, although the results are not conclusive yet.  It would be good if more research was directed at comparing their simulation to a biological brain.  This would make the project more grounded.

Although we may achieve human level compute power in 2014, it looks like memory capacity will lag by another 6 years, assuming low estimates. With high estimates, compute power is available in 2020 and memory capacity will lag by 5 years after that, to 2025.

However, if Flash memory or similar technology will do the trick, a factor of 4 in cost reduction will advanced the timeline by about 4 years.

H/T: Next Big Future

We deployed the simulator on a 4096-processor BlueGene/L supercomputer with 256 MB per CPU. We were able to represent 8,000,000 neurons (80% excitatory) and 6,300 synapses per neuron in the 1 TB main memory of the system. Using a synthetic pattern of neuronal interconnections, at a 1 ms resolution and an average firing rate of 1 Hz, we were able to run 1s of model time in 10s of real time!

This is excellent news, since it will now be possible to figure out what biological modeling aspects are important to functionality.

Since the human brain has 100 billion neurons, this represents 1/10,000 of a human brain. The computer was a $100 million BlueGene/L. So an improvement of 10,000,000 is required in order to model a human brain for $1M in real time.

However, the BlueGene/L is two years old, and it is about 20 times less efficient compared to commodity hardware (based on a quoted 360 teraflops). So the real improvement required is only around 100,000.

Based on this data, the human brain requires 10 Exa CPS, one order of magnitude above the high estimate use in my calculator. Human equivalent for $1M would be available around the year 2023.

Hardware specifically suitable for this application may bring this back to 1 Exa CPS and pull this back to the year 2020.