Authored by Dan Skinner, Vice President of Strategy & Business Development
Given the buzz around terms like AI, machine learning and neuromorphic, I suppose it shouldn’t be surprising to read some recent hype about the industry’s “first” silicon for neural network processing. Before we dive into specifics about the announcement, it’s important to review the predominant microprocessor architecture of the past 50 years, the von Neumann architecture, to understand if the announcement is really a game-changer or just the latest chapter of a decades-old story.
Commercial processors that power today’s computers are descendants of the von Neumann architecture. While innovative for its time, the biggest performance limiter of processors with this architecture is data movement into and out of memory, often called the “memory wall.” John Backus described this issue as the “von Neumann Bottleneck” in a 1977 speech at the ACM Turing Awards, where he said (in part),
“Surely there must be a less primitive way of making big changes in the store than by pushing vast numbers of words back and forth through the von Neumann bottleneck. Not only is this tube a literal bottleneck for the data traffic of a problem but, more importantly, it is an intellectual bottleneck that has kept us tied to a word-at-a-time thinking instead of encouraging us to think in terms of the larger conceptual units of the task at hand.”
Significant efforts have been poured into creating more efficient movement of data over the last 50 years, including on-chip cache memory and higher speed memory interfaces, but there has been no apparent effort to re-think the basic architecture to remove the memory wall.
Returning to the recent announcement of a “revolutionary” architecture, a notable quote from the news release claims that the new architecture “free[s] us from the limitations imposed by existing hardware.” Upon closer inspection, the new processor appears to be much more evolutionary than revolutionary, relying once again on incremental improvements to enhance vector operations and move more data on and off chip. These improvements include:
- Eliminating the cache and moving memory on-package, a move designed to make memory access more efficient.
- A high-speed communication link designed for massive bi-directional data movement, with the intent to network several chips together to create larger models than a single chip can process.
- Improved vector/matrix processing functions.
While these shifts are certainly a move in the right direction to solve big data problems, the game remains the same: distribute instructions and data from memory to the processor (or processors), perform vector/matrix operations, then write the results back to memory. Operating in the same box they’ve been in for 50 years, no matter how you rearrange the components, is hardly revolutionary.
So if the “first” processor for neural networks is really just a repackaged von Neumann processor, what would a truly game-changing neuromorphic processor look like?
To start, we should throw out the idea of an instruction-driven machine and turn to the original neural model: the human neocortex. Your brain processes a constant stream of sensory data – sight, sound, touch, smell – and uses the data itself to stimulate and traverse the synaptic pathways that connect neurons. For instance, the smell of smoke and the visual of fire would direct synapses in your brain to spur you to react to the danger in front of you, whether its grabbing a fire extinguisher, fleeing the situation or a number of other scenarios. While simplified, this shows the depth and power of a neural network and why it is built for speed and volume without bottlenecks.
It makes more sense to build a processor that recognizes patterns in the data stream (symptoms of fire = danger) and produces high-quality results from “noisy” data to reach or exceed the level of acuity found in the most powerful processor we know: the human brain. The neocortex also operates at a fraction of the energy used by today’s computers, meaning a truly neuromorphic focus would aim at building a processor that exceeds the performance of the brain while using a fraction of the energy.
Given the powerful, all-natural processor above your shoulders, you’ve probably concluded what’s next: we’ve created a biologically-inspired processor that removes the von Neumann bottleneck from the equation by emulating the human brain.
Designed from the ground up to remove the constraints of dated architecture, our Natural Neural Processor (NNP) is a streaming processor that uses the data itself to activate cells (i.e. neurons) in a user-configurable network (i.e. synapse) to discover patterns in the data stream. There is no concept of a separate memory and processing core in our NNP as memory and processing are fused.
In the words of our customers, our NNP is a true game-changer.
By making memory and processing one and the same, we eliminate the von Neumann bottleneck. Our processor doesn’t do matrix multiplication on floating point numbers like today’s processors, but neither does the human brain.
If we really want to create biologically-inspired processors to drive an AI revolution, then we shouldn’t focus on doing a better job of matrix math or convolution neural networks processing. We should build a pattern-matching, data-driven streaming processor that behaves more like the human neocortex.
At the start of the digital photography revolution many of the entrenched companies such as Kodak and Polaroid were working feverishly to make better chemical-film or cheaper cameras and completely missed the sea-change that was happening around them. We believe that the dawn of a new processing era is upon us. AI is the sea-change that will drive the revolution, and our NNP is the embodiment of a truly revolutionary neuromorphic processor.