UTFacultiesEEMCSEventsPhD Defence Bram van de Ven | Classification using dopant Network Processing Units

PhD Defence Bram van de Ven | Classification using dopant Network Processing Units

Classification using dopant Network Processing Units

The PhD defence of Bram van de Ven will take place (partly) online and can be followed by a live stream.

Bram van de Ven is a PhD student in the research group Nano Electronics (NE). Supervisor is prof.dr.ir. W.G. van der Wiel and co-supervisor is prof.dr. P.A. Bobbert both from the Faculty of Electrical Engineering, Mathematics and Computer Science (EEMCS).

The rise of artificial intelligence (AI) is driven by the increase in the processing power of our digital computers and the availability of large amounts of data. This allows a subset of AI that is defined using digital computers, also known as artificial neural networks (ANNs), to become present in today’s world. However, the growth of these ANNs is limited by both the Von Neumann bottleneck and the impending end of Moore’s law. The Von Neumann architecture separates memory and processing. Since ANNs require many memory accesses, communication between the memory and processor limits the size of ANNs. Simultaneously, Moore’s law states that the number of transistors in an integrated circuit doubles every two years. However, we are approaching transistors that need to consist of only a few atoms. This combination results in the requirement for unconventional computing approaches. Neuromorphic hardware, an example of unconventional computing, is inspired by the efficiency of the brain and uses interconnected computing units (similar to neurons in the brain) for AI applications. Neuromorphic engineering focuses on developing computational elements that can be trained and interconnected to perform brain-inspired computation tasks. In this work, we analyse the capabilities of dopant network processing units (DNPUs) in neuromorphic hardware.

As described in Chapter 2, we introduce the three main requirements for neuromorphic systems. First, the system needs to be responsive to external stimuli, such that information is processed. Second, this system needs to achieve the desired response from external stimuli, they need to be trainable. Third, learned states need to be stored in non-volatile memory. We explain the concept of evolution-in-materio (EIM), where a disordered system is trained using a genetic algorithm to perform the desired functionality. Finally, we explain variable range hopping (VRH) and indicate how VRH combined with Coulomb blockade allows DNPUs to be compatible with EIM.

The important aspects of the fabrication and electronic characterisation of DNPUs are described in Chapter 3. The fabrication part is divided into 4-inch, wafer-scale and 1x1 cm2, chip-scale. On the wafer-scale, masking SiO2 is used to define the dopant implantation region. These dopants (boron or arsenic) are implanted using ion beam implantation. After implantation, we dice the wafer in 45 1x1 cm2 chips. These chips are processed using electron beam lithography to define the electrode positions. These electrodes are fabricated using electron beam evaporation and lift- ff and are made of Ti/Pd for boron-doped or Al for arsenic-doped DNPUs. In the final step, we use reactive ion etching to reach the correct dopant concentration for VRH at 77 K. For the electrical characterisation, the DNPUs are placed in liquid nitrogen using a dipstick. This dipstick houses the electronic cables that connect the DNPU to the digital-to-analogue converter (DAC for input voltages) and analogue-to-digital converter (ADC for output voltages). The nA output of the device is mapped to a voltage using a low-temperature IV converter. The converter is placed close to the DNPU to reduce noise.

In Chapter 4, we show that, between 70 K and 160 K, the conduction of DNPUs is governed by variable range hopping (VRH). The tuneable nonlinear behaviour from this VRH is exploited to perform the reconfigurable Boolean logic and feature extraction benchmark. Using the input-output behaviour of the 16 extracted 2x2 pixel features, we simulate that the MNIST handwritten digit classification benchmark can be solved with an accuracy of 96% when combining DNPU feature extractors with a linear layer. For the benchmark tasks in this chapter, the DNPUs are trained using a genetic algorithm performing EIM. The trainability of DNPUs for complex nonlinear tasks makes them interesting for neuromorphic engineering. By applying a back-gate to suppress the conduction from band to VRH, we show that it is possible to perform the reconfigurable Boolean logic benchmark at room temperature.

The disadvantage of using a genetic algorithm to train DNPUs is the large number of measurements involved. When training many benchmarks, a GA becomes time intensive. To find a faster way of training multiple or complex tasks, in Chapter 5, we show that it is possible to train an artificial neural network (ANN) model to behave similarly to the DNPU. This ANN model is trained on the input-output relation of a DNPUs 7-dimensional input space. Using this trained ANN model, also known as a surrogate model (SM), we can use standard deep learning approaches to train the networks. At the initial cost of measuring all the input-output data, subsequent training runs are faster. In Chapter 5, using the SM to train the DNPUs, we show that DNPUs can perform the ring classification benchmark. This ring classification task consists of two sets of concentric rings (classes) separated by a certain distance from one another.

In Chapter 6, we use an SM and Vapnik-Chervonenkis (VC) dimension benchmark to analyse the computational capabilities of DNPUs. VC dimension defines the complexity of a network as the number of binary output tasks this network can perform. We show that a physical DNPU has similar capabilities as a 2-hidden-node ANN. The computation capabilities of the SM are in between an ANN with 2 and 3 hidden nodes. We compare the number of parameters needed to show that, in terms of memory usage, DNPUs have an advantage over fully connected ANNs. Following this analysis, we use the ring classification benchmark task to analyse the added computation capabilities of a network of interconnected DNPUs. Using SMs to train the DNPUs, we attempt to perform the ring classification task for a small separation.  We compare its classification accuracy to that of a network of 5 DNPUs connected in a 2-2-1 configuration. We observe that the network of 5 DNPUs can reach a higher accuracy on this ring classification task. Finally, by using the SM to create a network of multiple SMs, we show that it is theoretically possible to solve the MNIST handwritten digit classification task using a linear input layer and 10 DNPUs. An accuracy of 95 % is achieved on the test data.

In Chapter 7, we move past the standard 1-output DNPU configuration. Utilising multiple output electrodes, we attempt to extract more non-linear computation from the DNPU. This approach is inspired by the field of extreme learning machines (ELM) and reservoir computing (RC) where the network creates a complex mapping from the input to its internal states. These internal states are linearly mapped to the desired output to perform the benchmark task. In this Chapter, we show that the ELM framework allows us to extract the non-linear behaviour from a single 12 electrode DNPU. We build on this concept by showing that DNPUs require a certain degree of control to create a potential landscape capable of extracting more varied nonlinear behaviour. This extra variation allows us to move from solving VC-dimension 5 (5 inputs) to VC-dimension 7 (7 inputs). This increase shows that we can solve 96 extra binary classification tasks. Operating the DNPU in the tuneable ELM mode, using the nonlinearity of 3 DNPUs in parallel, the formant-based vowel recognition benchmark can be solved with an accuracy of 89.9 %. Furthermore, by comparing DNPUs to ANNs the number of parameters/ memory accesses needed is reduced from 72 to 6 and scales better for DNPUs.

The results in this thesis show both the potential and limitations of using DNPUs for neuromorphic computing. The highly tuneable nonlinear response combined with global tuneability allows DNPUs to perform complex tasks that require multiple operations in standard AI implementations. By combining the non-linearity of DNPUs with the memory of linear implementations for neuromorphic engineering, it seems likely that networks combining different material platforms can be created that efficiently perform brain-inspired computation. Using DNPUs, we show the advantages of using the tuneable nonlinear behaviour of disordered networks for neuromorphic hardware.