the wavecore - a scalable architecture for real-time audio processing
The subject of this thesis is real-time music/acoustical signal synthesis and processing, using a scalable domain-specific processor architecture. Basically there are two application classes within this domain.The first class is digitization of analog audio effects (e.g. musical instrument effects). Examples are reverberation (i.e.simulation of an acoustical space like concert hall), amplifier emulation and modulation effects like phasing and flanging.The second class is synthesis of sound for electronic musical instruments (e.g. keyboard, virtual piano, or sound rendering systems in virtual reality).
These applications consist of two distinct parts: real-time physical models and user interaction models. An interaction model embodies the interface between the user and the physical model. A physical model is a mathematical abstraction of a physical object, which needs to be executed in real-time. This is the actual signa lprocessing/synthesis part of the application. We focus on three classes of physical models. The first class is virtual analog modeling. These models are based on a mathematical abstraction of an analog signal processing device.The second class is digital waveguidemodeling. This class ofmodeling is based on a coarse abstraction of an acoustical object. The delay-line plays a central role within digital waveguidemodeling.The third class is detailed physicalmodeling of musical instruments and acoustical spaces. This class is the most computational intensive.
From a computational perspective, a physical model has different requirements compared to an interaction model. The most challenging requirements for a physical model are low latency (i.e. elapsed time between signal input and processed output), time predictability of real-time execution and scalability. For interaction models the most important requirements are flexibility and best-effort execution.
These are conflicting requirements, which lead to problems in the state-of-the-art approaches within the music/acoustical signal processing application domain.
The following contributions of this thesis address these problems:
● A scalable many-core (i.e. MIMD) processor: the WaveCore. Specific aspects of theWaveCore are a highly optimized instruction-set with domain specific elements, and explicit delay-line support. The WaveCore is optimized to locality of reference, leading to a high degree of time-predictability of execution and low processing latency.
● A heterogenous processor architecture, consisting of a general purpose provicessor for executing interaction models and the WaveCore processor, for executing physical models.
● A programming model for this heterogenous processor architecture, which is based on a data-flow concept. This programming model is conceptually close to the mathematical nature of many physical modeling problems, and matches closely to the WaveCore processor architecture. This makes the Wave Core efficient and programmable at a domain-specific abstraction level while keeping a high degree of control over processing latency, parallelism and time-predictability of execution.
● Application development tools, consisting of a compiler, a simulator and an FPGA-based WaveCore processor platform.
● Mapping and analysis of a wide range of audio effects and synthesis applications on the WaveCore processor. We evaluated the WaveCore processor architecture, and the associated programming methodology from three different viewpoints:
1) The first viewpoint is application cases. We have implemented several digital audio effects applications. The associated physical models are mathematical abstractions of analog circuitry (virtual analog modeling). In addition to this we implemented reverberation models, based on the digital waveguide principle. Finally we implemented detailed physical models of simple geometrical objects (e.g.string, membrane, plate) and combined these to show the feasibility of building virtualmusic instruments. For all these application cases we investigated the mapping of physical models onto the WaveCore processor, and we studied the programming abstraction and mapping efficiency.
2) The second viewpoint is programming. We implemented a graphic equalizer application in a domain-specific functional programming language for musical applications, called Kronos. This application has been automatically compiled to the heterogenous architecture, while keeping full control over latency and real-time constraints.
3) The third viewpoint is a benchmark analysis. We compared the WaveCore to other processors which are, or can be used for physical modeling. Within these benchmarks we compared the different solutions for area efficiency, instruction-set efficiency, time predictability and application development effort. Our research has shown that the proposed heterogeneous processor architecture covers the investigated applications quite well. Moreover, we showed that the WaveCore programming methodology fits well to existing domain specific functional languages. Despite the high abstraction level of programming, the mapping onto the processor has shown to be efficient, and yields a predictable and ultra-low latency. Moreover, the benchmark analysis has shown that the WaveCore is efficient with respect to area and mapping of applications.
Starting time 12.30h - Building Waaier - Prof.dr. G. Berkhoff-zaal
