Gauss-Newton Based Learning for Fully Recurrent Neural Networks
The thesis discusses a novel off-line and on-line learning approach for Fully Recurrent Neural Networks (FRNNs). The most popular algorithm for training FRNNs, the Real Time Recurrent Learning (RTRL) algorithm, employs the gradient descent technique for finding the optimum weight vectors in the recurrent neural network. Within the framework of the research presented, a new off-line and on-line variation of RTRL is presented, that is based on the Gauss-Newton method. The method itself is an approximate Newton’s method tailored to the specific optimization problem, (non-linear least squares), which aims to speed up the process of FRNN training. The new approach stands as a robust and effective compromise between the original gradient-based RTRL (low computational complexity, slow convergence) and Newton-based variants of RTRL (high computational complexity, fast convergence). By gathering information over time in order to form Gauss-Newton search vectors, the new learning algorithm, GN-RTRL, is capable of converging faster to a better quality solution than the original algorithm. Experimental results reflect these qualities of GN-RTRL, as well as the fact that GN-RTRL may have in practice lower computational cost in comparison, again, to the original RTRL.
Design of a Scalable Polyphony-MIDI Synthesizer for a Low Cost DSP
In this thesis, the design of a music synthesizer implementing the Scalable Polyphony-MIDI soundset on a low cost DSP system is presented. First, the SP-MIDI standard and the target DSP platform are presented followed by review of commonly used synthesis techniques and their applicability to systems with limited computational and memory resources. Next, various oscillator and filter algorithms used in digital subtractive synthesis are reviewed in detail. Special attention is given to the aliasing problem caused by discontinuities in classical waveforms, such as sawtooth and pulse waves and existing methods for bandlimited waveform synthesis are presented. This is followed by review of established structures for computationally efficient time-varying filters. A novel digital structure is presented that decouples the cutoff and resonance controls. The new structure is based on the analog Korg MS-20 lowpass filter and is computationally very efficient and well suited for implementation on low bitdepth architectures. Finally, implementation issues are discussed with emphasis on the Differentiated Parabole Wave oscillator and MS-20 filter structures and the effects of limited computational capability and low bitdepth. This is followed by designs for several example instruments.
Implementation of a Tx/Rx OFDM System in a FPGA
The aim of this project consists in the FPGA design and implementation of a transmitter and receiver (Tx/Rx) multicarrier system such the Orthogonal Frequency Division Multiplexing (OFDM). This Tx/Rx OFDM subsystem is capable to deal with with different M-QAM modulations and is implemented in a digital signal processor (DSP-FPGA). The implementation of the Tx/Rx subsystem has been carried out in a FPGA using both System Generator visual programming running over Matlab/Simulink, and the Xilinx ISE program which uses VHDL language. This project is divided into four chapters, each one with a concrete objective. The first chapter is a brief introduction to the digital signal processor used, a field-programmable gate array (FPGA), and to the VHDL programming language. The second chapter is an overview on OFDM, its main advantages and disadvantages in front of previous systems, and a brief description of the different blocks composing the OFDM system. Chapter three provides the implementation details for each of these blocks, and also there is a brief explanation on the theory behind each of the OFDM blocks to provide a better comprehension on its implementation. The fourth chapter is focused, on the one hand, in showing the results of the Matlab/Simulink simulations for the different simulation schemes used and, on the other hand, to show the experimental results obtained using the FPGA to generate the OFDM signal at baseband and then upconverted at the frequency of 3,5 GHz. Finally the conclusions regarding the whole Tx/Rx design and implementation of the OFDM subsystem are given.
A Multimedia DSP processor design
This Master Thesis presents the design of the core of a fixed point general purpose multimedia DSP processor (MDSP) and its instruction set. This processor employs parallel processing techniques and specialized addressing models to speed up the processing of multimedia applications. The MDSP has a dual MAC structure with one enhanced MAC that provides a SIMD, Single Instruction Multiple Data, unit consisting of four parallel data paths that are optimized for accelerating multimedia applications. The SIMD unit performs four multimedia-oriented 16-bit operations every clock cycle. This accelerates computationally intensive procedures such as video and audio decoding. The MDSP uses a memory bank of four memories to provide multiple accesses of source data each clock cycle.
Implementation of Uncoordinated Direct Sequence Spread Spectrum using Software Defined Radios
One of the major threats to wireless communications is jamming. Many anti-jamming techniques have been presented in the past. However most of them are based on the precondition that the communicating devices have a pre-shared secret that can be used to synchronize the anti-jamming scheme. E.g. for frequency hopping the secret could be used to derive the hopping sequence and for direct sequence spread spectrum the secret is used to derive the spreading codes. But how can the devices bootstrap a jamming-resistant communication without having a pre-shared secret? Christina Popper and Mario Strasser propose as scheme for Uncoordinated Frequency Hopping (UFH) and Uncoordinated Direct Sequence Spread Spectrum (UDSSS) in their papers [1] and [2] respectively. The goal of my project was an implementation of Uncoordinated Direct Sequence Spread Spectrum (UDSSS) using Software De ned Radios. The First version should serve as an easy to use and extendable proof of conceptfor the proposed scheme.
Algorithms and tools for automatic generation of DSP hardware structures
The increased complexity of Digital Signal Processing (DSP) algorithms demands for the development of more complex and more efficient hardware structures. The work presented herein describes the core components for the development of a tool capable of automatic generation of efficient hardware structures, therefore facilitating developers work. It comprises algorithms and techniques for i) balancing the paths in a graph, ii) scheduling of operations to functional units, iii) allocating registers and iv) generating the VHDL code. Results show that the developed techniques are capable of generating the hardware structure of typical DSP algorithms represented in data-flow graphs with over 2,000 nodes in around 200 ms, scaling to 80,000 nodes in about 214 s. Within the developed techniques, solving the scheduling problem is one of the most complex tasks: it is a NP-complete problem and directly influences the number of functional units and registers required. Therefore, experimental analysis was made on scheduling algorithms for time-constrained problems. Results show that simple list-based algorithms are more efficient in large problems than more complex algorithms: they run faster and tend to require less functional units.
High speed data collection with Blackfin DSP
This report covers a master thesis in embedded systems, the goal of which was to investigate the high speed data collection capabilities with a Blackfin DSP. Basic theory about sampling and noise is covered briefly from a practical point of view. The theory is intended to be useful for those diving into a ADC datasheet for the first time. After an investigation of the delimiting factors, suitable components were selected and a prototype ADC PCB was designed from scratch. The goal is to design a general low noise data collecting unit compatible with the Blackfin DSP. Finally simple DSP software is designed to prove that DSP can handle such a high datastream.Testing the ADC card with the target Blackfin platform indicates thatthe analog parts indeed works. An analog bandwidth of over 10MHz ismeasured at a resolution exceeding 10 bits with respect to noise. The digital parts intended to interleave the two channels digital streams into one Blackfin unit did not work as intended. Only one channel is supported as of now. The report contains suggestions for future work in this area.
Active Noise Control of a Forest Machine Cabin
Today, a high noise level is considered a problem in many working environments. The main reason is that it contributes to stress and fatigue. Traditional methods using passive noise control is only practicable for high frequencies. As a complement to passive noise control, active noise control (ANC) can be used to reduce low frequency noise. The main idea of ANC is to use destructive interference of waves to cancel disturbing noises. The purpose of this thesis is to design and implement an ANC system in the driver's cabin of a Valmet 890 forest machine. The engine boom is one of the most disturbing noises and therefore the main subjective for the ANC system to suppress. The ANC system is implemented on a Texas Instrument DSP development starter kit. Different FxLMS algorithms are evaluated with feedback and feedforward configurations. The results indicate that an ANC system significantly reduces the sound pressure level (SPL) in the cabin. Best performance of the evaluated systems is achieved for the feedforward FxLMS system. For a commonly used engine speed of 1500 rpm, the SPL is reduced with 17 dB. The results show fast enough convergence and global suppression of low frequency noise.
Evaluation of a Floating Point Acoustic Echo Canceller Implementation
This master thesis consists of implementation and evaluation of an AEC, Acoustic Echo Canceller, algorithm in a floating-point architecture. The most important question this thesis will try to answer is to determine benefits or drawbacks of using a floating-point architecture, relative a fixed-point architecture, to do AEC. In a telephony system there is two common forms of echo, line echo and acoustic echo. Acoustic echo is introduced by sound emanating from a loudspeaker, e.g. in a handsfree or speakerphone, being picked up by a microphone and then sent back to the source. The problem with this feedback is that the far-end speaker will hear one, or multiple, time-delayed version(s) of her own speech. This time-delayed version of speech is usually perceived as both confusing and annoying unless removed by the use of AEC. In this master thesis the performance of a floating-point version of a normalized least-mean-square AEC algorithm was evaluated in an environment designed and implemented to approximate live telephony calls. An instruction-set simulator and assembler available at the initiation of this master thesis were extended to enable; zero-overhead loops, modular addressing, post-increment of registers and register-write forwarding. With these improvements a bit-true assembly version was implemented capable of real-time AEC requiring 15 million instructions per second. A solution using as few as eight mantissa bits, in an external format used when storing data in memory, was found to have an insignificant effect on the selected AEC implementation’s performance. Due to the relatively low memory requirement of the selected AEC algorithm, the use of a small external format has a minor effect on the required memory size. In total this indicates that the possible reduction of the memory requirement and related energy consumption, does not justify the added complexity and energy consumption of using a floating-point architecture for the selected algorithm. Use of a floating-point format can still be advantageous in speech-related signal processing when the introduced time delay by a subband, or a similar frequency domain, solution is unacceptable. Speech algorithms that have high memory use and small introduced delay requirements are a good candidate for a floating-point digital signal processor architecture.
Decoding Ogg Vorbis Audio with The C6416 DSP, using a custom made MDCT core on FPGA
Ogg Vorbis is a fairly new and growing audio format, often used for online distribution of music and internet radio stations for streaming audio. It is considered to be better than MP3 in both quality and compression and in the same league as for example AAC. In contrast with many other formats, like MP3 and AAC, Ogg Vorbis is patent and royalty free. The purpose of this thesis project was to investigate how the C6416 DSP processor and a Stratix II FPGA could be connected to each other and work together as co-processors and using an Ogg Vorbis decoder as implementation example. A fixed-point decoder called Tremor (developed by Xiph.Org the creator of the Vorbis I specification), has been ported to the DSP processor and an Ogg Vorbis player has been developed. Tremor was profiled before performing the software / hardware partitioning to decide what parts of the source code of Tremor that should be implemented in the FPGA to off-load and accelerate the DSP.
Implementation of a Tx/Rx OFDM System in a FPGA
The aim of this project consists in the FPGA design and implementation of a transmitter and receiver (Tx/Rx) multicarrier system such the Orthogonal Frequency Division Multiplexing (OFDM). This Tx/Rx OFDM subsystem is capable to deal with with different M-QAM modulations and is implemented in a digital signal processor (DSP-FPGA). The implementation of the Tx/Rx subsystem has been carried out in a FPGA using both System Generator visual programming running over Matlab/Simulink, and the Xilinx ISE program which uses VHDL language. This project is divided into four chapters, each one with a concrete objective. The first chapter is a brief introduction to the digital signal processor used, a field-programmable gate array (FPGA), and to the VHDL programming language. The second chapter is an overview on OFDM, its main advantages and disadvantages in front of previous systems, and a brief description of the different blocks composing the OFDM system. Chapter three provides the implementation details for each of these blocks, and also there is a brief explanation on the theory behind each of the OFDM blocks to provide a better comprehension on its implementation. The fourth chapter is focused, on the one hand, in showing the results of the Matlab/Simulink simulations for the different simulation schemes used and, on the other hand, to show the experimental results obtained using the FPGA to generate the OFDM signal at baseband and then upconverted at the frequency of 3,5 GHz. Finally the conclusions regarding the whole Tx/Rx design and implementation of the OFDM subsystem are given.
Implementing IS-95, the CDMA Standard, on TMS320C6201 DSP
IS-95 is the present U.S. 2nd generation CDMA standard. Currently, the 2nd generation CDMA phones are produced by Qualcomm. Texas Instruments (TI) has ASIC design for Viterbi Decoder on C54x. Several of the components in the forward link process are also implemented in hardware. However, having to design a specific hardware for a particular application is expensive and time consuming. Thus, the possibility of the alternative implementations is of great interest to both customers and TI itself. This research has achieved in successful implementation of IS-95 entirely in software on TI fixed-point DSP TMS320C6201, and met the real time constraint. IS-95 system, the industrial standard for CDMA, is a very complicated system and extremely computationally demanding. The transmission rate for an IS-95 system is 1.2288 Mcps. This research project includes all the major components of the demodulation process for the forward link system: PN Descrambling, Walsh Despreading, Phase Correction & Maximal Ratio Combining, Deinterleaver, Digital Automatic Gain Control, and Viterbi Deccc:r. The entire demodulation process is done completely in C. That makes it a very attractive alternative implementation in the future applications. It is well known that ASIC design is not only expensive and but also time consuming, programming in assembly is easier and cheaper, but programming in C is a much easier and efficient way out, in particular, for general computer engineers. During the whole process, efforts have been devoted on developing various specific techniques to optimize the design for all the components involved. These developments are successfully achieved by making the best use of the following techniques: to simplify the algorithms first before programming, to look for regularity in the problem, to work toward the Compiler's full efficiency, and to use C intrinsics whenever possible. All these attributes together make the implementation scheme great for DSP applications. The benchmark results compare very well to the TI-internal hand scheduled assembly performance of the same type of decoders. The estimated percentage usage of all the components (excluding PN) is only 21.18% of the total CPU cycles available (4,000 K), which is very efficient and impressive.
Design of a Scalable Polyphony-MIDI Synthesizer for a Low Cost DSP
In this thesis, the design of a music synthesizer implementing the Scalable Polyphony-MIDI soundset on a low cost DSP system is presented. First, the SP-MIDI standard and the target DSP platform are presented followed by review of commonly used synthesis techniques and their applicability to systems with limited computational and memory resources. Next, various oscillator and filter algorithms used in digital subtractive synthesis are reviewed in detail. Special attention is given to the aliasing problem caused by discontinuities in classical waveforms, such as sawtooth and pulse waves and existing methods for bandlimited waveform synthesis are presented. This is followed by review of established structures for computationally efficient time-varying filters. A novel digital structure is presented that decouples the cutoff and resonance controls. The new structure is based on the analog Korg MS-20 lowpass filter and is computationally very efficient and well suited for implementation on low bitdepth architectures. Finally, implementation issues are discussed with emphasis on the Differentiated Parabole Wave oscillator and MS-20 filter structures and the effects of limited computational capability and low bitdepth. This is followed by designs for several example instruments.
Signal Processing Requirements for WiMAX (802.16e) Base Station
802.16e provides specifications for non line of sight, mobile wireless communications in the frequency range of 2-6 GHz. It is well implemented by using OFDMA as its physical layer scheme. The OFDM symbol time (sT) is to be selected depending on the channel conditions, available bandwidth and, simulations provide a means of selecting right values of sTin different channel conditions. Additionally it has been shown that certain values of sT outperform others in all conditions, thus invalidating their use. Moreover, a solution proposed by INTEL is also analyzed. One of the major requirements of OFDM is high synchronization. Detecting the timing offset of a new mobile user, entering the network, which is not time aligned using cross-correlation and ‘auto-correlation’ in time domain and cross-correlation in frequency domain at the base station has been simulated. Results point that the processing load can be significantly reduced by using frequency domain correlation of the received data or by using ‘auto-correlation’ followed by cross-correlation on localized data. The use of adaptive antenna system in 802.16e improves the system performance, where beamforming is implemented in the direction of desired user. Capon’s method and MUSIC method have been simulated to compute the direction of arrival for OFDMA uplink. A new user, while in the ranging process, transmits data with unknown time offset and unknown direction. The thesis describes the procedure to find the two unknown one after another.
Least Squares and Adaptive Multirate Filtering
This thesis addresses the problem of estimating a random process from two observed signals sampled at different rates. The case where the low–rate observation has a higher signal–to– noise ratio than the high–rate observation is addressed. Both adaptive and non–adaptive filtering techniques are explored. For the non–adaptive case, a multirate version of the Wiener–Hopf optimal filter is used for estimation. Three forms of the filter are described. It is shown that using both observations with this filter achieves a lower mean–squared error than using either sequence alone. Furthermore, the amount of training data to solve for the filter weights is comparable to that needed when using either sequence alone. For the adaptive case, a multirate version of the LMS adaptive algorithm is developed. Both narrowband and broadband interference are removed using the algorithm in an adaptive noise cancellation scheme. The ability to remove interference at the high rate using observations taken at the low rate without the high–rate observations is demonstrated.
Active Noise Control of a Forest Machine Cabin
Today, a high noise level is considered a problem in many working environments. The main reason is that it contributes to stress and fatigue. Traditional methods using passive noise control is only practicable for high frequencies. As a complement to passive noise control, active noise control (ANC) can be used to reduce low frequency noise. The main idea of ANC is to use destructive interference of waves to cancel disturbing noises. The purpose of this thesis is to design and implement an ANC system in the driver's cabin of a Valmet 890 forest machine. The engine boom is one of the most disturbing noises and therefore the main subjective for the ANC system to suppress. The ANC system is implemented on a Texas Instrument DSP development starter kit. Different FxLMS algorithms are evaluated with feedback and feedforward configurations. The results indicate that an ANC system significantly reduces the sound pressure level (SPL) in the cabin. Best performance of the evaluated systems is achieved for the feedforward FxLMS system. For a commonly used engine speed of 1500 rpm, the SPL is reduced with 17 dB. The results show fast enough convergence and global suppression of low frequency noise.
Active control of automobile cabin noise with conventional and advanced speakers
Recently much research has focused on the control of enclosed sound fields, particularly in automobiles. Both Active Noise Control (ANC) and Active Structural Acoustic Control (ASAC) techniques are being applied to problems stemming from power train noise and road noise (noise due to the interaction of the tires with the surface of the road). Due to the low frequency characteristics of these noise problems, large acoustic sources are required to obtain efficient control of the sound field. This creates demand in the automobile industry for compact lightweight sources. This work is concerned with the application of active control to power train noise, as well as road noise in the interior cabin of a sport utility vehicle using advanced, compact lightweight piezoelectric acoustic sources. First, a test structure approximately the same size as the automobile was built to study the principles of active noise control in a cavity. A finite element model of the cavity was created in order to optimize the positions of the error sensors and the control sources. Experimental work was performed with the optimized actuator and sensor locations in order to validate the model, and draw conclusions regarding the conditions to obtain global control of the sound field. Second, a broad-band feedforward filtered-X LMS algorithm was used to control power train noise. Preliminary power train noise tests were conducted using arrangements of four microphones and up to four commercially available speakers for control. Attenuation of seven decibel (dB) at the error sensors was measured in the 40-500 Hz frequency band. The dimensions of the zone of quiet generated by the control were measured, and show that noise reductions were obtained for a large volume surrounding the error sensors. Next, advanced speakers were implemented for active control of power train noise. The results obtained with different arrangements of these speakers were very similar to those obtained with the commercially-available speakers. These advanced speakers use piezoelectric devices to induce the displacement of a speaker membrane, which radiates sound. Their lighter weight and compact dimensions are a significant advantage over conventional speakers, for their application in automobile. Third, preliminary results were obtained for active control of road noise. The controller used an optimized set of four reference signals to control the noise at one error sensor using one control source. Two sets of tests were conducted. The first set of tests was performed on a dynamometer, which simulates the effects of the road on the tires. The second set of tests was performed on a rough road. Reduction of two to four decibel of the sound pressure level at the error sensor was obtained between 100 and 200 Hz.
Gauss-Newton Based Learning for Fully Recurrent Neural Networks
The thesis discusses a novel off-line and on-line learning approach for Fully Recurrent Neural Networks (FRNNs). The most popular algorithm for training FRNNs, the Real Time Recurrent Learning (RTRL) algorithm, employs the gradient descent technique for finding the optimum weight vectors in the recurrent neural network. Within the framework of the research presented, a new off-line and on-line variation of RTRL is presented, that is based on the Gauss-Newton method. The method itself is an approximate Newton’s method tailored to the specific optimization problem, (non-linear least squares), which aims to speed up the process of FRNN training. The new approach stands as a robust and effective compromise between the original gradient-based RTRL (low computational complexity, slow convergence) and Newton-based variants of RTRL (high computational complexity, fast convergence). By gathering information over time in order to form Gauss-Newton search vectors, the new learning algorithm, GN-RTRL, is capable of converging faster to a better quality solution than the original algorithm. Experimental results reflect these qualities of GN-RTRL, as well as the fact that GN-RTRL may have in practice lower computational cost in comparison, again, to the original RTRL.
Optimization of Audio Processing algorithms (Reverb) on ARMv6 family of processors
Audio processing algorithms are increasingly used in cell phones and today’s customers are placing more demands on cell phones. Feature phones, once the advent of mobile phone technology, nowadays do more than just providing the user with MP3 play back or advanced audio effects. These features have become an integral part of medium as well as low-end phones. On the other hand, there is also an endeavor to include as improved quality as possible into products to compete in market and satisfy users’ needs. Tackling the above requirements has been partly satisfied by the advance in hardware design and manufacturing technology. However, as new hardware emerges into market the need for competence to write efficient software and exploit the new features thoroughly and effectively arises. Even though compilers are also keeping up with the new tide space for hand optimized code still exist. Wrapped in the above goal, an effort was made in this thesis to partly cover the competence requirement at Multimedia Section (part of Ericsson Mobile Platforms) to develope optimized code for new processors. Forging persistently ahead with new products, EMP has always incorporated the latest technology into its products among which ARMv6 family of processors has the main central processing role in a number of upcoming products. To fully exploit latest features provided by ARMv6, it was required to probe its new instruction set among which new media processing instructions are of outmost importance. In order to execute DSP-intensive algorithms (e.g. Audio Processing algorithms) efficiently, the implementation should be done in low-level code applying available instruction set. Meanwhile, ARMv6 comes with a number of new features in comparison with its predecessors. SIMD (Single Instruction Multiple Data) and VFP (Vector Floating Point) are the most prominent media processing improvements in ARMv6. Aligned with thesis goals and guidelines, Reverb algorithm which is among one of the most complicated audio features on a hand-held devices was probed. Consequently, its kernel parts were identified and implementation was done both in fixed-point and floating-point using the available resources on hardware. Besides execution time and amount of code memory for each part were measured and provided in tables and charts for comparison purposes. Conclusions were finally drawn based on developed code’s efficiency over ARM compiler’s as well as existing code already developed and tailored to ARMv5 processors. The main criteria for optimization was the execution time. Moreover, quantization effect due to limited precision fixed-point arithmetic was formulated and its effect on quality was elaborated. The outcomes, clearly indicate that hand optimization of kernel parts are superior to Compiler optimized alternative both from the point of code memory as well as execution time. The results also confirmed the presumption that hand optimized code using new instruction set can improve efficiency by an average 25%-50% depending on the algorithm structure and its interaction with other parts of audio effect. Despite its many draw backs, fixed-point implementation remains yet to be the dominant implementation for majority of DSP algorithms on low-power devices.
Benchmarking a DSP processor
This Master thesis describes the benchmarking of a DSP processor. Benchmarking means measuring the performance in some way. In this report, we have focused on the number of instruction cycles needed to execute certain algorithms. The algorithms we have used in the benchmark are all very common in signal processing today. The results we have reached in this thesis have been compared to benchmarks for other processors, performed by Berkeley Design Technology, Inc. The algorithms were programmed in assembly code and then executed on the instruction set simulator. After that, we proposed changes to the instruction set, with the aim to reduce the execution time for the algorithms. The results from the benchmark show that our processor is at the same level as the ones tested by BDTI. Probably would a more experienced programmer be able to reduce the cycle count even more, especially for some of the more complex benchmarks.
