5. Testing

The ideal artificial reverberation is created when an anechoic input signal is convolved with an acoustic impulse response. This ideal reverberation will be compared with the hybrid algorithm to evaluate its transparency. The hybrid algorithm was implemented as a Steinberg VST plugin equipped to perform on any computer running one of the Steinberg sequencing software packages (Cubase, Cubasis, etc.). As stated in Chapter 1, the goal of this thesis is to eliminate the inverse relationship between high fidelity artificial reverberation and computation cost. Therefore, in testing this new architecture we will be focusing on the similarities of its output with that of the ideal (linear convolution) output as well as examining the processor demands imposed by the implementation.

Testing the performance of this algorithm was done with a double-blind comparator test. The test determines if a person can discriminate between the output of the hybrid algorithm (convolution of truncated impulse response and recursive filtering) and the direct linear convolution of the full impulse response.

5.1 Source Material
All sound files are 16 bit, 44.1 kHz, mono .WAV files. Each sound was processed and exported using Steinberg Cubasis 3.7

 

Impulse Responses Used (44.1kHz, 16bit, mono)

IR1.wav
medium hall (taken from Real Reverberation Winamp Plugin)

IR2.wav
Bergamo Cathedral, Italy (recorded by A. Farina using swept sine wave, w/ aurora software and dodecahedron loudspeaker)

Input Signals Used (44.1kHz, 16bit, mono)

speech.wav
male voice speaking "echo" (recorded directly to PC using Cool Edit Pro)

orchestra.wav
from Symphony #9 in C minor (from Denon – Anechoic Orchestral Music Recording)

drums.wav
drum set loop

guitar.wav
dry electric guitar (recorded directly to PC using Cool Edit Pro)

Two impulse responses were tested with four music samples described in the table above. These music samples encompass an adequate cross-section of typical sound sources that would use artificial reverberation effects.

5.2 Time-Frequency Comparison
Comparing the Energy Decay Relief plots of the original and hybrid impulse responses will determine the degree of transparency that the hybrid algorithm will produce. Figure 5.1 shows the Energy Decay Relief plot of the full impulse response (IR1.wav) and the impulse response of the corresponding hybrid algorithm.


Figure 5.1: EDR of original and hybrid impulse response
(IR1.wav)

Figure 5.2 shows the Energy Decay Relief plot of the full impulse response (IR2.wav) and the impulse response of the corresponding hybrid algorithm.


Figure 5.2: EDR of original and hybrid impulse response
(IR2.wav)

The drum set loop input signal has a wide frequency range with heavy transient attacks from the percussion instruments. This sample proved to be the best overall test for proper comparison because it provided more obvious solutions to voicing needs.

Figure 5.3 shows the waveform of the drum set loop using the medium hall impulse response (IR1.wav). The top waveform is the output due to direct linear convolution while the bottom is the output of the hybrid algorithm.


Figure 5.3: Waveform of drum set sample

Figure 5.4 shows the frequency response of each of the signals in Figure 5.3. This is the overall frequency content of the entire drum set loop providing only a general view.


Figure 5.4: Frequency response of drum set sample

With the frequency content suitably identical, the reverberation decay becomes the final concern. This portion is controlled with the comb and all-pass filter feedback parameters and gain relative to the convolution stage.

Figures 5.5 and 5.6 show the time and frequency contents of the decay portion.


Figure 5.5: Waveform of drum set decay


Figure 5.6: Frequency response of drum set decay

When using the medium room impulse response (IR1.wav) each of the 4 input samples showed similar transparency, but with some clear distinctions with particular instrumentation. For example, the pick attack of the guitar sample and high brass notes of the orchestra sample were more distinct in the hybrid output. However, these anomalies were not apparent in the frequency response plots.

Figures 5.7 and 5.8 illustrate the time and frequency response of the reverberated guitar sample using the medium hall impulse response.


Figure 5.7: Waveform of guitar sample
(IR1.wav)


Figure 5.8: Frequency response of guitar sample
(IR1.wav)

Figures 5.9 and 5.10 illustrate the time and frequency response of the reverberated orchestra sample using the medium hall impulse response.


Figure 5.9: Waveform of orchestra sample
(IR1.wav)


Figure 5.10: Frequency response of orchestra sample
(IR1.wav)

Figures 5.11 and 5.12 illustrate the time and frequency response of the reverberated speech sample using the medium hall impulse response.


Figure 5.11: Waveform of speech sample
(IR1.wav)


Figure 5.12: Frequency response of speech sample
(IR1.wav)

Figures 5.13 and 5.14 illustrate the time and frequency response of the reverberated drum set sample using the Bergamo Cathedral impulse response.


Figure 5.13: Waveform of drum set sample (IR2.wav)


Figure 5.14: Frequency response of drum set sample (IR2.wav)

Figures 5.15 and 5.16 illustrate the time and frequency response of the reverberated guitar sample using the Bergamo Cathedral impulse response.


Figure 5.15: Waveform of guitar sample
(IR2.wav)


Figure 5.16: Frequency response of guitar sample (IR2.wav)

Figures 5.17 and 5.18 illustrate the time and frequency response of the reverberated orchestra sample using the Bergamo Cathedral impulse response.


Figure 5.17: Waveform of orchestra sample (IR2.wav)


Figure 5.18: Frequency response of orchestra sample (IR2.wav)

Figures 5.19 and 5.20 illustrate the time and frequency response of the reverberated speech sample using the Bergamo Cathedral impulse response.


Figure 5.19: Waveform of speech sample (IR2.wav)


Figure 5.20: Frequency response of speech sample (IR2.wav)

5.3 ABX testing
The testing mechanism used was the PCABX testing software. The user interface is shown in Figure 5.21.


Figure 5.21: PCABX user interface

The linear convolution was performed for all four inputs for both impulse responses in MATLAB and output to a .WAV file (input "A"). The hybrid algorithm was implemented as a Steinberg VST real-time plugin and exported as .WAV (input "B"). PCABX randomly chooses which of the two shall be input "X" and the user must determine which of the two is "X".

Each of the 20 test subjects tested each of the 8 output samples 20 times each. If the subject was able to correctly identify "X" greater than 75% of the time, it was assumed that they could discriminate between the hybrid algorithm output and the linear convolution. If greater than 75% of the test subjects were able to discriminate, it was assumed that the hybrid algorithm output was not transparent to the linear convolution.[3]

averages

IR1.wav

IR2.wav

drums.wav

66.43%

85.71%

guitar.wav

71.07%

85.36%

orchestra.wav

73.93%

76.07%

speech.wav

78.21%

86.79%

No clear affirmation can be made about the quality of this hybrid algorithm based on the listening test results. The averages seem to show that the test subjects were able to discriminate between the linear convolution and the hybrid output for a majority of the input samples. However, investigating the standard deviations across the 20 test subjects reveals huge deviations from the average.

standard deviations

IR1.wav

IR2.wav

drums.wav

19.16%

23.60%

guitar.wav

18.21%

21.52%

orchestra.wav

20.77%

21.32%

speech.wav

13.10%

19.77%

For example, the average listening test score for the guitar sample under the Bergamo Cathedral impulse response, was 85.36% but the scores were on average 21.52% away from this value. An average score of 85.36% would suggest that the subjects were able to discriminate between the two signals overall. However, a 21.52% standard deviation indicates that some of the subjects could discriminate very well, while others could not. With these high standard deviation values, we can not conclude that the hybrid algorithm output can not be distinguished from the linear convolution

chapter 4 | chapter 6 | table of contents | title page