Print This Page

9. Quality assessment methods

In chapters 1 and 3 of this white paper, definitions are proposed to support discussions on audio quality and sound quality. Chapter 8 presented a selection of issues on operational quality. For discussions to make sense, in addition to definitions, information about the audio system is required - including judgement statements on the audio quality, sound quality and operational quality. Operational quality is generally covered by comparing the published system devices and the system design’s operational specifications to the operational requirements. For the audio quality and sound quality however, things are a little more complicated. The assessment of both audio quality and sound quality is a subject of discussions in the professional audio market(*9A). Contributing to the discussions from a ‘Performance & Response’ viewpoint, this chapter presents two basic methods for the quality assessment of an audio system: analysis of electrical measurements, and listening.

9.1 Quality assessment through electronic measurements

To assess the audio quality of a networked audio system, the Performance of the system can be measured by electronic measurement equipment such as level meters, oscilloscopes, FFT analysers and impulse response analysers. Because the requirements for the Performance of a system are so clearly defined - see chapter 1, the measurements can be analysed and interpreted using strict definitions and specifications. In cases where definitions and measurements of a found audio quality problem don’t exist yet, a new measurement method has to be invented. But it is assumed that - after more than a century of world wide research in the field of electronic sound reproduction systems - the majority of audio quality issues in audio systems have been defined, electronically measured and analysed. Most manufacturers of electronic equipment list the most relevant measurements in their specifications - although sometimes different definitions are used, making comparison between devices from different manufacturers difficult or even impossible.

To assess the sound quality of an audio system, the Response of the system can be measured using the same electronic measurements as used for the measurement of the system’s Performance. But the result of an electronic measurement doesn’t say anything about sound quality by itself - it has to be translated. A complication is that the translation of electronic measurements to (individual) hearing experiences is not standardized. For example: a measured equalizer curve can be interpreted as ‘good sounding’ by one individual, and ‘bad sounding’ by the next. To obtain an ‘average audience’ translation matrix of physical sound characteristics to perceptional sound characteristics, clinical research is required using a large population of listeners. In contradiction to electronic Performance measurements, research on the translation of electronic Response measurements to sound quality perception has not been conducted on large scale yet(*9B). Instead, the sound quality (Response) of many audio devices (or DSP algorithms - ‘plug-ins’) are most commonly referenced to individual opinion leaders in the professional audio field, articles of respected journalists in professional audio magazines, or to the overall sound quality image of the manufacturer. Of course, also many individual listening sessions are conducted to assess the sound quality of audio devices, but then there are many complications involving the translation of the hearing sensation to the device’s sound quality. This topic is presented in chapter 9.2.

Electronic measurements of analogue and digital systems is not difficult as both analogue and digital measuring equipment is widely available. The measurement of the acoustic parts of a system can be performed by the same equipment, but it requires a calibrated measurement microphone to transform acoustic signals into analogue signals. Often, individual parts of the system can be measured by either probing internal circuits in the device, or by bypassing parts of the system. A special case of bypassing is the measurement of the Performance of a system with the Response of the system bypassed - eg. switching off all processing. A big advantage of electronic measurements is that the system can be measured using a controlled test signal - making the measurement independent of the sound source, and also making it possible to reproduce the measurement at different times and locations for confirmation or to obtain a high statistical significance.

Return to Top