ISO/IEC 14496-3:1998 pdf

MPEG-4 audio coding integrates the worlds of speech and high quality audio coding as well as the worlds of sound synthesis and the representation of natural audio.  The sound synthetis part is comprised of tools for the realisation of symbolically defined music and speech. This includes MIDI and Text-to-Speech systems. Furthermore, tools for effects processing and 3-D localisation of sound are included, allowing the creation of artificial sound environments using artificial and natural sources. 
Synthetic audio is described by first defining a set of ‘instrument’ modules that can create and process audio signals under the control of a script or score file.  An instrument is a small network of signal processing primitives that can emulate the effects of a natural acoustic instrument.  A script or score is a time-sequenced set of commands that invokes various instruments at specific times to contribute their output to an overall music performance.  Other instruments, serving the function of effects processors (reverberators, spatialisers, mixers), can be similarly invoked to receive and process the outputs of the performing instruments.  These actions can not only realise a music composition but can also organise any other kind of audio, such as speech, sound effects and general ambience.  Likewise, the audio sources can themselves be natural sounds, perhaps emanating from an audio channel decoder, thus enabling synthetic and natural sources to be merged with complete timing accuracy.
TTS is becoming a rather common interface and plays an important role in various multi-media application areas.  For instance, by using TTS functionality, multi-media contents with narration can be easily composed without recording natural speech sound.  Moreover, TTS with FA/AP/MP functionality would possibly make the contents much richer. In MPEG-4 activity, common interfaces for TTS and TTS for FA/AP/MP are proposed. The proposed MPEG-4 TTS functionality; Hybrid/Multi-Level Scalable TTS, can be considered as a superset of the  conventional TTS framework. This extended TTS can utilize prosodic information of natural speech in addition to input texts and can generate much higher quality of synthetic speech. The interface and its bitstream format is strongly scalable; for example, if some parameters of prosodic information are not available, it then generates the missing parameters by rule.  Still the basic idea for the scalable MPEG-4 TTS interface is it can fully utilize all the provided information according to the level of user’s requirements. The functionality of this extended TTS thus ranges from conventional TTS to natural speech coding and its application areas, from simple TTS to AP with TTS and MP dubbing with TTS.