2744total visits.
Ivan Capeller
Professor of Sound for Cinema at the School of Communications (Escola de Comunicação) of the Federal University of Rio de Janeiro (ECO-UFRJ),
Professor of Semiotics and Culture for the Post-Graduation Program on Sciences of Information (PPGCI-UFRJ) of the same university.
Abstract
In what ways do the study of sounds and of the audible field allows a re-evaluation of questions regarding C.S.Peirce’s semiotics and its implications to the philosophy of language? This essay is an attempt to rethink the relationship between the mimetic and the semiotic elements of language through a research on how the process of hearing relates to sounds and meaning. To draw a map of the audible field, one must follow Peirce’s triadic logic in a double articulation of Jacques Rancière’s three political orders of sensitivity (ethical, poetical and aesthetical) with Michel Chion’s three ways of hearing (reduced, causal, semantical). Peirce’s three logical categories (firstness, secondness and thirdness) enables numberless triadic combinations between ways of hearing, orders of sensitivity and the signifier’s operations. The audible field unfolds itself according to the following partition: a) reduction to the ethical dimension of a strict regulation of its own volume and pitch variations; b) representation according to mimetic criteria of causal attribution of sounds to its supposed sources; c) linearization into a signifying sequence of organized coded sounds. The final result is a semiotic understanding of the audible field as an organized gap between sounds and signs.
Keywords: Hearing. Aesthetics. Semiotics.
Résumé
Dans quelle mésure l’étude des sons et du champ de l’audible permet-il une reévaluation des certaines questions relatives à la philosophie du langage? Cet essai veut repenser la relation entre les éléments mimétique et sémiotique du langage avec l’aide du champ de l’audible et a partir de ce que les sons et l’écoute nous renseignent sur la corrélation entre les processus de codification et signification du langage et les formes de répartition sociale du voir et du dire. Le champ de l’audible est toujours traversé par des oscillations capables de déborder les limites du sensible – soit en découpant plus clairement ses marges et frontières, soit en défaisant irrémédiablement les lignes qui lient le son au sens. Le champ de l’audible n’est pas capable de s’organiser, du point de vue sémiotique, avec la même précision et rigueur structurelles vérifiables pour les champs du visible et de l’énonçable. Une cartographie des fêlures qui partagent le champ de l’audible basée sur la logique des catégories phénoménologiques de Peirce devra permettre une articulation des trois régimes de partage du sensible avec trois modes d’écoute. Le champ de l’audible peut être déplié entre ses modes d’écoute, d’une façon simultanée et/ou successive, selon la tripartition: reduction à la dimension éthique d’une stricte regulation des ses propres variations d’intensité et hauteur, répresentation selon des critères mimétiques d’atribution causal des sons à ses sources, qu’elles soient supposées ou pas, sémantisation dans une sequence signifiante des sons organisés selon un code donné.
…ce seul objet dont le néant s’honore
Mallarmé
Theory of Language and Audible Field
The purpose of this essay is to rethink the relationship between the semiotic and the mimetic elements present in every language through a set of questions raised by the audible field; what do sounds and hearing may tell us about the correlation between language’s codification and semantization processes and the different social forms of seeing things and speaking about them?
The traditional duality that has been established between the verbal and the visual fields not only silences the presence of sounds and the role of hearing in semiotic non-musical processes, but also overlook completely the innumerable possibilities of inter-semiotic translation inherent to the verbal discourse’s sphere as well as to all kinds of synaesthetic phenomena in which words, images and sounds get together in order to provide some sort of (un)codified meaning.
These different possibilities of articulating meaning were already put forward by Walter Benjamin in two important (although relatively neglected) texts respectively called Doctrine of the Similar and On the Mimetic Faculty. In a speculative attempt to think the general relationship between meaning and language, different modes of articulation between its mimetic and semiotic elements are to be considered, according to Benjamin: whenever we think about signs as mere arbitrary instruments of communication, we tend to repress the mimetic potential of language’s synaesthetic faculties, since mimetic phenomena of correspondences and similarities (sensuous or non-sensuous alike) are probably at work in language’s filo- and onto-genetic development in a much more powerful way than processes related primarily to a set of conventional signs. Thus, a radically anti-functionalist theory of language is deemed necessary in order to dislocate the conventional arbitrariness of the sign from its usual structuring linguistic function:
“(…) If language, as is evident, is not an agreed upon system of signs, we will be constantly obliged to have recourse to the kind of thoughts that appear in their rawest, most primitive form as the onomatopoetic mode of explanation. (…) The key which finally makes this thesis fully transparent lies concealed in the concept of a non-sensuous similarity. For if words meaning the same thing in different languages are arranged about that signified as their centre, we have to inquire how they all – while often possessing not the slightest similarity to one another – are similar to the signified at their centre.” (Benjamin, 2005, 696)
The Benjaminian doctrine of non-sensuous similarities, as opposed to the epistemic emphasis in the paradigmatically synchronic aspects that structure the verbal languages, points rather to the syntagmatic diachronic dimension of language as a synaesthetic process in which the mimetic production of correspondences does not only represent a specific type of articulation between a signifier and a signified – as in the onomatopoeia cases – but the incessant reshaping of three possible opposite conceptual pairs that criss-cross all language(s) in themselves and in between them all, that is, as well in an intra- as in an inter-semiotical axis of translation:
“It is thus non-sensuous similarity that establishes the ties not only between what is said and what is meant, but also between what is written and what is meant, and equally between the spoken and the written. And every time, it does so in a completely new, original, and underivable way. The most important of these ties may, however, be the one mentioned last – that between what is written and what is said. For the similarity which reigns here is comparatively the most non-sensuous. It is also the one which takes the longest to be reached.” (Benjamin, 2005, 696-697).
The passage from a meaningful intention to its respective forms of expression, oral or written, alters incessantly the threshold between thoughts, words and images. Between concrete, sensuous correspondences and abstract symbolic conventions, images and sounds are able to establish with ideas different relations based on correspondences and non-sensuous analogies that precipitate into oral sayings as well as into the visuality of written signs. Phonology and graphology may be thought of, thus, as diachronic studies of the strata deposited throughout the centuries into language by analogical ways of thinking[1]:
“Graphology has taught us to recognize in handwriting images that the unconscious of the writer conceals in it. It may be supposed that the mimetic process which expresses itself in this way in the activity of the writer was, in the very distant times in which script originated, of utmost importance for writing. Script has thus become, like language, an archive of non-sensuous similarities, of non-sensuous correspondences. But this aspect of language, as well as of script, does not develop in isolation from its other, semiotic aspect. Rather, the mimetic element in language can, like a flame, manifest itself only through a kind of bearer. This bearer is the semiotic element. Thus, the nexus of meaning of words or sentences is the bearer through which, like a flash, similarity appears. For its production by man – like its perception by him – is in many cases, and particularly the most important, tied to its flashing up. It flits past. It is not improbable that the rapidity of writing and reading heightens the fusion of the semiotic and the mimetic in the sphere of language. (BENJAMIN, 2005, 722)
And in a brief appendix to the Doctrine of the Similar, Benjamin warns us to the fact that
“Our gift for seeing similarity is nothing but a weak rudiment of the once powerful compulsion to become similar and also to behave mimetically. And the lost faculty of becoming similar extended far beyond the narrow perceptual world in which we are still capable of seeing similarities.” (BENJAMIN, 2005, 698).
Benjamin unfolds a large overview about the relationship between language and history in which the mimetic properties of language seem to be gradually overwhelmed by its “communicational” semiotic functions while the formal aspects of a given language – heretofore to be understood as its “code” – become preponderant. Nevertheless, although consecrated by the verbal language cultural hegemony and its epistemological primacy as a central object of study for any linguistics or semiology, the binary polarization between the oral and the written hides the fact that this opposition is the intra-semiotic equivalent of an inter-semiotic opposition of a wider spectrum. This is so because the written and the oral are both verbally codified forms of articulating audible and visible signifiers into verbal statements; but there are forms of meaning that are able to reshape the many different possible correspondences between these three fields of expression without necessarily crystalizing and fossilizing the mimetic analogical processes at work in sedimented layers of symbolical conventions organized into linguistic strata composed by a certain mixture of statements and visibilities.
Hence the necessity of a renewed concept of language capable, not only of expanding its investigative scope towards non-verbal codes, but, above all, capable of digging the conceptual duality between words and images in order to unveil the audible field as the excluded third of the sign’s double articulation postulated by Saussure (acoustic image + idea). Thus, behind the linguistic dyad of the spoken and the written signs in which thought paradigms must codify visible and speak able syntagms, emerges a semiotic triad that intertwines synaesthetically the audible to the visible and to the speakable: images, words and sounds establish between themselves series of mimetic correspondences, sensuous or non-sensuous, motivated or not, that belong to pre-signifying, counter-signifying or post-signifying strata of meaningful articulations, causing shakedowns and “sismical” perturbations in the relative stability of the more superficial layers of discursive enunciation.
Beyond the specific importance attached, in this theoretical perspective, to the so-called sound studies and to the semiotic research on the audible field, it is also important to highlight the inevitable passage from a binary logic, based on the verbal model of double articulation, to a ternary logic that multiplies the possibilities of a double articulation of words to images (and vice-versa), words to sounds (and vice-versa), as well as sounds to images and vice-versa, introducing them all to an incessant movement of syntagmatic reshaping of meaning that’s not entirely strange to the Peircean conception of a semiotics considered as a noetics, that is, conceived as a logical process in which thought is able to shape and reshape the very fabric of representations that determines its own objects in a continuous movement, criss-crossing and crossfading intra and inter-semiotically all signs, languages and codes.
Benjamin’s theory of language is akin to Peirce’s triadic conception of the sign in a very specific point: his doctrine of the similar presupposes the same possibility of a continuous analogical kinaesthetic and synaesthetic passage between different fields of expression, linguistic codes and specific signs. Crossing these boundaries, according to Benjamin, implies a material, mimetic process subject to multiple meaning possibilities determined by a floating association of ideas. On the other hand, according to Peirce, a typology of signifying functions must be the result of a logical deduction due to the iteration between the phenomenological categories of experience (firstness, secondness, thirdness) and the conceptual categories at work in the signifying process (representâmen, object, interpretant). The signifying process in itself is thought of as a continuous passage, following a tight semio-logical path, from more or less vague pre-signifying possibilities of articulation (hipossemes) to the establishment of a polysemic hermeneutical circuit of statements and assertives more or less adequate to their object(s). Thus, imitation by mimetic correspondence apparently finds itself regulated in a generative sign system that is no less based on the primacy of the verbal modes of enunciation over other visual and auditive modes of expression than other comparable semiological models.
Nonetheless, our main hypothesis here is that applying Peirce’s semio-logics of thought to the audible field will enable us to rethink the tension pointed out by Benjamin between the mimetic and the semiotic element of any language or code, inasmuch as this tension is not plainly thinkable just inside each and any verbal language individually considered or even, inter-semiotically speaking, just between the order of verbal statements and the order of visual images. The audible field, compared to the visual and the verbal fields, is considered as a less reliable and more unstable field of codified meaning. Its gaps and fissures reveal more easily the inherent dichotomy or tension belonging to the very fabric of human language, all the more so, when they have been mapped with rigorous semio-logical Peircean parameters.
Of all these gaps and fissures, the one capable of splitting our modes of hearing in a three-way partition that presides over our most basic and common auditioning performances should be the object of a much more considered attention from all language and communications researchers.
Threefold Hearing and the Audible Field
The Brazilian semiotician Lucia Santaella has already highlighted the correspondence between the three Peircean phenomenological categories of experience and the three modes of hearing postulated by Michel Chion in his fundamental work L’Audio-Vision:
“Michel Chion (…) hás established three different modes of hearing that relate to different objects: causal hearing, semantical hearing and reduced hearing. As their correspondence to the Peircean categories is strikingly obvious (…) I’ll change its order to (1) reduced hearing, (2) causal hearing and (3) semantical hearing. (…) Chion (…) establishes a triad that, although not directly based on Peirce, presents a perfect correspondence with his categories if we take the listener’s behaviour, during the act of hearing, as the sole axis of classification.” (SANTAELLA, 2005, 85-86).[2]
Although there are many other typologies of the audible field, based on hearing and listening processes inasmuch as some intrinsic features of sounds considered in themselves, the typology developed by Chion has the advantage of avoiding the subordination of hearing and the audible field to cognitive epistemic criteria that tend to classify it in terms of a greater or lesser organizational complexity of sounds – considered by themselves – or of hearing considered in its greater or lesser capacity as an emotive, energetic or intellectual interpretant of sounds. Too bound to symbolic conventionalism understood as the structure of any codification process, these criteria cast over the audible field the same repressive attitude towards the synaesthetic-mimetic faculties of language already pinpointed by Benjamin regarding other forms and fields of semiotic expression.
In the specific case of the audible field, music is the privileged form of a would-be linguistic paradigm, or third order legi-sign, for all cultural configurations of the audible field. It is, nevertheless, a most problematic solution when compared to other linguistic and semiotic paradigms in charge of regulating the edge between what’s to be seen and what’s to be spoken. Music is the kind of synaesthetic, mimetic expression par excellence, based on the acoustic materiality of the sympathetic resonance phenomenon and on the Peircean firstness of its emotive effects and iconic allures. Not by any chance, the history of music presents a great number of anachronisms regarding the history of literature and of the arts, both aesthetically (presence of the baroque Kapelmeister J.S.Bach during the first half of the classicist 18th century, presence of the ultra-romantic Wagner throughout the whole impressionist and naturalist artistic trends of the final 19th century[3]) and semiologically, since music was the last of the classical arts to be fully codified and regulated by an ideal poetics of artistic representation, and the one in which this process was never entirely realized, especially regarding the polemical field of musical semantics..
But why is the audible field unable to organize itself, semiotically speaking, with the same precision and structural coherence easily verifiable for the visual and verbal fields?
Because the audible field, even when it’s doubly articulated by a musical code, is kinaesthetically and synaesthetically covered up by a plethora of verbally regulated statements and socially controlled visibilities that erase almost completely the possibility of an autonomous thirdness (expressed in the artistical perspective of “pure music”) – almost always already captured by orality (articulated speech, singing and lyrical voices) as well as by the visible field whenever a voice is attributed to a body and/or synchronized to it through movement, rhythm and/or dance.
However, between statements and visibilities, a semiotics of the audible field can be reasonably articulated through an expanded concept of music conceived to be applied as a paradigm to different ways of third-order autonomous organizations of sounds. It’s a clever solution, extremely helpful in demonstrating how the history of modern music (starting to unfold from the late nineteenth century tonal system dissolution) points to a neat reduction of its audible structures into hyposemic configurations of the second and first orders. As a result, a rigorous application of Peirce’s semio-logical principles should be able to demonstrate (through logical deduction) the possibility of extracting and locating – in an expanded but still musically oriented audible field – all kinds of musical and non-musical sound structures composed or proposed by the great exponents of concrete, electronic and electro acoustical music[4].
On the other hand, the already mentioned correspondence between Peirce’s three phenomenological categories of experience with Chion’s three modes of hearing, allows for a semiotics of the audible field that doesn’t exclude musically organized sounds without positing them as the necessarily universal, supreme and final interpretants of the audible field. The concept of “reduced hearing” may bring to our minds some sounds and structures musically thought of by Xenakis or John Cage, but it may also be used as a movie production’s sound mixer working technique or pertain to situations where we can’t identify and locate the sounds that are surrounding us or distinguish the real or imaginary character of its sources; “causal hearing” may refer us directly to certain musical instruments or to a singer’s voice, but it also helps us identifying day-to-day noises or ordinary people’s voices on the telephone; “semantical hearing” might well be about audition abilities capable of understanding very complex musical structures, but it also refers to orality and the central role played by verbal organized speech in all human societies.
Even more important is the fact that the three-way division of the audible field by the three modes of hearing reveals, inside the audible field, the same repressive processes posited by Benjamin’s Doctrine of the Similar: only reduced hearing would be able to unveil firstness’s audible primacy as such, since causal hearing is pervaded by an indexical secondness related to what’s already presentable as a visible object, whilst semantical hearing is inextricably bound up to thirdness-related already symbolically structured sounds (like a set of verbal phonemes or of musical notes and chords); these sounds act as signifiers tied together to a chain of audible enunciations organized into verbal discourses or into musical syntagma.
Therefore, the audible field founds itself fissured into three distinct possibilities of organizing the experience of sounds correlated to the three peircean categories of phenomenological experience: thirdness is correlated to a semantical hearing capable of activating the required interpretants for a verbal or musical auditive message; secondness relates to causal hearing inasmuch as sounds can act as traces or indexical marks of the supposed presence of its real or imaginary sources or objects, be them visible or not; and firstness correlates to a special mode of hearing in which the listener’s attention is reduced to following the free floating amplitude and frequency variations of sounds (its’ representamens) in their random modulation and environmental propagation.
Semiotics of the Audible Field
The correspondence between Chion’s three modes of hearing and Peirce’s three categories shapes a semiotics of the audible field that allows for an understanding of the way the generative component of verbal language covers up this field almost entirely through the invocation of semantical hearing. Nonetheless, the audible field presents a high rate of semiotic instability and a synaesthetic mimetic potential capable of acquiring unpredictable counter or post-significant properties. That’s why the everlasting fissures of the audible field better unveil the constitutional tension between the mimetic and the semiotic elements inside any language: even a semiotics understood as noetics won’t lead to a comprehensive overview of all morphological aspects of the audible field that would allow for a complete map of the main lines that criss-cross its many strata and multiple layers of expressive articulation. When it doesn’t find itself completely tied up to orality and verbal meaning, or to the visual field through different synchronical effects, the audible field is the most resilient counter-significant factor of cognitive disruption: a cry is always a condemnation of articulated reasonable speech, as well as a dissonance can always be heard as an open stance against musical harmony and noises are often posited as the last desperate possible opposition to a disquieting silence that gaze sometimes upon the visual realm.
Thus, the formulation of a theory of language articulated to the audible field must rely upon a theory of mimêsis understood not only as a supplementary element of meaning, but also as a social process that calls for a cultural and political regulation on a supra-semiotic level of analysis. We find the principles of such a theory in the threefold distribution of the sensible conceptualized on Jacques Rancière’s Politics of Aesthetics. Despite his traditional emphasis in the visual and verbal aspects of the arts and dimensions of language, the audible field can be constantly eavesdropped in Rancière’s three modes of distribution of the sensible under the form of a contradiction: unpredictable bodily sounds (moaning, crying, sighing…) are generally considered as noisy destroyers of the rational order brought in by articulated speech through words (and regulated by the arts of oratory and rhetoric), but, when inscribed in a chorographical collective social order, organized sounds are able to synchronize the citizen’s body to the Polis’ rule and offer a political and cultural alternative to theatrical mimicry and its simulacra.
Considering the mimetic phenomenon as a social and a political problem, there must be a distribution of the audible necessarily correlated to the distribution of the visual and the verbal, because the three distinct modes of organizing and distributing the sensible – the ethical, the poetic-mimetic and the aesthetic – are bound up to the fact that mimetic instability isn’t easily controllable by any kind of aesthetic or semiotic codification:
“Mimêsis is not the law that brings the arts under the yoke of resemblance. It is first of all a fold in the distribution of ways of doing and making as well as in social occupations, a fold that renders the arts visible. It is not an artistic process but a regime of visibility regarding the arts. A regime of visibility is at once what renders the arts autonomous and also what links this autonomy to a general order of occupations and ways of doing and making.” (RANCIÈRE, 2004, 22).
The concept of mimêsis thus considered, the problem of a potential spreading of politically uncontrollable “noisy” statements through the audible field may imply onto-political questions much more dangerous than the ones generally proposed regarding the greater or lesser cognitive trustworthiness of words and images. But the late cultural hegemony of music – considered as an ideal model for a semiotic-cognitive organization of the world of sounds – nonetheless, is always a major recurrent problem to many-a-different attempt to sketch a semiotics of the audible field.
That’s why music’s notorious delay, regarding the history of literature and the plastic arts, in producing a sound poetical code capable of regulating its synaesthetic-mimetic expressiveness (a goal that was achieved only with the tonal theory of harmonics, at the beginning of the 18th century) escapes entirely Rancière’s analytical scope when dealing with the constitution of the 17th’s century poetic-mimetic arts regime. However, the discrepancies and anachronisms that permeate the history of music, when analysed through the audible fissures unveiled by a threefold partition of hearing, offer precious evidence about the relationship between the mimetic and the semiotic elements – not only for the audible field but also regarding language itself.
In any case, Rancière’s solution for the onto-political problem of the social status of the work of art, understood as a specific type of codified regulation of mimetic phenomena, demonstrates the continuous and recurrent character of his logic: from social and religious restraint (ethical regime) to representational codification (poetical-mimetic regime) and cultural dissemination (aesthetical regime), all verbal statements and visual representations are to be regulated and controlled not only as sensible (and sensuous) material messages striving to be heard and seen, but first and foremost as meaningful ideas urging to become effective thoughts: “a mode of articulation between ways of doing and making, their corresponding forms of visibility, and possible ways of thinking about their relationships (which presupposes a certain idea of thought effectivity)” (RANCIÈRE, 2004, 10).
This solution is akin to Peirce’s triadical logic in its ordinal, recursive and inclusive character. This doesn’t mean, however, that establishing correspondences and similarities between a semiotics understood as a noetics and the threefold ways of distributing the sensible must imply a circular demonstration of a supposed conceptual identity between semiotic thought and aesthetical theory, since this way of thinking would lead precisely to a theoretical mirroring of the repressive processes at work over the mimetic aspects of language – so dear to the aesthetical reflexion – by language’s most prominent semiotic aspects. As a result of the inherent tension between the mimetic and the semiotic in every language related phenomenon, possible correspondences (sensuous and non-sensuous) between Peirce’s, Chion’s and Rancière’s triads won’t necessarily determine identities of form and/or contents between the elements pertaining to each one of them: there’s only serial isomorphic and structural homology, because the isomorphic structures belonging to the meaning generative processes can reveal similarities and differences alike between the audible and the visual and verbal fields. This incessant shaping and reshaping of similarities and differences between these three fields happens through a certain movement traced by meaning around language (and by language around meaning) as a regular feature of language’s transformational component and its pre-significant, counter-significant and post-significant instances.
In Chion’s triad of the three modes of hearing, language is simultaneously the ground and the goal of all hearing activities, through a sort of cognitive circularity that allows, precisely, for the extraordinary correspondence, highlighted by Santaella, with the Peircean phenomenological triad and that brings on a thorough description of the continuous possible shifting from hearing’s pre-significant instances (phenomenological reduction of hearing, indexical causality suggested by the perception of sounds) to plainly meaningful and cognitive hearing instances (linguistic hearing and musical structure’s hearing).
Rancière’s triad of the distribution of the sensible points in the opposite direction to demonstrate the prominently ethical character of the pre-significant instances of the sensible – always in need of social control – while postulating the possibility of a second regime of sensible representations organized by one or more poetic-mimetic codes regulating both the order of statements (what’s to be said or not) and the order of visibilities (what’s to be seen or not), and thus corroborating Santaella’s association between Peircean secondness and the indexical primacy of figurative images and of the visual field.
As for the way Rancière’s aesthetical regime is intertwined with the Peircean concept of thirdness, we hit here the very core of Benjamin’s assertion about how modern theories of language had consummated the repression of language’s constitutive mimetic element: between the aesthetical and the semiotic, the resistance presented by the first to its entire absorption by the second (a crystal clear phenomenon when it comes to all known musical languages) represents, at the same time, an advance and a withdrawal concerning language’s cognitive ambitions. A withdrawal expressed by its sheer resistance to a closed semantical structure, installing an open fissure between its form and its contents and exposing the materiality of its plan of expression as a void signifier; and an advance inasmuch as this same resistance allows for fast passage to an aesthetical, counter or post-significant regime in which the adequacy between the order of statements and the order of visibilities is no longer guaranteed by the traditional poetic-mimetic codes of representation, therefore being in need of incessant re-interpretation:
“The aesthetic regime of the arts is the regime that strictly identifies art in the singular and frees it from any specific rule, from any hierarchy of the arts, subject matter and genre. Yet it does so by destroying the mimetic barrier that distinguished ways of doing and making affiliated with art from other ways of doing and making, a barrier that separated its rules from the order of social occupations. (…) The aesthetic state is a pure instance of suspension, a moment when form is experienced for itself. (…) The aesthetic regime of the arts, it can be said, is the true name for what is designated by the incoherent label “modernity””. (RANCIÈRE, 2004, 23-24).
This conception of a modern, aesthetical regime of distributing the sensible highlights its counter and post-significant aspects, bringing Peircean’s thirdness self-reflexiveness closer to the open features of its transformational component in order to overcome the all-encompassing epistemological fantasy of a final interpretant of all language capable of closing in the universal semiotic process in an ultimate meaningful statement.
By criss-crossing Chino’s three modes of hearing with Ranciére’s three ways of distributing the sensible, we can deepen up and amplify the mapping of the audible field already sketched above through Peirce’s semiotics. Applying recursively Rancière’s three regimes of distributing the sensible to Chion’s three modes of hearing in a conceptual cross-fade operated by Peirce’s three phenomenological categories of experience, we’ve obtained a semiotic diagram of the fissures of the audible field that can be read either vertically, beginning with the three modes of hearing, or horizontally, through its three distinct audible regimes:
| Audible Regimes/Modes of Hearing | Reduced hearing
(1ª) Representamen |
Causal hearing
(2ª) Object |
Semantical hearing
(3ª) Interpretant |
| Ethical Regime of modulation and propagation of sounds (1º) | Sound Objects:
Variations of frequency, phase and amplitude of sound waves. (concrete and electroacoustic Music). |
Voice as Object:
Vocalization, phonation, intonation. (phonoaudiology, psychanalysis). |
Modal music:
Noises, timbres, languages. (Étiquette, Sociology, Anthropology). |
| Poetic-Mimetical Regime of codification and representation of sounds (2º) | Audio Signals:
Recording, editing and mixing sound tracks. (radio, cinema and television). |
Voice as Chant:
Epics, lyrics, dramatics (recitals, poetics). |
Tonal music:
Genre, styles, authors. (rythm, melody, harmony). |
| Aesthetical Regime of dissemination and interpretation of sounds (3º) | Sound Tracks:
Atmospheres, art instalations, sound machines. (sound effects and sound design). |
Voice as Speech:
Diction, prosody and accent. (linguistics, elocutionary). |
Discourse:
Enunciation, persuasion, interpretation. (oratory, rethorics and hermeneutics). |
Table 1: Semiotics of the audible field
D, M&E (Cries and Whispers)
In the present diagram of the audible field and its fissures, sound and language are articulated around voices, music and a variety of different types of noise that may emanate from a certain ambience – real or imaginary – or not. That’s the same threefold conception of the audible field commonly used for any movie sound pre-mixing of its tracks, precisely called by mixing engineers as D, M & E (Dialogue, Music & Effects)[5]. This doesn’t mean that sound for cinema techniques should be necessarily considered as a consolidated new paradigm capable of thinking the audible field in such a satisfactory way that it could eventually replace musical paradigms with all its insufficiencies and anachronisms. It means, rather, that sound movies, as well as the musical avant-garde movements of the 20th century, have not only explored the hyposemical limits of the audible field in its full Peircean sense, but have also presided over an unprecedented, never heard before process of expansion and dilation of the audible field towards unsuspected aesthetical frontiers; and while sound cinema has built itself around the practice and experience of new possibilities of hearing, the avant-garde musical theory provided the knowledge and the concepts, related to this expansion of the audible field, that allows it to be mapped by applying an approach that’s simultaneously semiotic and aesthetic.
The most basic layer of reduced hearing is referred to an ethics of modulation and propagation of sounds that encompasses apparently distinct problems and phenomena – from the laws against excessive noise ambience to problems concerning the common right to free public speech. Reduced hearing is the result of a phenomenological reduction of the audible field to its own variations of sound amplitude, frequency and phase, as perceived by a human ear as volume, pitch and placement (spatial localization) information, in a pre-significant layer of sound objects that, considered as pure quali-signs, were uncovered and charted by last century’s concrete and electroacoustic musical researches.
Audio signals are sound representamens deprived of any codified formal relationship between themselves. As sin-signs, they already lead the audible field towards secondness inasmuch as reproduced sounds are always experienced as imaginary doubles indicating a correlated supposed real source or cause. Their particular potential for noise and nuisance (based on its electrical amplification through public speakers) calls for codified practices of modulation, broadcasting and amplified reproduction, and its possible interpretants are the physical properties of a sound considered in itself: frequency and wavelength (emotive interpretant), volume sensation and relative sound intensity (energetic interpretant) and its own waveforms analysed as such (intellectual interpretant).
The art of shaping and reshaping sounds through analogic and digital audio signals came to be known in cinema circles as Sound Design. Although partial and limited, the possibilities of an aestheticized reduced hearing related to the meaningful semiotic functions of peircean thirdness, are linked to auxiliary signifying roles of high mimetic expressiveness and efficacy played by certain sounds, be them musical or not, in a huge variety of narrative genre and/or staged shows: theatrical plays, movies and animated cartoons tend to offer a wide range of articulated sound effects (from Foley sounds of an almost graphic character to incidental soundtracks, ambiances and sound atmospheres or soundscapes) – that aren’t organized and presented as an autonomous code of representation, but as auxiliary legi-signs that may be replicated to collaborate with signifying processes mainly conveyed by visual and verbal codes.
Nevertheless, for an expanded semiotics of the audible field, the most important feature commonly displayed by sound films is the need to synchronize sounding voices to a moving body or lip’s image. Far from being only a technical trick or effect, lip-sync reveals the central role played by the voice as a privileged object of hearing. Voice is the causal object of human hearing inasmuch as our own auditive physiology demonstrates that our ears were shaped (and sharply tuned in) to listen to the whole frequency range or spectrum of human voices in the most pitiful acoustic or otherwise perceptive conditions. So, inside and outside cinema, voices are the most important object for the human hearing and not only for cognitive reasons related to semantical hearing; and although causal hearing can be referred to an infinite number of possible sound objects, its main goal is to relate to a human voice in at least three different ways: iconic, indexical and symbolic.
Voice as an object is linked to pre-significant, ethical questions related to its iconic qualities. Its immediate objects refer to the flowing vocal variations (modulation, intonation, vocalization) studied by phono audiology; silence as a sound object can be considered one of its most eloquent audible statements. Its dynamic objects belong to the realm of psychoanalysis and are of course in need of incessant interpretation. “Cries and Whispers” (as in the homonymous Ingmar Bergman movie) can be considered as iconical opposite signs that delineate the borders of this particular field of sound objects.
Voice as chant is the most important sound object regarding the relationship between voice as an object and language. Precisely in-between the mimetic-expressive element of language and a fully codified semiotics, either musical and/or verbal, chant is probably the most universally stable sound object of the audible field, for there is no culture in which the vocal function isn’t strongly marked by sound indexes related to its timbre and tuning. In the intersection between causal hearing and the possible ways of poetically articulating sounds and voices into language and meaning, the complex phenomenon of orality arises precisely as an ethno-musicological lyrics whose immediate object is the singing voice as a synaesthetic passage from the audible to the verbal that keeps a strong mimetic and imagistic expressive intensity. Its dynamic objects include the huge variety of lyrical and poetical ways of combining music and speech through verse, and Aristotle’s threefold division of the epic, the lyric and the dramatic forms of mimêsis is the first intellectual attempt to synthesise their possible interpretants into a unified systemical Poetics.
Voice as speech is the central object of causal hearing in its third, symbolic layer. Its immediate object is language as such, considered as an abstract code, and its dynamic objects are the non-discursive components of speech embedded in the sound materiality of voices. A less semantic and more aesthetic mode of hearing a speaking voice does not pay attention to the meaning of its discourse and its multiple possible interpretations, but rather focuses on its elocutionary, trying to situate, in a socio-cultural level, a given speech through its symbolic, linguistically organized, characteristics: its diction, prosody and accent.
Singing and speaking voices are sound objects that reach the limits of the audible field to intermingle with the visual and verbal fields through language. Metaphors understood as poetic images, for example, are at the core of this process, since they aren’t generally pure visual or verbal forms of expression, spreading out mainly through oral (and audible) social resonance. But the audible field is also capable of generating autonomous codes of expression, called in our culture “music”. The concept of semantical hearing may thus cope with a semiology of music as well as with a semiology of the verbal languages (or linguistics). Understood as “a pure combination of sounds”, without any other reference to visual and/or verbal signs, music is an artistic discipline traditionally linked to the influx of a well-established poetic-mimetic regime and under the influence of a highly semantical hearing. The history of music shows how difficult was the process of establishing an autonomous poetics of the audible field, for most of the so-called modal music is still comprised by a specific cultural ethics that provides the audible field its proper forms of expression, generally related to verbally transmitted myths and/or to gesturally (and visually) performed rites. The iconical firstness that primarily informs this kind of music is of course what gives it its ethnic folkloric flavour, while its culturally regulated ethics is what tends to maintain the audible field’s strong mimetic sensory qualities in a state of permanent contention. Musical ideas, at this point, are only Peircean rhemas at the first cognitive stage of semantical hearing, still subordinated to a given set of verbal and visual expressions and having its interpretants outside the range of the audible field: a certain culture’s étiquette and politeness’ rules may configure a first set of emotive interpretants of a given piece of music: its appropriateness for different social occasions, the type of emotions that cultural convention wants it to supposedly convey, the greater or lesser social prestige of a particular style of music in a specific society, etc. A second set of energetic interpretants would configure a whole sociology of a specific genre or kind of music, describing as completely as possible its socio-cultural context, depicting its historical development and stating its proper place in the history of music. Final interpretants would belong to a general anthropology of music whose main three goals would be to provide a full overview of the history of all types of music, to formulate a general theory of sounds and music (as in Pierre Schaeffer’s Traité des Objets Musicaux) and to provide a closure to the audible field by eventually stating its final interpretants.
The obvious impossibility of a such a resolution of the audible field demonstrates, a contrario, the structural necessity of the fissures and anachronisms between the history of music and the history of literature and of the plastic arts: last creation of the classical 17th century poetic-mimetic régime, tonal music is in the very edge of the aesthetical regime and it is adequately situated, in our diagram, in this inter-section. As an autonomous, self-referential system, it offers a representational image of a possible logical interpretant of the audible field and, as such, allows for a complete development of pure musical ideas as sound organized propositions or dici-signs. There’s no doubt that, in pure musical terms, thirdness can be thought of as the triadic relationship between rythm (1º), melody (2ª) and harmony (3ª), and these elements act recursively as music’s own internal interpretants, unfolded into its emotive (rhythms), energetic (melodies) and intellectual ones (harmonies). Nevertheless, tonal music doesn’t have to be considered, in this diagram, as the final and exclusive interpretant of Peirce’s thirdness in the audible field.
Last but not least, semantical hearing’s most celebrated cognitive achievement is our ability to follow oral discourse and listen to a given set of arguments. This is the most abstract and verbal spellbound layer of the audible field: oratory, rhetorics and hermeneutics are related to its emotive, energetical and intellectual interpretants. Although highly regulated and controlled in its synaesthetic expressive capacity, the mimetic element of language often permeates verbal discourse through numberless correspondences – sensuous and non-sensuous – unforeseen by the code or language in use: homophonies, puns, lapses, cacophonous sounds and undesirable rhymes unveil to us on a daily basis how language, amongst cries, whispers and arguments, resonates and amplifies the ever present fissure that is carved between sounds, signs and hearing.
So, what does the diagram of the audible field may tell us about the general relationship between language, sounds and images, as well as about the tension between the mimetic and the semiotic elements in every signifying process? While the first question raises a set of problems related to aesthetics understood as a theory of plastic arts and literary genre, the second one is primarily related to issues regarding the possibility of an epistemological paradigm capable of providing a conceptual synthesis between the theory of language and the theory of knowledge. In the short scope of this article, we’ll be able only to hint at some preliminary hypotheses concerning these two questions.
For the aesthetical research in general, the main improvement brought by a semiotics of the audible field is the conceptual dislocation of the traditional duality between the verbal and the visual fields of expression and it’s no less traditional description as a pair of complementary opposite (convergent or divergent) poles. Unveiling sounds as the excluded middle of words and images – a theoretical possibility unforeseen by Rancière’s aesthetics – allows the crossfading and criss-crossing of all kinds of poetical genre codifications and stylistic models based on a supposed matricidal primacy of the verbal codes over the visual ones (or vice-versa) and points towards a better understanding of the synaesthetic processes at work in the phaenomenon of language from its very start, thus establishing the conceptual foundations for an all-encompassing aesthetical paradigm based on the primacy of mimetic material hybridizations over its subsequent decanting and distillation into poetically codified particular forms of art and expression.
This hypothesis of a primacy of the mimetic element of language over its properly semiotic element can also lead to a complete reshaping of our epistemological prejudices about the relationship between language and thought. This becomes possible because, through a semiotic mapping of the audible field, it gets easier to demonstrate how verbal and musical third-order symbolical codes may arise from the more fluid and sensuous second and first-order signifying articulations already at work when voices and sound objects happen to be heard. The great scientific challenge of finding a conceptual ground to deal with the diachronic problem of the origins and evolution of language as well as with the systemic analysis of its synchronic structures, therefore, can be solved only outside the realm of verbal grammar and structural or transformational linguistics, for its secret lies in kinaesthetic and synaesthetic layers of intra-semiotic hybridizations that can be historically traced and semiotically described only through a speculative grammar (as in Peirce’s semiotics) capable of mapping all possible meaningful articulations between sounds, signs and images.
Bibliographical References
Benjamin, W. (2005) “Doctrine of the Similar”, in Selected Writings, Vol.2 (Part 2) 1931-1934 (M. W. Jennings, editor), Belknap Press, Harvard, pp. 694-698.
Benjamin, W. (2005) “On the |Mimetic Faculty”, in Selected Writings, Vol.2 (Part 2) 1931-1934 (M. W. Jennings, editor), Belknap Press, Harvard, pp. 720-722.
Carpeaux, O. M. (1967) Uma nova história da música, Livraria José Olympio Editora, Rio de Janeiro.
Chion, M. (2005) L’Audio-Vision – son et image au cinema, Armand Collin, Paris.
Forlenza, j. e stone, T. (1993): Sound for Picture – an inside look at audio production for film and television, Mixbooks, Winona.
Heller-roazen, D. (2010) Ecolalias – sobre o esquecimento das línguas, Ed. Unicamp, Campinas.
Peirce, Ch. S. (1975) Semiótica e filosofia – textos escolhidos, Ed. Cultrix, São Paulo.
Pinto, J. (1995) 1,2,3 da Semiótica, Ed. UFMG, Belo Horizonte.
Rancière, J. (2004) The Politics of Aesthetics – The Distribution of the Sensible, Continuum International Publishing Group, London.
Santaella, L. (2005) Matrizes da Linguagem e Pensamento – sonora, visual, verbal, Ed. Iluminuras, São Paulo.
Schaeffer, P. (1966) Traité des objets musicaux, Éd. du Seuil, Paris.
[1] See also HELLER-ROAZEN, Daniel: Ecolalias – sobre o esquecimento das línguas, Ed. Unicamp, Campinas, 2010
[2] The translation from the portuguese original version is mine.
[3] See CARPEAUX, Otto Maria: Uma nova história da música, Livraria José Olympio Editora, Rio de Janeiro, 1967. p. 85.
[4] See SANTAELLA, Lucia: Matrizes da Linguagem e Pensamento – Sonora, Visual, Verbal, Ed. Iluminuras, São Paulo, 2005, pp. 97-184.
[5] See FORLENZA, Jeff e STONE, Terri (Orgs.): Sound for Picture – an inside look at audio production for film and television, Mixbooks, Winona, 1993.