On the ontological properties of audiovisual texts

Abstract

The article focuses on epistemological issues of studying audiovisual texts. Audiovisual text is considered as a type of semiotically heterogeneous messages intended for visual and auditory perception. Moreover, audiovisual texts containing a verbal component, which can be presented both in aural (auditory) and graphic (visual) forms, can serve as a subject of linguistic inquiry. The study reveals a high degree of variability in the terminological nomination of the investigated object, which is common in Russian studies of texts of a semiotically syncretic nature. Starting from the analysis of the origin and denotative load of terms proposed at different periods in Russian and international literature to describe semiotically heterogeneous artifacts, as well as observations of the structure and functioning of audiovisual content of various types, formats and genres, some characteristics that can be considered as ontological properties of audiovisual texts were identified. Such properties might include polymodality, multichannel and polycode nature and, in some cases, multimedia as well. An informational and semiotic matrix of the archetypal audiovisual text is suggested, based on the concepts of sensory (perceptual) modalities, information conveying channels and the code nature of components interplaying in the meaning-making within audiovisual works.

References

1. Vashunina I.V., Matveev M.O., Nistratov A.A., Tarasov E.F. 2020. Kreolizovannyj tekst: smyslovoe vosprijatie: kollektivnaja monografija [Creolised text: perception of sense]. Moscow. (In Russ.)

2. Voroshilova M.B. 2013. Politicheskij kreolizovannyj tekst: kljuchi k prochteniju [Political creolised text: cues for reading]. Ekaterinburg. (In Russ.)

3. Ejger G.V., Juht V.L. 1974. K postroeniju tipologii tekstov [Towards typology of texts]. Lingvistika teksta: materialy nauch. konf. pri MGPIIJa im. M. Toreza [Text linguistics: proceedings of conference held at Moscow State Institute of Foreign Languages]. Vol. 1, pp. 103–109. (In Russ.)

4. Kalinin A.Y. 2020. Pragmaticheski determinirovannye priemy audiovizual’nogo mediaperevoda [Pragmatics-driven techniques of audiovisual media translation]. Moscow State University Bulletin. Series 19. Linguistics and Intercultural Communication, no. 2, pp. 45–57. (In Russ.)

5. Kosenko V.S. 2021. Smyslovoe vosprijatie kreolizovannyh tekstov [Perception of sense of creolised texts]: Avtoref. diss. ... kand. filol. Nauk. Moscow. (In Russ.)

6. Nekrasova E.D. 2014. K voprosu o vosprijatii polimodal’nyh tekstov [On multimodal perception of the text: a psycholinguistic experiment]. Tomsk State University Bulletin, no. 378, pp. 45–48. (In Russ.)

7. Polubichenko L.V. 2021. Mezhsemioticheskij perevod v semioticheski geterogennyh tekstah [Intersemiotic translation in semiotically heterogeneous texts]. Moscow State University Bulletin. Series 19. Linguistics and Intercultural Communication, no. 4, pp. 130–145. (In Russ.)

8. Polubichenko L.V., Donskaja M.M. 2007. Semiotika verbal’nogo i neverbal’nogo v mul’timedijnom reklamnom diskurse [The interplay of verbal and non-verbal semiotics in multimedia commercials]. Moscow State University Bulletin. Series 19. Linguistics and Intercultural Communication, no. 2, pp. 36–52. (In Russ.)

9. Rzheshevskaja A.A. 2019. O polimodal’nosti i perspektive v diskurse [On polymodality and persperctive in discourse]. Moscow State Linguistic University Bulletin. Humanities, no. 7 (823), pp. 43–51. (In Russ.)

10. Sonin A.G. 2006. Modelirovanie mehanizmov ponimanija polikodovyh tekstov [Modelling the mechanisms of polycode texts understanding]: Avtoref. diss. ... d-ra. filol. Nauk. Moscow. (In Russ.)

11. Sonin A.G. 2003. Obshhepsihologicheskie i kognitivnye mehanizmy ponimanija mul’timedijnyh tekstov [General psychological and cognitive mechanisms of multimedia texts understanding]. Voprosy psiholingvistiki [Issues in psycholinguistics], no.1, pp. 43–56. (In Russ.)

12. Sorokin Ju.A., Tarasov E.F. 1990. Kreolizovannye teksty i ih kommunikativnaja funkcija [Creolised text and their communicative function]. Optimizacija rechevogo vozdejstvija [Optimising the speech effect]. Moscow, pp. 180‒186. (In Russ.)

13. Stepanov Ju.S. 1971. Semiotika [Semiotics]. Moscow. (In Russ.)

14. Beaugrande R. de, Dressler W. U. 1992. Introduction to Text Linguistics. London, New York: Longman.

15. Cashmore E., Cleland J., Dixon K. 2018. Screenagers. Screen Society. Palgrave Macmillan, pp. 37–50.

16. Gambier Y. 2013. The Position of Audiovisual Translation Studies. The Routledge Handbook of Translation Studies, ed. by C. Millan and F. Bartrina. London, Routledge, pp. 45–59. 

17. Gottlieb H. 2018. Semiotics and Translation. In The Routledge Handbook of Translation Studies and Linguistics. Abingdon, Routledge, pp. 45–63.

18. Jakobson R. [1959/2004]. On linguistic aspects of translation. The Translation Studies Reader. New York and London: Routledge, pp. 138–143.

19. Jakobson R. [1963/2003]. Essais de linguistique generale. Les fondations du langage. Paris, Minuit.

20. Jakobson R. 1971. Language in Relation to Other Communication Systems. Selected Writings, vol. 2, Mouton, The Hague, pp. 697–708.

21. Kress G. 2011. What is mode? The Routledge handbook of multimodal analysis (2nd ed.). London, Routledge, pp. 54–67.

22. Li Z. N., Drew M. S., Liu J. 2004. Fundamentals of multimedia. Upper Saddle River (NJ): Pearson Prentice Hall.

23. Martinelli D. 2020. What you see is what you hear: Creativity and communication in audiovisual texts. Springer International Publishing.

24. Mulder J. W. F. 1994. Written and spoken languages as separate semiotic systems. Semiotica, vol.101(1/2), pp. 41–72.

25. Ventrella F. 2015. Visual Turn. International Encyclopedia of the Social & Behavioral Sciences Reference Work. 2nd Editio, pp. 207–213.

26. Zabalbeascoa P. 2008. The Nature of the Audiovisual Text and its Parameters. The Didactics of Audiovisual Translation. Amsterdam and Philadelphia John Benjamins, pp. 21–39.