Preliminary investigation into the construction of multimodal grammar
DOI:
https://doi.org/10.1558/lhs.v10i1.27273Keywords:
dimension, level, multimodal grammar, semiotic system, typeAbstract
The present article is intended to investigate the issues in the construction of multimodal grammars according to level, type and dimension of semiotic systems. It is found that, in terms of level, only three-level semiotic systems are appropriate for the construction of a multimodal grammar; in terms of type of semiotic systems, generally speaking, it is not necessary to construct a grammar for the semiotic systems of the iconic type and those of the indexical type; in terms of dimension of semiotic systems, those of different dimensions exhibit different grammatical patterns and systems, so it is necessary to construct a separate grammar for each semiotic system. The compositional devices: framing, information value and salience can be used to determine the units of grammar in a semiotic system. It is also found that the discourse from a two-level semiotic system can be a one item text, or a text of one recursive item, or of a series of items in a fixed pattern.
References
Barthes, R. (1977) Image, Music, Text. London: Fontana.
De Saussure, F. (1915/1972) Course in General Linguistics. London: McGraw-Hill Book Company.
Halliday, M. A. K. (1978) Language as a Social Semiotic: The Social Interpretation of Language and Meaning. London: Edward Arnold.
Halliday, M. A. K. and Hasan, R. (1985/1989) Language, Context and Text: Aspects of Language in a Social Semiotic Perspective. Oxford: Oxford University Press.
Hu, Z. (2007) Multimodalization in social semiotics. Language Teaching and Research 22(1): 1–10.
Jewitt, C. (ed.) (2009) The Routledge Handbook of Multimodal Analysis. London and New York: Routledge.
Kress, G. and van Leeuwen, T. (1996/2006) Reading Images: the Grammar of Visual Design. London: Routledge.
Lemke, J. L. (2005) Important theories for research topics on the website. J. L. Lemke Online Office.
LeVine, P. and Scollon, R. (eds) (2004) Discourse and Technology: Multimodal Discourse Analysis. Washington DC: Georgetown University Press.
Machin, D. (2007) Introduction to Multimodal Analysis. London: Hodder Arnold.
O’Toole, M. (1994/2010) The Language Of Displayed Art. London: Leicester University Press.
Pharies, D. A. (1985) Charles S. Peirce and the Linguistic Sign. Amsterdam: John Benjamins. http://dx.doi.org/10.1075/fos.9
Van Leeuwen, T. (2005) Introducing Social Semiotics. London and New York: Routledge.
