Social Semiotics and Visual Grammar : A Contemporary Approach to Visual Text Research

This article outlines a visual text reading based on social semiotics approach, that is the visual grammar. It is an account of the explicit and implicit knowledge and practices around a resource, consisting of the elements and rules underlying a culture-specific form of visual communication. It required a general comprehension of social semiotics to obtain a deeply understanding to visual grammar. The differences between social semiotics and general semiotics expressed to accomplish its purpose. The concern of visual grammar is the relationship between ‘drawn participants’ and social life. Visual grammar can be positioned as a contemporary approach to visual texts meaning-making. Study of visual arts seems to entail visual grammar to play a more important role amid the phenomenon of visual culture that increasingly characterizes the lives of today’s society.


INTRODUCTION
Contribution of this paper to art studies is expected to widen horizon of 'meaning production', mainly in social semiotics perspective.Specifically it is expected to be useful in analysing visual texts in today's contemporary life.This paper aims to indicate the social semiotics as one point of view of art research, which has different meaning from general semiotics, and to present the structure of visual reading as one of the social semiotic analysis models.Therefore, Social semiotics of Hodge and Kress (1988) and Reading images, the grammar of visual design of Kress and van Leeuwen (2006) are exposed as the main references.
Issues in this paper are the role of visual texts in social semiotics context and ways to analyse it using visual grammar model.Exposure begins with historical background and further presents the definition of social semiotics and its differences to general semiotics.It also examines the key elements and principles of analysis.Visual grammar is positioned as a model of social semiotic analysis.

Social Semiotics
The term of semiotics is derived from a Latin word semio, absorption of the Greek semeion, which means a sign or signal.This term refers to an ancient use in medicines; a diagnosis act named semeiosis process.Late in 17th century John Locke proposed the term of semiotics into the branch of philosophy which means the doctrine of signs.At the end of the 19th century until early in 20th century was the beginning period of modern semiotics marked by the birth of two main ideas: Semiotics by Charles Sanders Peirce  in America, and semiology by Ferdinand de Saussure  in France.Pierce's idea of semiotics based on philosophical fields and forms a triadic structure, such as all nature phenomena (representamen) that known as a sign (object) which determines a cognitive effect upon a person (interpretant).Saussure's idea of semiology is based on linguistics fields and builds a diadic structure.These are the general systems or rules of the signs to express ideas (langue) and to show how people use the system of the signs in particular group (parole).In the following periods, this Saussurean concept in linguistics domain is developed into non-linguistics communication domain.
At least, three schools of semiotics have applied ideas from the domain of linguistics to non-linguistic modes of communication.The first was Prague School in the 1930s and early 1940s, which developes the work of Russian Formalists by providing it with a linguistic basis.Notions such as 'foregrounding' were applied to language as well as to the study of art (Mukarovsky, Honzl, Jakobson, and Bogatyrev).The second was Paris School in the 1960s and 1970s, which applies ideas from de Saussure and other linguists to painting, photography, fashion, cinema, and music (Schefer, Barthes, Lindekens, Metz, Nattiez).The third was late in 1980s that marked by developing Halliday's systemic-functional linguistics, by a number of scholars in Australia (Threadgold, Thibault, O'Toole, Kress, van Leeuwen).
In Semiotics Encyclopedia, social semiotics refers to two different entities but related to each other: firstly, 'social semiotics' without capital letters is broadly oriented semiotics.It is heterogeneous and encompasses many disciplines related to the social dimension of meanings, both in communication media, production, interpretation and circulation, as well as in their implications in the social process as cause and effect; Secondly, 'Social Semiotics' with capital letters is the school of linguistics and semiotics which specifically addresses the social aspects (Hodge).Halliday (1978) has introduced the term of social semiotics (as cited in Hjorland, Birger and Nicolaisen, 2007, p. 1) as follows: "By their everyday acts of meaning, people act out the social structure, affirming their own statuses and roles, and establishing and transmitting the shared systems of value and of knowledge.[...] We have to proceed from the outside inwards, interpreting language by reference to its place in the social process.This is not the same thing as taking an isolated sentence and planting it out in some hothouse that we call a social context.It involves the difficult task of focusing attention simultaneously on the actual and the potential meanings, interpreting both discourse and the linguistic system that lies behind it in terms of the infinitely complex network of meaning potential that is what we call the culture."(Hjorland, Birger and Nicolaisen, 2007) Ten years later Hodge and Kress systemized this Halliday's term into Social Semiotics.The ideas was based on Marx that human real consciousness could never be anything other than ideology.They interpreted Marx's point as the involvement of each individual in the exchange of ideas with one another in the real world as the basis and the source of consciousness into a unified process of semiotics with agents, objects and power that come from the material and social world, and it can only be understood from its own world.Hodge and Kress have given the definition of social semiotics as follows: Social semiotics is primarily concerned with human semiosis as an inherently social phenomenon in its sources, functions, contexts and effects.It is also concerned with the social forms, through semiotic texts and semiotic practices, in all kinds of human society at all periods of human history (p.261) (Hodge, Robert, and Kress, 1988: 261).
The term of semiotic text in this sense contains a broader meaning, that all kinds of signs are the vehicle of social activities.Manning shows comprehensively that social semiotics considers as a social life, a structure of the group, belief/religion, and cultural practices.The meaning of social relations is analogous to the structure of the language, hence all human communication actions could be seen as a sign or text, that should be read first in order to understand the meaning (Manning and Cullum-Swan, 2009).The firm and straight statement expressed by Gottdiener and Lagopoulos (1986) that social semiotics is a materialistic analysis of the ideology in everyday life (Randviir & Cobley, 2010).

The Distinguishing Aspects to General Semiotics
Some definitions above clearly express a complexity of the social semiotic study that covers all types of human communication actions in certain communities.This is how it differs from general semiotics.The cause of this differences, in Thibauld's words, there is a spirit of social semiotics to intervene in general semiotics, both in theoretical and practical aspects of meaning (Thibault, 1991: 6).Some of the distinguishing aspects include: mindset, focus of study, the perspective on signs, and the scope of analysis.

Mindset
Vannini (2007) observes the mindset of general semiotics emphasizes to the important role of the structure.General semiotics believe the interrelation of semiotic system depends on codes or rules which order a convention of meanings in a family system, etiquette, mathematics, or arts.He also reveals a loyalty of general semiotics to the structures, not in understanding the way people produce, consume, and negotiate the rule of semiotics (Vaninni, 2007).Therefore a mindset of general semiotics accentuates the structures, then it is called structural semiotics.Social semiotics, on the contrary,, is refusing all preferences of the structures, and positioning the semiotic as a site of social statuses.

Focus of Study
According to Van Leeuwen (2005), general semiotics focus on the signs, whereas the focus of social semiotics changes to the way people use semiotic resources, either to produce objects and communicative events, or to interpret it, which is also as a form of semiotic production in particular situation and social practices (Van Leeuwen, 2005).This statement is in line with Thibauld, that social semiotics focus on the material aspects of the dialectical linkages of the relationships pattern in textual meaning, as well as its use in certain domains of social practices (Thibault, 1991: 7).
As for the specifications or specific domains of social practice, Hodge and Kress have confirmed that semiotics in social practices not only connect the power to the meaning, but also refers to the meaning of the power (Hodge and Kress, 1988: 2).Within a place as like a market where the transaction happens, there are power stratification and dynamics interplay between the producer, distributor or dealer, and consumer of ideology, including its practical uses.Powerful producers are able to create meanings and the rules, and conditioning in order to be absorbed by the consumers.In this mechanism, social semiotics focuses on study signs to the formation process of producer regime, competition for power, use practices, and their effects on consumer's regime and social life in general.This explanation is in line with Manning (1977Manning ( /2009) that is the power and authorities do stabilize the expression and enforce concreteness a meaning of sign in the culture (Manning and Cullum-Swan, 2009: 618).So, in the social semiotics frame, material reality of the sign is always analysed its signified, concept or connotations based on ideological systems in daily practices of a particular group of people, while the general semiotics analyse on the signs itself.

The Perspective of the Signs
Sign is seen by general semiotics as a structured entity that each part and its interrelations must be scrutinized in order to find its meaning.Social semiotics views the signs in a different perspective, which according to van Leeuwen (2005), as a kind of code or a vehicle, whose presence depends on the specific resources (Van Leeuwen, 2005: 3).He based his view on Halliday (1978) that the grammar in the language system is not a code, not a set of rules to construct sentences correctly, but rather a "resource for making meaning".
Van Leeuwen defines the semiotics resources as all acts and artifacts used to communicate, both physiologically produced -with vocals and muscles instruments to make a facial expression-or technological way -with pen, ink and paper, computer software, clothes, scissors and others.Traditionally all these are so-called 'signs'.These are objects and actions that can be observed and that have been drawn into the domain of social communication.So the perspective in social semiotics is not on the sign as a semiotic object, but on various objects and events that become semiotic resources.

The Scope of Analysis
The development of social semiotics is not merely an extension of the scope of structural semiotics, but as stated Thibauld, is a social intervention to semiotics that considered too static and narrow angles.Furthermore, Hodge and Kress even place a Saussure's scope of analysis as antiguide, which constructs the scope of social semiotic analysis by against the scope of structural semiotics as Saussure suggested (Hodge and Kress, 1988: 18).So, social semiotics negates structural semiotics scope by turning its scope to be a limit to social semiotics as shown in the following table.
In relation with these differences, contend Vannini, social semiotics study have to recognize the dynamics of power, ranging from research design through publication and its reception, as well as utilize the research results.For this reason, social semiotics analyses have to ensure reflexivity that informs various aspects of the dynamics of power associated with the semiotic process.With this reflexivity it does not just mean the ideological dialectic can be read explicitly, but also the specific nature and polyvocal process (inclusion of many voices and points of view as well as excavation and retrieval of oppressed knowledge as a form of resistance against the dominant discourse and knowledge, Bakhtin, 1981) can be revealed as the uniqueness and validity of social semiotics (Vaninni, 2007).

Key Elements
The subject of semiotics, both general and social is the message.It is realized as a sign or text, that is a structure or message trace is perceived, which is ascribed to a particular social group.Text is a material realization of the message, as a product of the semiotic system and as a historical document of the constitution.While the general semiotic view text as a fixed entity, social semiotics highlight on dialectical struggle between text and its semiotic systems, which always presents a specific semiotic action, namely discourse.Departing from the discourse, it can be traced other key elements of social semiotics in details: genre and style.

Discourse
The term discourse derives from the French discours; it means speech.At first, this term is used in the linguistics field.But since it evolved into various branches of the social sciences and humanities, then there is a diversity of meanings within the scope of their respective disciplines.The use of this term in social semiotics is based on Foucault (1976, p. 83): In the most general and vaguest way, discourse denoted a group of verbal performances; and by discourse, then, I meant that which was produced (perhaps all that was produced) by the groups of signs.But I also meant a group of acts of formulation, a series of sentences or propositions.Lastly -and it is this meaning that was finally used (together with the first, which served in a provisional capacity)-discourse is constituted by a group of sequences of signs, in so far as they are statements, that is, in so far as they can be assigned particular modalities of existence (Foucault, 1976: 83).Van Leeuwen defines discourse as a social construction of knowledge of some aspects of reality (Van Leeuwen, 2005: 93).By means that the discourse is developed in a particular social context, and in a consistent way with the interests of the executants; either in a large context such as multinational corporations, or small context such as certain family; also in the context of formal institutions such as the press and schools, as well as informal context like a conversation in a restaurant.

Genre
The term genre is generally interpreted as a text in any code that has a particularity due to follow certain rules to produce a distinctive type.This term can be equated with the type, stream, or a school of thought.Text could be typical when it has characteristics that can also be recognized in other similar texts.The reason is, people produce a certain text because of its willingness to follows the rules; it is a kind of recipes, traditions, customs, or a model that has been adhered to generations.Since each rule has implications for distinction with other rules, then any text that is made under a rule would reflect the genre espoused.Genre can help a person to build or strengthen the meaning of the text, to construct knowledge, identify the agents of the text and set the audience characteristics.
Van Leeuwen explains there are three aspects of the construct of a genre, namely: characteristics of content, characteristics of the form and characteristics of the function.The content refers to the similarities characteristics of subject matter in the discourse, rather than a discussion of the matter itself, for a discussion of a matter is the domain of discourse.The form is seen from the similarities of the structure, and sequence or relation between elements contained in the content.And the function reveals from the sameness of usability in related discourse.Genre in social semiotics among other things: sociolinguistics, pragmatism, ideology, feminism, and gender.
Based on these characteristics seem that there is a variety of genres in particular discourse.Every genre is the material manifestation of the message, and can be identified by the characteristic equation, both the content, form, and function.The role of genre then is as semiotic resources that have been classified, or as a template of communicative activity.

Style
Merriam-Webster Dictionary defines style as a particular manner or technique by which something is done, created, or performed.Van Leeuwen categorizes the style to three kinds: individual style, social style, and lifestyle.He explains that the idea of individual style foregrounds individual differences.There are spaces for individual particularities although the way everybody acts usually regulated by social standard.And the idea of social style express is not only someone personality and attitudes, but also their social status, 'who we are', in a stable category such as economic class, age, gender, social relationships, and 'what we do' in terms of someone's involvement in socially regulated activities and the role he/she played within them.
Lifestyle in Merriam-Webster Dictionary is described as a particular way of living: the way a person lives or a group of people live; this description combines both individual and personal style.According to van Leeuwen, lifestyle is more socially than individually orientation.He argues that the existence of social or communal style spread across the cities of the world is characterized not by the established individual position, but by shared consumers behaviour.Besides, lifestyle is also social as marked by many appearances, which is almost every 'distinctive way of life' that can be expressed in fashion and accessory styles, culinary tastes, type of communication tools, type of vehicles, sort of sport activities, and so on.Lifestyle is another social because of the role it plays in the marketing sector, when traditionally social indicators such as class, gender, age, education have been replaced by the 'lifestyle market segmentation' technique which classify consumers through a combination of consumption patterns and attitudes.

Analyzing Principles
The principles of analysis in social semiotic need to be placed in the research paradigm.Paradigm in research is defined as the basic belief system or worldview that guides researchers in selecting and determining the fundamental ways, ontological and epistemological.Worldview that has received by the researchers so far consists of four paradigms: positivism, post positivism, critical theory, and constructivism.With regard to critical theory, this refers to some alternative paradigm, in which among others include: neo-Marxism, feminism, and materialism.This paradigm is divided into three trends: postmodernism, post structuralism, and a mixture of both.The study of social semiotics which has several characteristics, among others: based on Marxian view, the denial of the structure, and his interested in the discourse of the oppressed, show a strong tendency to be in the post structural critical paradigm.
Referring to Guba andYvonna (1997/2006), ontologically, critical theory's view to the reality is liquid, and shaped time after time by series of social, political, cultural, economic, gender, ethnic, factors that then crystallize into a 'real' structure.Meanwhile, epistemologically, researcher and subject of research are interactively related with researcher values and other people who latch to affect research.Therefore, the results of the study are mediated by an attitude that does not get the differences between ontology and epistemology, because these both are interwoven with the interaction between the researcher with a particular object or group.Methodologically, this paradigm characterized by dialogue of researcher and research subject, which should be a dialectical dialogue, in order to change the ignorance and misunderstanding into a discourse based on a deeper awareness (Guba, Egon and Lincoln, 2009).In the critical paradigm, the role of researcher's view seems clearly important.Researcher does not isolate an object of study from any other influences such as positivism thought.There are very wide space for researcher to interact with various components that could be related to object of study to generate values, discourse, consciousness, and a new reality.Match to such analysis principle, there is a visual text analysis model called visual grammar.Every semiotic fulfills both 'ideational' function, a function of representing 'the world around and inside us' and 'interpersonal' function, a function of enacting social interactions as social relations.All message entities -texts-also attempt to present a coherent 'world of the text', what Halliday calls the 'textual' function -a world in which all the elements of the text cohere internally, and which itself coheres with its relevant environment (p.15) (Kress & Van Leeuwen, 2006: 157).

Visual Grammar
In visual grammar, structure of visual language is equated with linguistics almost similarly.Grammar in linguistics describes the combining of words into clauses, sentences and texts.Likewise, visual grammar explains the procedure to combine visual elements -people, places, and objects-in visual statement in a certain complexity and extension.Just as the structure of linguistic, visual structure refers to the specific interpretation of the experience of form and social interaction.Furthermore Kress and Leeuwen stated that: 'What is our visual grammar a grammar of?' First of all we would say that it describes a social resource of a particular group, its explicit and implicit knowledge about this resource, and its uses in the practices of that group.Then, second, we would say that it is a quite general grammar, because we need a term that can encompass oil painting as well as magazine layout, the comic strip as well as the scientific diagram.[...], an account of the explicit and implicit knowledge and practices around a resource, consisting of the elements and rules underlying a culture-specific form of visual communication (p. 3) (Kress & Van leeuwen, 2006: 3).
Kress and Leeuwen stated that approach of visual grammar starts from the social aspect.It structure is not a simple reproduction of the structure of reality, but instead, producing a reality picture bonded to the interests of the social community that produces an image, circulate, and read it, so it becomes ideological.Meaning expressed by printmakers, photo-graphers, designers, painters and sculptors in first and foremost are the social meanings.To explore these meanings, there are aspects and rules of visual grammar that have to be analysed: elements and structure, position of the participants, modality, composition, and materiality.

Elements and Structure
Knowledge of visual grammar consists of the elements, structures and rules that underlie specific cultural forms of visual communication.Visual elements are called the participants.Every visual text includes two main participants: drawn-participants (all humans, events, objects, and other forms appear in the image), and interactive participants (image-makers and image-viewers).Image-maker is a person or group who creates the image; and image-viewer is a person or community who uses the image.Each participant, whether potential or actual become semiotic resource are related to each other, either realistic or virtual, and formulated on its social meaning.All participants are connected or not connected to form a variety of relationships, structures, and interaction processes in two possible forms of representation structures: narrative and conceptual.
Narrative Representation Structure (NRS) presents the ongoing actions and events.Similar to the structure of a sentence that contains the elements of subject-verb-object, a subject in NRS called actor, and objects called goal.In visual text, actor can be a living creature, object, or other objects that are characterized prominent than the other, either by position, size, color sharpness and contour sharpness.A verb is synonym to the sort of action, or the events experienced, or relations between actors to goal, and called transaction.
The conceptual representation structure is a relationship between participants in a particular class, statically, and timeless.This structure can be understood as a process of classification or taxonomy, such as the sequence or chain related to participants to interact; at least one set of participants will act as a subordinate associated with at least one other participant as a superordinate.

Position of Participants
The position of the participants in visual grammar is a combination of relationships among participants in the picture, also with interactive participants.Interactive participants are real people who produce and understand the images in social community context that regulates what the picture may 'say', how it should be said, and how it should be interpreted, to some degrees and in different ways.Some terms are demand-offer relation, social distance, horizontal line, vertical angle.

Modality
The term modality comes from linguistic field, referring to a value of the truth or credibility of statements attributed to the facts.Modality in verbal language expressed in words such as: believe me, most likely, perhaps, actually.Linguistic resources of modality have a very important role in society.This source allows people to make the necessary truth together, to be able to form groups that believe the same things, so it can act together in one world integrally and effectively.In the same way also allows one to deny the truth of other people, with all the potential consequences owned, from exile someone from the group to the wars of religion and ideology.However, something that considered being true in visual grammar is not associated with the absolute truth.Truth in modality is a limited sense in answer to the question 'how true something is revealed?'Visual modality can depict people, places, and other things that look real, like really exist, or as if the imagination, fantasy, caricature.Rating of modality is a social, depending on what is considered to be real (or true, sacred) in social groups particularly targeted by the representation.High modality means closer to reality, and as opposed to leave away.This can be set from the depths of the parts, contrast, color saturation and intensity, as well as sharpening-blurring background.

Composition
The composition is the arrangement of all 'drawn participants' elements as a whole.The meaning of composition is interpreted through three interrelated systems: information value, salience, and framing.System of value information is a placement of elements in image zones that reflect the values of specific information.Division of left and right side, based on the habit of writing, means that the left side is an area of written information, already happened, oriented to the past, certain things (called given), and the right area is a place to be inscribed, expected to occur, oriented to the future, uncertain things (called new).Based on philosophical relations, the topside reflects ideal things, and the bottom side is close to reality.In division of center-edge, related to the social aspect, the center area reflects a source or mediator of the edges or marginal (called margin).
System of salience is the arrangement of elements in the 'drawn participants', made to guide an attention of viewers in different degrees, such as: placement in the foreground or background, relative sizes, contrast tonal value (or color), the difference in sharpness.Salience determines a hierarchy among participants on the overall picture.By considering location of the most prominent participants, it can be set such 'given' is more important than 'new', or vice versa.So as 'ideal' is more prominent than 'real' and vice versa, and also to the center-margin relations.
System of framing associated with the presence or absence of a boundary line.It is realized by the elements that create a dividing line, real or imaginary, or by the actual frame line, that disconnects or connects the elements of the picture.Framing in image indicates contextually co or private ownership.System of framing detect element or group of elements that are sometimes disconnected, decompose each other, or one element connected to some other elements, and all elements joined together.Problem in system of framing is a matter of degree: elements in the composition may be strongly or weakly framed; the stronger the framing of an element, the more it is presented as a separate unit of information.The more the elements of the spatial composition are connected, the more they are presented as belonging together, as a single unit of information.

Materiality
Various terms related to the material aspects in visual grammar is called production.Aspects of materiality, though is given things, but it is an important part to produce a meaning.Gold or bronze, original oil painting or reproduction, and any material brought meaning, include social significance.Each type of medium states a selection of ideational, interpersonal and textual function in the significance system.Materiality focuses on color meanings analysis related to Halliday's metafunction.
In the ideational function, color clearly can be used to denote people, places and things as well as classes of people, places and things, and more general ideas, such as the colors of flag, corporate identity, safety codes, sign systems, maps.Color is also used to convey 'interpersonal' meaning: to impress or intimidate, to warn against obstructions and other hazards.It used by people to present themselves and the values they stand for, to say in the context of specific social situations.Color also functions at the textual level.Color can be used to create coherence in texts.Textbooks make wide use of this, whether in 'reading schemes' or in mathematics texts to indicate 'levels' of difficulty, or in science textbooks to provide topical unity.The whole procedure of visual grammar analysis in social semiotics meanings is illustrated in the graphic as follows after the conclusion.

CONCLUSION
The social semiotics provide the visual text a vital role, as one of the material form of ideology, which is produced circulated, consumed, and contested, both in large and small scale, or formal and informal environments in certain communities.This role allows researcher to place the term of ideology not only as a product in the political practices, but also as the semiotic resources that is moving dynamically in the market of meaning-making in all aspects of culture.
One model of social semiotic analysis of visual text is the visual grammar, an account of the explicit and implicit knowledge and practices around a resource, consisting of the elements and rules underlying a culture-specific form of visual communication.
The aspects analysed consist of: elements and structure, position of the participants, modalities, composition, and materiality.Researcher can develop this analyses in a wider framework surpassed the limitations of structural semiotics, by involving directly into the semiotic resources around knowledge and cultural practices in everyday life.As a study which is situated in critical paradigm, there are very wide spaces for researcher to interact with various visual semiotic resources that can be related each other to generate values, discourses, and a new way of seeing in a more inclusive consciousness.

Procedure of Analysis
Kress and van Leeuwen base visual grammar on Halliday's theoretical idea of metafunctions: