Images, perceptions and reality!

Notes from the Author

This work has alrerady been a previous small portion of a thesis dissertation at Universidade Federal Do Río de Janeiro (1989). It was then presented in Argentina, Germany to finally be published at in the UK. This is a slightly modified version so as to be introduced into this blog. Thanks. The author.


“If within the human species the word is a privileged channel for defining, objectifying and constructing reality, nevertheless reality cannot be exclusively defined by means of the word : images, sounds, conducts, rites… are other ways of generating and communicating “multiform” aspects (not necessarily complementary and, in some cases, antagonistic) of social representations.” De Rosa and Farr, (2001b)


The concepts of Full-Content-Image-based Interfaces, Iconic Function and their components are presented. The approach gained quantitative support in the theory of information ABRAMSON (1981),  whereas qualitative enhancement was found in some premises from “An Image-Based Method to develop Educational Software”, MANGIONE (1989).

This approach, is based upon an integrative vision of reality which for educational purposes, involves the Human, Social and Technical Factors. In some other applications such as mass media communication, primitives may well be defined according to the domain of the problem.

This work presents A first-approach method for image reading  and writing which is built upon the concepts of Explicit and  Implicit Primitives. The approach is highly   structured and  clearly defined since it is built upon a set of   Primitives, Macros and basic iconographic elements.  These components are tightly vinculated through a Mereological Theory; BUNT,   (1985),  and The Whole-Part Relations (-a branch of the Inclussion Semantic Relations-), WINSTON, CHAFFIN & HERRMANN, (1987).

There are also two basic taxonomies which have been used so as to provide this approach consistency and further projection to  computational,  educational and diagnosing  applications: A  taxonomy for Visual Events, COBUM, (1975) and the taxonomy of   Iconic Substitutions, DURAN, (1973).  Examples are given on how to read and  write  different kinds of images and the way they could be  coded  making  use  of the methodology,  techniques  and  tools  developed  herein.


The approach is entirely semantic since Whole-Part Relations  are used into a tight conjugation with all kinds of visual events  and  a  group  of valid icons substitutions.  The  scope  of  the  proposal as  a whole is the study of Imagery and Social Representation, under  the  latest   conceptions   of   linguistics  and  imagery.

As it is indicated by the title this is a method: it  is neither a authoring system  nor a Computer-Aided-Instruction program. It is an epistemic approach especially developed to show how  imagery can be treated as a reading/writing object. It is important to make clear that this iconographic approach is complementary to the existing ones since it privileges intellectual work based upon visual perception. Regarding some unfeasible abstract or logical applications within an iconographic approach, we suggest physical and conceptual modeling when limit situations appear. It is then important to see the boundaries of iconographic treatment.



Assuming that the two basic concepts of WHITE, (1987) regarding information technology are true, i.e., increasing amount of information and information selection criteria, we shall introduce  this section by discussing some of the most important aspects of text, imagery, opposite and complementary tendencies. 

TEXT AND IMAGE-BASED INTEFACES: Handling graphics, meaning words.

Far from our purposes is the intention of uselessly critize the real and well recognized merits of text-based interfaces. Most of what we have in this life is through language and its sentential representation. As the reader can see , we are not blaming text when it is, so far, the only means we have to talk about images and besides that, it is so powerful that it can talk about itself.

Although we have particularly nothing against text, we are advocated to the task of writing about images in addition to compensate some of the text-abusing problems. Imagery, in this case, will be treated as a special topic. It is certainly our main concern and we are definitely trying to enlarge its field, to enhance its semantic domain and to expand its information organization power up to the point where text has come. Nevertheless, there is no reason to assume we would not be accepting the contribution of both text and image-based interfaces.

Finally and according to De Rosa and Farr (2001b), “if the study of the ways of communication allows the reconstruction of the nature, structure and dynamics of social representations, then the scholar’s duty is to consider all the possible  channels of communication and not only the verbal one.”


WITTGENSTEIN, (1981), was a critical observer of images and its attributes. He said that an image may or may not transmit what the reality is. Then, the character of an image as a true-value transmiter, will be under discussion. He said that an image may transmit both what the reality is and what is not since image is only a reflection of the reality and it is “used” to show how the reality is but it does not prove to be true.

Some contemporary authors are very concerned about the final translation of images into propositions, WINN, (1982). From a semantical point of view, we assume that there is a close relation between what we called Iconic Functions and logic thinking (which is basicaly propositional), otherwise there would never be perceptions PRIETO CASTILLO, (1988), equivalent to awareness, percepts, WINN (1982), as a previous stage for concepts, indeed, thinking activity from visual experiences.

The organizational power of image is best defined by WHITE, (1987) when she says that we have been trained to learn how the reality is through text. That is, from left to right and from the top to the bottom which is certainly poor if compared to the amount and quality of information that may be processed at a glance (spontaneous time-space distribuition is acquired), with a simple eye movement. Search and Recognition are  always linear in sentential representation being more complex but flexible and integrative under visual treatment.

Inference is largely independent of representation (provided the rules are applied to the same contents). Search and Recognition are quite different in text and image representations. The balance of cost among Search, Recognition and Inference is mainly on the Recognition stage because it is the major efficiency difference between the sentential and the iconic (-diagrammatic in the original-)  representation. A visual (iconic) system  provides, at virtually zero cost, all kinds of perceptual inferences, LARKIN & SIMON, (1987).


A Full-Content Image is a graphical representation of the reality. This representation may both have analogical and logical correspondence with the reality where for educational purposes, Human, Social and Technical Primitives are present, either explicit or implicitly.

An Explicit Primitive denotes the reality through a graphical, symbolic or analogical representation. On the other hand, an Implicit Primitive is the one which arises from a deeper level of ” iconic reading” because it was never “written” onto the image. It  has been connotated within the contents, being “pure meaning” but not the “word” itself.

It should not be confused what is not represented (because it simply does not exist) with what is implicitly represented. This means that some of the Primitives may not appear explicitly but they may have been actually taken into account under a different representation figure, that is, they were connotated into the image.

We can then say that an Explicit Primitive (Ep) is the one which holds a complete matching between the agent’s external and internal representation. On the other hand, an Implicit Primitive (Ip) is the one the agent can detect because of his semantical connections rather than by  an analogical or somehow graphical correspondence with the reality.

WINN (1982) makes reference to concepts like these as follows: “…the correspondence between visual information and reality is not important but that the correspondence between the internal and external forms of representations…”


We define an Iconic Function as a set of Primitives (a Full-Content Image) semanticaly connected to produce a true-value representational entity which reflects both by analogy and logic icons either the real or formal world. Internal coherence is an essential requirement to satisfy the true-value condition.

As a consequence, it will be finally necessary to have a one-to-one correspondence among the Primitives of our function and the elements from reality. Indeed, the mutual correspondence between the Representational Entity with the real or formal world as a whole and the corresponding true-value relations among different parts of that picture. Otherwise, we would only have a full-content image but not an Iconic Function.


Every Primitive participating into an Iconic Function will always have some attributes and features which specificaly let the agent communicate. Some of these attributes and general features are, MANGIONE, (1989):

i.- Shapes: made up of lines, squares, circles, points as well as some other irregular or alternative elements;

ii.- Size: which may be represented by a pixel, a letter box or a full size screen. Size will depend on perceptual faculties especially on distance comprehension from the agent’s point of view, ARHEIM, (1985).

Sizes are highly dependent on the dimensions of the screen and more than five different sizes are not advisable. Shapes should be treated in addition to their nature of geometric or pictorical where the latter requires – for the same resolution-, as much as double amount of pixels for the desired final presentation, HEPP & ALVAREZ (1988).

iii.- Chromaticity: (depending on the technical attributes of the machinery) it will be of a big help for further applications in diagnosing and prescriptions.

As long as chromaticity is important, bright colors should be treated with caution since they provoke  a persistence effect on visual perception mechanisms. Moreover, use of complementary colors should be well administrated since they drag to a vibration effect, HEPP & ALVAREZ (1988).

iv.- Text, Windows and Special Functions:  (such as rotation, translation, scaling and some other functions to work on the shapes once we have got them done).

As in many other fields related to structured methodologies, we will based our Primitives on the concept of Macros as subsidiary procedures of Primitives. We will be calling Macros those Structured Graphical Objects with some or most of the attributes and features mentioned above. They will be easily called up by their names just as we do in structured programming.

Within this morphological approach, an Iconic Function may be specified as follows:

Iconic Function (I.F) = F(X,Y,Z … )


A Primitive for Human Component                      X = f(r, s, t, u)

A Primitive for Social Component                        Y = g(r, s, t, u)

A Primitive for Technical Component                   Z = h(r, s, t, u)

are the Global Variables or Primitives and,

r = shape

s = size

t = chromaticity

u = special functions

are the local variables of the problem.

As WHITE, (1987) has warned us about emerging problems of information overloading, HEPP; ALVAREZ, (1988) recommended not to use more than seven concepts on the same screen. Exceeding this limit may cause short term memory to fail easily.


We will present some comments related to a comparative study, matching both text and imagery information transmission power. It should be noticed that the survey does not either make a proposal for meassuring up the amount of information understood by the agent or what should be called “information”. We know that information varies from one subject to another but we are interested in the quantitative aspects of transmiting coded information so as in any Mass Media.

The quantitative approach we are talking about is the originally given by ABRAMSON, (1981). He made some Radio and Television comparisons in terms of capacity for information transmition. We now make some adaptations on this  calculus to apply it to some text and images coded messages always into a computational enviroment, in this case, a simple computer.

Making some figures, we can say that the probability P of an E event to accur is P(E) so when it occurs we have transmited

I(E) = log  (1/ P(E))  information units

Let’s suposse now that we are not talking about two different means like Abramson’s radio and television sets but about the same means as a computer video screen with minimal graphic attributes. Let’s also assume that we are going to analyse the amount of information this very same screen is able to transmit both when using sentential representation (text) and imagery.

Just to take a simple screen we say that this one has 800 x 600 pixels and has only the possibility of generating black and white images. We are putting some restrictions to facilitate the calculus and to show the reader that it is not even necessary to add chromaticity to prove that even in a restricted environment, imagery proves to be superior than text in its instant information transmition power.

Suposse that the screen generates 10 different shades of gray. Thus:

10^(800×600)   =   10^(480.000)

different images may be generated. Assuming that all of them are equally probable, the probability of one of them is:


Now, let’s  consider another point of view. Assume we have:

i.- A set of 3,000 available words to describe what an image holds;

ii.- The standard screen is 80 columns and 25 rows what yields 2,000 box-letter positions;

iii.- We will work with 5 letters long average words.

Thus, 2,000 boxes in the screen and a five-letter length  for each word, we have 400 possible words on the screen except that we need some blanks to separate them. So, let’s assume we take off one (blank) space every five letters. This yields about 320 word on the screen to explain the content of our image.

Amount of information given by an image is:

I(E)Image = 800 * 600 * ((log 10)/(log 2)) = 1.536.000 bits

Amount of information given by text is:

I(E)text= 320 * (log 3,000/log 2) =3,696.93 bits

And the ratio between them is:

I(E)Image/I(E)text= 415.48

Finally we can say that, in terms of instant transmition information power, each of our images is around a 400 hundred words. In spite of the fact that authors have some doubts about Abramson’s calculus and phylosophical questions have risen, his calculus has proven to be good enough in our approach so as to technically depict the advantages of using imagery for information transmition.


Besides the morphological features previously mentioned, it should be remember that there exists a whole variety of psychological mechanisms vinculating external and internal  representation. Awareness, belief and knowledge  phenomena will be revised so as to introduce some interesting concepts into imagery.

According to FAGIN & HALPERN, (1986), there is a kind of belief based upon the intuition about certain real or formal P fact. This is called Implicit Belief (L). On the other hand, there is another kind of  belief called Explicit Belief (B), which is associated to Awareness (A). Thus we say that,

B = LxA

Maybe the easiest way to refer belief to awareness is the following. Let’s suposse we have an Iconic Function, say Fi. As we have just anticipated, every Iconic Function can be divided into Primitives. We then say that the agent may have Explicit Belief of the Fi function if he is Aware of every primitive in the function. Thus:

A(p1) ^ A(p2) ^ A(p3) ^ … A(pk) –> B(FI)


A        –> the modal operator for Awareness;

B        –> the modal operator for Belief;

p1,2,3,k –> any primitive of  the function;

^        –> logic conjuction.

The expression says that an agent explicitly believes in a function FI (in the sense of accepting it, being sure about its contents or to assume it like possible) if only he is aware of all the primitives in that function.

This point is extremely important when we are referred to instructional and educational imagery. The use of imagery has certain advantages, according to PARK, (1985), WHITE, (1987), MANGIONE, (1989) which are fully complentary with FAGIN & HALPERN, (1986) approach in addition to detect and diagnose what an agent perceives (-is aware of-) and the way he does it: explicitly, implicitly or by an harmonic semantic combination. Socially speaking, it  is also advisable to take into account De Rosa and Farr, (2001b), when they introducie  -in terms of belief-, the following concept: ”the code of iconic representation can be considered as a specific and privileged means of expression of beliefs and of “irresistible” representations, deeply rooted in our social memory.”


ARHEIM, (1985) states that the cognitive operations (namely, thinking) are not a privilege of mental processes beyond perception but ingredients of perception itself. There is something else to say about this subject especially regarding to mental spaces and visual perception operations. WITTGENSTEIN, (1982), for example, remarked that we observe the organization of an object or better, we observe something related to its organization, perhaps a characteristic of its organization. This is obviously achieved in terms of two basic operations: abstraction and generalization, ARHEIM, (1985). What an agent perceives are global qualities rather than singular features.

Needless to say those two operations are closely related to each and every single subject. Perception of an object  is not transferable from one subject to another although it is possible to transmit the impressions that an image has produced over a particular subject, paraphrasing WITTGENSTEIN, (1982). Besides that, visual perception, percepts construction and concept building depends on personal experience (individual source). The richer the visual experience (internal representation), the more focalized the subject’s attention on external representation aspects. At the same time, a pattern arises from common features emerging from many objects and the integration of their attributes.

This activity of integrating common features from n objects is  conducted by two essential cognitive operations, namely, Generalization and Abstraction. The former operation may be defined as the one consisting in the detection of the transmitted patterns by an  object. Should the sample involve many objects, generalization also implies synthesis. On the other hand, Abstraction is defined as the operation consisting in recognizing (extracting) the pattern emitted by the object from the context.

The ability of perceiving an object as immutable is to produce the highest possible generalization. Abstraction is activated by generalization for a further reconstruction of the reality.


We have to analyse one example so as to make clearer the concept of implicit primitives, inside and outside the picture. How would we analyse a written piece of music?

First of all we have to say that:

i.- The piece of music has been coded to be perceived by the sight sense;

ii.- It has been meant to be uncoded by the same sense but to be checked up by the ear sense and performed mainly with the hands;

It will also be of a big help  to admit that:

i.- It is not a typical picture but;

ii.- It is a text-like representation entity since it uses symbols (whole, half, quarters and so on);

iii.- The reading implies that this is not a simple description. It has a descriptive component but it also has a performative component too. That is where the agent interacts with the image.

We can now begin our analysis in terms of primitves and the way they contribute to the picture as a whole. It should be recognized that:

i.- There is an Explicit Technical primitive being this read as the formula that coded music is and the instrument(s) it has been coded to be played on;

ii.- There is an Outer Implicit Primitive, namely, the Human Factor since he is the only one able to do this;

iii.- The Social primitive is the most complicated to be found and seen. How would we know whether the piece belongs to a Barroco author or to the Romantic period or to any of the possible ones?

The question has at least two answers: the literate musician knows just by the reading of the piece and the execution confirms it. The iliterate may be missing this primitive unless he developed a sense to distinguish intuitively one period from another.

As the reader can see, this keeps a close relation between what we have just seen about belief, awareness and knowledge. Let’s see.

For the literate, not only he intuitively believes the music belongs to a social, musical or artistical movement but also he is aware of it because he perceives it by the reading and checks it by playing the piece. So he has an intuition and some experience of what he sees and plays and that is formally written in our modal logic as:

B1 Brand.6 = L1 Brand.6  ^ A1 Brand.6

which is read as: the agent 1 has an intuition of Brandenburg Concert number 6 and besides that, he is conscious (aware) of the same piece. This yields an explicit belief of the musical piece.

On the other hand, an iliterate with all his talents may implicitly believe in the piece of music because  he only knows it by ear but would not match reading with performing criteria. So he is based upon an intuitive believe (L).

L2 Brand.6

in this case the formula is simply read as follows: the agent 2 intuitively believes in the piece of music. This yields that our piece of music is a “true-value representation” for the literate but it is not for the ilitarate.


Summarizing, we can say that every image can be observed, analysed and interpreted under the light of this simple first-approach method.

We then suggest that:

1.- Amount and kinds of Primitives must be determined in terms of either Explicit or Implicit components;

2.- The domain of Implicit Primitives must be carefuly studied in terms of Inner Implicit Primitives (Iip) and Outer Implicit Primitives (Oip).

3.- The character of the primitives must be clearly detected in terms of Descriptive (Ds) and Performative (Pf) components.

Applying this to the examples developed in this work we finally say that:

A set of paraboles are a Non Interactive Iconic Function where there is nothing to be performed. It is a description. It is a graphical representation of a mathematical formula. The only primitives participating in such a  picture are inside of the territory of the representation entity.

A Warning Traffic Signal and/or the piece of music are Interactive Iconic Functions:

i.- They are interactive because they demand man participation to be performed. Otherwise they would not satisfy the true-value condition.

ii.-They are also a description because they have already been coded too;

iii.- Some of the primitives will be inside the domain of the picture and we call them Inner Implicit Primitives (school in the signal). Likewise, there may well be  some other primitives outside  the domain of the picture and those will be called Outer Implicit Primitives (man and car in the same traffic signal).

Let´s analyse the complete meaning of this Traffic Signal according to our methodology. It is on the power point figure. (Let´s read!)

This is not the only topic we should have into account. There is also necessary to notice that, there are three  main component or Primitives as they are:

a.- A Human Primitive,

b.- A Social Primitive and,

c.- A Technical Pritive.

Let´s take a deep look at this ireading:

As it is clearly seen in this picture, Social factor has been depicted.

And completng the other tow components, we finally get the Warning Traffic Signal!



An Iconic Function would not be made just upon a simple  aesthetic combination of components. It is also necessary a  semantic foundation to support everything that may be constructed  from Macros and Primitives integration. Otherwise, we run the  risk of producing what was best called by THIOLLENT, (1989) “a  sort of bricolage”.

We need some linking relations which are,  in this  case,  a  branch   of  the  general  semantic  relations  called  Meronimic  Relations.  According  to  WINSTON,  CHAFFIN &  HERMANN,  (1987),  “mero”  comes from a greek word meaning “part  of”.

The  consistency  of Meronimnic Relations is of  a  definite  relevance  for  both deductive and  inductive  iconic  inference.  Nobody  would  be able to produce meaningful pictures and  images  unless meronimic relations were actively contributing to  achieve  this goal.  Within the piece of music or into a set of paraboles,  there  are  semantic relations supporting the final coherence  of  the  work.

As  we can easily see from this taxonomy that there are  six  kinds  of  meronimic  relations all of them of a great  value  if  applied correctly. We will just mention some of the guidelines to  apply the taxonomy to the graphic domain and what is the way they cooperate in a sense-making picture (namely, an Iconic Function).

i.- First  of  all it should be considered the fact that  we  have  based  our  proposal on  Elements,  Macros  and  Primitives  participation so that meronimic relations will be working in each  of these levels and among them;

ii.- Besides that,  it will also be of a big help if we kept  in   mind  that  a  Full-Content- Image-Based   Interface   would  exceptionally  be  based  upon only one image so  that  meronimic  relations will be linking not only the parts of a “graphic frame”  but also,  different frames so as to reach a global meaning (just  as  it  is  necessary to have  sequences  of  images  semanticaly  linked).

iii.- Inference   (either   inductive  or  deductive)   will  completely  depend on these semantic relations.  Otherwise  there  will not be any possible Iconic Inference.


They may get achievable in many ways. We propose  three different  ways of developing Iconic Dialogs: The first one which we call “a  Whole-based” Iconic Dialog; the second, “the Part-based” Iconic  Dialog and the “Natural-Language-Primitive-based” Iconic Dialog.

We will explain each of these alternatives since either one can produce the desire communication effect with the agent. Regardless the kind of Iconic Dialog we are interested in, Meronimic  Relations are simply crucial to achieve any satisfactory iconic  interaction.

The Whole-based Iconic Dialog

When we first presented the concept of Iconic Function, we  said that it is the representational entity which completely  reflects the real or formal world under study (i.e. the entity  that holds true value).

What  we  are going to do here is simply to propose  a  way  of  semantic  work which consists in dismounting the entity according  to one or many of the modalities listed below. We will simply  go from the Whole to the Part splitting  up the Continent meaning as  a whole, all the way  down to the content meaning of every single  primitve,  macro  or  element.  It  is  easy  to  see  that  this  iconographic  entity may well be semanticaly divided according to  the following criteria:

II.1.1.- “Split up”  according to the primitives: which  are the dominant components in this approach;

II.1.2.- “Split up” according to the macros:  currently  components of the primitives as subsidiary procedures;

II.1.3.- “Split   up”  according  to  basic   elements:  meaning the maximum possible dismounting level.

II.1.4.- “Split  up”  according  to  similarities   (or  differences).

The Part-based Iconic Dialog

Related to the previous  one, this is exactly the opposite  way of establishing an Iconic Dialog. We will go from content  meanings to continent meanings. We will develop a semantical  integration activity all the way up  to the global meaning of the  scene itself i.e., we start up from the part being the Whole our  final goal.

It  is  easy to see that the same kind of activities may  be  developed from this point of view only with a change in the  kind  of  operation.  Instead of “splitting up”,  the logical operation  will  be  now to “make up” different portions of  the  entity  to  finally get to the whole.

The Natural-Language-Primitive-based Iconic Dialog

We know man is not very used to establish an Iconic Dialog,  mainly if he has got to “shoot” the first “word”. Remember this:  word is not exactly written into a sentential representation. It  is imagery itself. This is why we propose some alternative ways  of facing this new problem according to human possibilities and at a low cost in terms of cognitive resources versus expected outco­mes.

Creating  an Iconic Dialog starting from the  primitives  of  natural language is feasible under one of these three modalities:

II.3.1.- User-guided  dialogs:   which  clearly  speaks  about   the  initiave  of  the  agent  to  establish  the  iconic  communication;

II.3.2.- Machine-guided Dialog:  in which the  beginnig  of  the dialog relies on the machine attributes to develop  these  activities;

II.3.3.- Man-Machine coordinated dialogs:  where either  the  agent  or  the machine can establish and  direct  the  whole  course of the communication.

Considering  the  normal  effort   that  Iconic  Dialog  may  require  from  the  agent,  he will sometimes be allowed  to  use  sentential language, but the method requires either the system or  the  machine or an heuristic environment to answer by icons.  The  agent  may  always  have  the chance to choose  one  of  the  two  modalities  which  are merely proposed so as to  facilitate  this  otherwise, hard task:

i.- The agent will feed the system or environment with words  referring just nouns (or objects) to what he will receive  either  an explicit or implicit iconic primitive;

ii.- The  agent will feed the system or the environment only  with verbs, infinitive or conjugated but just verbs.

Both  ways are meant to lead us to work in the  frontier  of  iconic-formal-propositional representations. It should be noticed  that the level of abstraction used in this image-word interactive  dialogs  is especially applicable to people with complete  domain  of  the oral and writen (sentential) language.  For children,  it  will have to be simplified so as to produce simple iconic dialogs  based upon the restricted set of iconic primitives children  know  and their also restricted set of natural language primitives.

The  ratio between the amount of recognizable set of  images  and  words will have to be strongly marked in favor of the images  since a (sentential) language matches a single code (word) with a  single concept.  On the other hand,  there are not two  identical  pictures to represent the same concept ARHEIM,  (1985); (there is  one  word  to  the concept of “lake” but there are  thousands  of  pictures  of  lakes  and  they  represent  the  same  concept  in  different ways and with a whole variety of attributes).

It  should also be remembered that an iconic  representation  (diagrammatic   representation  in  LARKIN     SIMON,   (1987)),  preserves  explicitly  the information about the topological  and  geometrical relations among the components of the problem,  while the sentential representation preserves other kinds of  relations  like  temporal  or  logic  sequence.  Iconic  primitives  meaning  topological  and  geometric  properties would not  be  the  right  answers to temporal or logic questions unless a thorough analysis  is  made  by an interdisciplinary  team;  phsychologists,  Visual  communication  experts,   analysts,  knowledge  engineers  should  participate coordinately to keep expectations of success.

As in any other interactive way of communication, there must  be an answer from the machine for a dialog to take place. This is  why  we are going to need some equivalences and substitutions  so  as to let the agent and the machine interchange  icons,  figures,  pictures,  indeed,  meanings.  This  is why we propose a list  of possible  kinds of substitutions in addition to facilitate fluent  man-machine   interchange.   This  is  what  into  a   sentential  representation is called synonyms and paraphrasing.


There would not be any possible  communication  process  unless we have a  repertoire  of  images,  namely  icons,  diagrams,  figures,  and  the likes to produce  a  permanent  and fluent interchange.  This  repertoire will frequently need synonyms and alternative  figures  to the same “word”.  This is why we propose the use of a taxonomy  for substitutions which will be developed as follows.

When  sentential representation is used -as LARKIN &  SIMON,  (1987), likes to call text-,  paraphrasing may well be one of the  possible  solutions to avoid ambiguity;  but,  what about working  into an imagery environment?  Do we have some techniques and  tools  to produce a similar effect into imagery?

This  is  a   part of our proposal and it is  based  upon  a  taxonomy  from  DURAN,  (1973) which accurately adjusted  to  the  approach,  facilitates Iconic Dialogs seeking Iconic Inferences.

Duran’s taxonomy holds four kinds of susbstitutions:

i.- Identical  element substitution:  from which,  everybody  knows  at least,  “emphasis”,  (other than hyperbole and litote).  See examples in DURAN, (1973), MANGIONE, (1989);

ii.- Similar  element substitution:  divided in “formal  and content   substitutions”;

iii.- Different  element substitution:  which may be one  of  the   best  suited  to  work  with  Inner  and   Outer   Implicit   Primitives. Different element substitution is divided into:

a.- Cause by effect substitution;

b.- Effect by cause substitution;

c.- Object by destiny substitution;

d.- Whole by part substitution.

iv.- Opposite element substitution.

If the question, “What are these substitutions good for?” is  still in the reader’s mind,  the answer is another question:  How  does  the reader think the agent could ever produce an  inference  if he gets stucked up at the  first or second image he is exposed  to?  Would  he  ever  be  able to understand every  line  of  the emerging  Iconic  Dialog?   Of  course,   not.  The  iconographic  treatment  deserves  as  many  chances  of  well  developed   and  meaningfull  communication   just as it can be performed  through  text.  Synonyms, paraphrasing and some other usefull figures must  also  be  translated  and  materialized  into  any   iconographic  approach.

To  make these substitutions possible we have to make  clear  that   there  will  be   necessary  some  criteria  in  terms  of  equivalences   between   what  is  being  substituted   and   the  substituting primitive. We then say that:

i.- Two  representations are informationally  equivalent  if  all  the information in the one is also inferable from the  other  and viceversa.  Each could be constructed from the information in  the other, LARKIN & SIMON, (1987);

ii.- Two  representations are computationally equivalent  if  they  are  informationally  equivalent  and,   in  addition,  any  inference   that  can  be  drawn  easily  and  quickly  from  the  information given explicitly in the one, can also be drawn easily  and  quickly from the information given explicitly in  the  other  and viceversa, LARKIN & SIMON, (1987).


COBUN, (1975)  contributed to this work with his taxonomy  for visual events. No matter how many substitutions there must be  into an iconic communication process untill the agent gets the  right meaning. What is really important is to lead him straight­ forward to where the concepts are.

In  addition to produce very organized and  highly  coherent  substitutions,  we  should also know that there are several kinds  of  visual events.  Whatever we do when substituting  primitives,  macros and elements in the picture,  we have to be aware that  we  will  sometimes produce a change in the “kind of event” the agent  is exposed to.  Remember that there are events sharply related to the reality and on the other hand, there exists a rich variety of  events clearly belonging to fantasy or figurative domains.

Any  time  you  lead  your agent’s attention  to  a  special  concept,  make sure that  (you are and that) he is aware he might  be  conducted  through  some  alternative  paths  which  are  not  strictly  related to the reality  and to the nature of the  world  under study.

We summarize Cobun’s taxonomy as follows:

i.- Documentary event: which is recorded as it occurs;

ii.- Reconstruction  event:  that  one  that could  not  get  registered as it happened;

iii.- Contrived event:  especially devised to show the agent  how things would be “if…” some conditions took place;

iv.- Logical  situation event:  produced to show  the  agent  predicted outcomes;

v.- Inductive situaton event: a typical “what if…” event;

vi.- Deductive  situation event:  especially devised to give  the agent some elements for problem-solving activities;

vii.- Open-ended  situation  event:   the  one  with  a  non  deterministic solution;

viii.- Demonstration event:  the one which has been designed  to show a process step by step;

ix.- Expository  event:  ideally  applicable to  show   from  beginning to conclusions any event with closure;

x.- Participatory  event:  an essentially interactive  event  devised to present objective-oriented contents;

xi,  xii,  xiii.- Theatrical,  Fantasy and Animated  events:  very apropriated in children applications;

xiv.- Data event: mainly related to statistics;

xv.- Programmed  event:  which may well involve most of  the  just mentioned events.

Under  the  light  of this classification  it  is  advisable  before doing anything else to:

i.- Classify   every  scene  within  the   iconic   sequence according to the event it is involved in;

ii.- Make  some  “chapter-like” array (a semantic  array  in  our approach) with the images within the sequence;

iii.- Give  the  whole sequence of images the  name  of  the  general characteristic event it is involved in.

These  general  hints  will  certainly make  clear  how  the  taxonomy is applied  to a well organized iconographic treatment.  Nevertheless, it should be remembered that according to De Rosa and Farr, (2001b); the construction of a theory of imagery categorization could well be as follows:

  • “as a source able to activate social representations or favour the development of new social representations ;
  • as a product of social representations, i.e. an iconic-symbolic synthesis, a condensed materialisation of one social representation, a direct expression of the objectification process, in other terms as itself as a social representation ;
  • as a medium, a specific form of transmission, linked to differentiated channels (traditionally the visual arts, but also television, cinema, photography, new audio-visual technologies, internet etc.), through which new or pre-existing social representations are conveyed.”


We have already developed both an algorithmic and an heuristic approach in MANGIONE, (1989). We will simply say that either  one will hold an Image-based proposal for a First-Approach-Method  for Image Reading and Writing as well as to develop Full-Content- Image-Based Interfaces.

DWYER,  (1980) defines algorithmic approaches as being  pre- determined  sequences  to  achieve some  goals.  The  algorithmic  approach suggests to begin by the output specifications in  terms  of  objectives  to  further devise the procedures to get  to  the  desired outputs.  An algorithmic procedure offers the possibility  to  give  the  problem structure and  precision,  contributing  to  encapsulate   that  precision  into  a   reproduceable   way.   An  algorithmic  modality  will always produce the same outputs  when  activated by the same inputs under the same process.

On  the  other hand,  there is the chance  of  promoting  an  iconic  heuristic environment.  This would be somehow opposite to  the behaviorist-fashioned-algorithmic approach. The basic support  of this modality are the Cognitive Psychology and the heuristics.

i.- Heuristics   favor   an   open-ended   modality    where microworlds are explored through some procedures meant to let the  agent conduct the whole session.

ii.- Heuristics  work locally since when there is a good one  to  solve a problem within certain domain,  it temporarily  loses  the  sense  of the integrity it is   involved  in.  Nevertheless,  heuristics tend to satisfy the objectives for  a global result.

iii.- Heuristics  imply several procedures where  the  final  state  of the first procedure  is the initial state of the second  one and so on.  Basicaly,   heuristics do not say what to do  but  how we could do something.


We have developed a set of modal logic operators to give the  approach the same formal treatment as we did with awareness,  belief and knowledge. We gave them the name of Graphic Modal  Operators. We have created this set of operators which are divided into categories closely related to what is expected to be the  reality of human being levels for perceiving tasks and cognitive  operations, such as: BONNET, (1988); LIVET, (1988);

i.- A  Sensory  level,  which gives the first  sketch  of  a  visual  activity;  it is considered as being part of the  stimuli  itself;

ii.- A Perceptive level, where different configurations take  place through structured representations;

iii.- A   Semantic-Cognitive   level,   which   deals   with  intermodal aspects catalyzing processes coming from the  previous  levels.  It  is in this level where modelling and meanings happen  to take place.

In  order to create an apropriate environment to develop our  Graphic   Modal  Operators  we  have  divided  them  into   three  categories such as,

i.- Psychologic and Perceptual level, with the Search (Sh),  Visualize (V)‚ and Perceive (P)‚ operators where:

*  Search is the visual inspection within the  search  space  seeking structural elements of the picture;

*  Visualize  indicates  the agent is able to  localize  the  elements in the search space;

*  Perceive  indicates the agent is aware of what is in  the  search space.

ii.- Cognitive   Level,   which  holds  the  Recognize  (R),  Abstraction   (Ab),   Generalization   (G)‚  and   Inference   (I)‚    operators;

*  Recognize indicates the coincidence between internal  and  external representation;

*  Abstraction  indicates  the agent is able to extract  the  object from the context;

*  Generalization implies  the capabilities of the agent  to  extract the pattern from an object (class);

*  Inference  indicates the  agent arrives to  a  conclusion  having started out from one or more premises.

iii.- Instrumental  level,  which is also  divided  in:

a.- Graphic:  with the Union (U)‚ and Division (D)‚  operators

*  Union‚  says the agent is able to produce  graphic-logical  union;

*  Division‚  means  an  action completely  opposite  to  the  previous one;

b.- Semantic:  with  the  Makeup (M)‚ and  the  Splitup  (Sp)‚  operators.

*  Make Up‚  implies the capability to semanticaly  integrate  parts into a whole;

*   Split  Up‚ implies capabilities exactly opposite  to  the  previous one.


The proposal we presented involves what we consider partially an answer to the emerging demands for Imagery  Literacy. Educational, technical, psychological and social aspects were taken into account to develop this communication where semiotic, semantics and graphic representation participate in an  integrative effort to conduct the human understanding to  comple­mentary and richer methods for a better comprehension of reality.

Man-to-man  and  man-machine  communication  processes  were  treated  as  been  transparent considering either one  like  “the  agent”.  Obviously  the  difference is based on the  quality  and  domain of application of this approach.

We  presented in this work the concepts  of  Full-Content  Images  and Iconic Functions.  We also promoted Iconic Dialogs to  reinforce the Iconic Inferences.  Within this approach, there are  three  useful taxonomies which let us produce  this  quantitative  and  qualitative graphic information interchange between men  and  for man-machine communication.

Artificial  Intelligence,  Ergonomics and Cognitive Sciences  are plenty of methods, techniques and tools to produce consistent  applications  of this approach either for training and  education  topics  as  well as in knowledge engineering or in  the  emerging  area of graphic interfaces.

The main contributions of the proposal are:

i.- The setting of the existence of Inner and Outer Explicit  and Implicit Primitives;

ii.- Modular and Graphic-Object-Oriented Macros;

iii.- The  mechanic  for Iconic Dialogs and some  hints  for  evaluating  and  diagnosing  tasks  closely related to  visual thinking and perception activities.

We  keep  our  hopes up that,  in the same way we  read  and  understood  a whole lot of images in our efforts to  bring  light  over this topics,  the method is effective and the reader will be  able  not only to apply it on his own but it he will also be able  to  produce satisfactory applications in systematic and  systemic  training,   education   tasks  and  in  the  design   of   either  conventional  or intelligent man-machine Full-Content-Image-based  Interfaces.

Juan Mangione,

Mendoza, 1 de diciembre de 2018. 19:38´


ABRAMSON, N.(1981); Teoria de la Informaci’on y codificaci’on, Ed. Paraninfo.

ANDERSON,J.R. et alii; (1984);Cognitive Principles in the Design of Computer Tutors. Proceedings of the Sixth Anual      Conference of the Cognitive Science Society. Boulder, Colorado.

ARHEIM, R.; (1985); El Pensamiento Visual; Eudeba.

BOLT, R. A. (1979); Spatial Data Management. Cambridge Massachusetts, M.I.T., 1979, apud HAWKRIDGE; D., New information technology and education. Johns Hopkins University Press.

BONNER, J. (1987); Computer courseware, frame-based or intelligent. Educational Technology,

BONNET, C. (1988); La Perception Visuelle des formes et du movement. Intellectica, 5-88-1; p.57.

BORK, A. (1980);  Learning through graphics; The computer in the school: tutor, tool, tutee; Taylor, R. (Ed.), Teachers  College.

BUDIN, H; TAYLOR,R; KENDAL, D (1987); Las computadoras y los estudios sociales. Rev. Informática Educativa, pp. 7.

BUNT, H; Mass Terms and Model-theoretic semantics.; Cambridge  University Press, 1985.

CERRI, S (1985); The intersection of A.I. and Education:  Investing  in  tough-intensive  endeavors; TRAPPL, R. Impacts of Artificial Intelligence, North-Holland., T.C. (1975); Target: Rational Design; Instructional Media      and Technology (Ed.) Sleeman, P & Rockwell, D.M., Halsted      Press; John Wiley and Sons.

COBUM,  T.C. (1975); Target: Rational Design; Instructional Media and Technology‚ (Ed.) Sleeman,  P & Rockwell,  D.M.,  Halsted    Press; John Wiley and Sons.

CURRAN, S; CURNOW, R. (1984); Juegos, Imágenes e sonidos; ed. Gustavo Gili S.A., Barcelona.

CYRS, T.E. (1975); Modular Approach to Curriculum Design using the System Approach; Instructional Media and Technology  (Ed.) Sleeman, P & Rockwell, D.M., Halsted Press; John Wiley      and Sons.

DE ROSA, A.S., FARR, R. 2001b Icon and symbol: Two sides of the coin in the  Investigation of Social Representations. In BUSCHINI F. & KALAMPALIKIS N. (eds.) Penser la vie, le social, la nature. Mélanges en hommage à Serge Moscovici. Les Editions de la Maison des Sciences de l¹Homme, Paris: 237-256.)

DURAN, J. (1973); Retórica   Imagem Publicitaria; Analise das imagens. Vozes.

DWYER, T. (1980); Heuristic Strategies for using computers to enrich  education, TAYLOR, R (Ed.); The  Computer  in the School: tutor, tool, tutee.Teachers College, Columbia University.

FAGIN, R; HALPERN, J. (1986); Belief, Awarness and Limited Reasoning: Preliminary Report; IBM. San José, California.

GOLDBERG, A; (1984) Smalltalk – 80 The Interactive Programming   Environment, Xerox, Palo Alto Research Center; ADDISON- WESLEY.

HAWKRIDGE; D., (1983); New information technology in education.  The Johns Hopkins University Press.

HEPP, P; ALVAREZ, C; (1989); Diseño de Interfaces Hombre-Computador con Multimedios. Anales IX Conferencia de la Sociedad Chilena de Ciencia de Computación y XV Conferencia      Latinoamericana de Informática. Santiago, julio 10, 14.

HOFMEISTER, A. (1984); Microcomputer Application in the classroom CBS College Publishing.

KRATHWOHL, D; BLOOM, B; MASIA, B. (1970); Taxonomy of educational objectives; David McKay Co. Inc., New York.

LARKIN, J.H.; (1987); SIMON, H.A.; (1987); Why a  diagram is (sometimes) worth ten thousand words. Cognitive Sciences, 11, 65-69.

LEGGE, D; BARBER, P. J. (1977); Informaçao e Habilidade; Organiz. de Herriot, P; City University, Londres; Zahar  Editores.

LIVET, P.  (1988); Image, Perception e Representation.; Intellectica, 5-88-1, p. 33.

MALONE, T.W. (1981); Toward a Theory of Intrinsically Motivation  Instruction; Cognitive Science, 1981, 5, 130-145,  apud  PARK, O; SEIDEL, R.J.; Conventional CBI  Versus Intelligent CAI: Development for the Future Systems.Educational Technology,, 15-21.

MANGIONE, J.C. (1984); Primer Curso  de  Inform’atica Educativa; Mimeografado, Escuela de Comercio Mart’in Zapata, Universidad  Nacional de Cuyo.

MANGIONE, J.C. (1985); Capacitación de docentes en el nivel medio.Primeras Jornadas de la Computadora en la Educaci’on; CRICYT-ME, Mendoza, Argentina.

MANGIONE, J.C. (1986);  Informática Educativa:   Una  Experiencia y una perspectiva. Primer Congreso Federal de Informática Educativa, Santa Fe, Argentina, Julio de 1986. Anales del      Primer Congreso Federal de Informática Educativa; COFEIN.

MANGIONE, J.C. (1986a);  Educaci’on Vial como factor de mejora de la conducta social. Primer Simposio Argentino de Seguridad y Educación Vial.

MANGIONE, J.C. (1986b);  El computador como  recursos  didáctico. Primera Feria Local de Informática, Mimeo. Mendoza,      Argentina.

MANGIONE, J.C.; OSES, M.R. (1986); Educación Vial y el replanteo de las normas: sustituci’on de la prohibición por la persuasi’on. Primer Simposio Argentino de seguridad y  Educaci’on Vial.

MANGIONE, J.C.; VALDEMOROS, M.P. (1986);  Informática y Educación  Vial; Primer Simposio Argentino de Educación y Seguridad  Vial, Buenos Aires, Argentina.

MANGIONE, J.C.; PRILUSKY, J. (1987); Redes  computarizadasen la educación.; Rev. Informática Educativa., Setiembre, N6.

MANGIONE, J.C. (1988); Informática e Educaçao Nao-Convencional; Boletín de Informática rcii – UNESCO, Año 6, Suplemento 1.

MANGIONE, J.C. (1988a); Tendencias  da comunicaçao em Linguagem Natural no Contexto Tecnológico, 40a Reuniao Anual da Sociedade Brasileira para o Progresso da Ciencia.

MANGIONE, J.C. (1988b); A imagem como meio cognitivo, a informática como recurso didático. Primeiro Congresso Internacional Imagem, Tecnologia e Educa,cao; Nucleo de Produçao da UFRJ.

MANGIONE, J.C. (1988c); Uma  abordagem geral  sobre  crença  e conhecimento no contexto social; VIII ENEGEP, Sao Carlos, p. 394-398.

MANGIONE, J.C. (1988d);  Uma abordagem geral  sobre  crença e conhecimento no contexto social; XVII Jornadas Argentinas de Informática e Investigación Operativa. (Versao completa), p.      794-814.

MANGIONE, J.C., (1989); An image-based method to develop educational software; Master’s Dissertation, Federal  University of Rio de Janeiro.

MANGIONE, J.C.; (1989a); Etica para la educaci’on del ciudadano en  la sociedad informatizada. Primer Simposio Internacional  de Etica para Situación Contemporánea. Universidad de Buenos      Aires. Septiembre 4-8.

MANGIONE, J.C.; (1989b); Transferencia y Adaptación de Tecnolog’ía: una visión desde la Gestión de Recursos Humanos. Primer Simposio Internacional de Etica para Situación Contemporánea. Universidad de Buenos Aires. Septiembre 4-8.

MANGIONE, J.C.; CLEIMAN, D.F.; (1989) Redes de Computadoras y Procesos Cognitivos. Tercer Congreso Federal de  Informática Educativa. Mendoza, Septiembre 4-6.

MANGIONE, J.C.; CLEIMAN, D.F.; THIOLLENT, M.J – M; (1989); Transferencia, Adaptaçao e Nacionalizaçao de Tecnología Educacional. IX Conferencia de la Sociedad Chilena de Ciencia de la Computaci’on y XV Conferencia Latinoamericana de Informática. Santiago de Chile, 10- 14 de julio.

MANGIONE, J.C.; CLEIMAN, D.F.; THIOLLENT, M.J-M.; (1989a); O  computador na escola: a questao administrativa, a questao pedagógica. Primeras Jornadas de Informática Educativa. San      Luis, Noviembre 24-26..

MANGIONE, J.C.; CLEIMAN, D.F.; THIOLLENT, M.J-M.; (1989b); video-jogos e programas educativos: análise e comparaçao. Primeras Jornadas de Informática Educativa. San Luis,      Noviembre 24-26..

MANGIONE, J.C.; CLEIMAN, D.F.; THIOLLENT, M.J-M.; (1989c); Fundamentos Psicológicos, Comunicaçao Visual e Tecnologia  Informática: Uma abordagem morfológica da Comunicaçao Visual homem-computador. Primeras Jornadas de Informática   Educativa. San Luis, Noviembre 24-26..

MANGIONE, J.C.; CLEIMAN, D.F.; THIOLLENT, M.J-M.; (1989d); Inteligencia Artificial e Informática Educativa: Interseçao e Integraçao. Primeras Jornadas de Informática Educativa.      San Luis, Noviembre 24-26..

MANGIONE, J.C.; (1989c);Tecnología, Universidad y el Autor Latinoamericano. Jornadas del Pensamiento Latinoamericano, Univ. Nacional de Cuyo, Mendoza, Noviembre 16, 18.

METZ. C. (1973); Além  da  analogía, a  imagem; Análise  das imagens, Vozes, p. 7-18.

NORMAN, D. (1987); Qué pasa en la mente  del alumno?;  Revista Informática Educativa, Año 2, n’umero 4.

ORTEGA, G.G. de; MANGIONE, J. C. (1985); La Computadora en la Escuela Secundaria; Revista Estrada, N 23, Año 6,  Setiembre, p. 25-27.

PARK, C. S. (1985); Interactive  microcomputer graphics; Addison Wesley.

PARK, O.; SEIDEL, R.J. (1987); Conventional CBI Versus Intelligent CAI: Development for the Future Systems. Educational Technology, 15-21.

PENINOU, G. (1973); Física e Metafísica  da  Imagem Publicitária; Análise das Imagens, Vozes, 60-81.

PRIETO CASTILLO, D; (1988); El Derecho a la Imaginaci’on.  Comunicación. Ed. Paulinas. N8.

RICH, E. (1985); Natural Language Understanding: How Natural  can it be?; Charles Weisbin (Editor); A.I. Applications. The engineering of knowledge-based system. Proceedings of the      Second Conference. Miami Beach. Florida; IEEE press, North-Holland.

SCHMUCLER, H. (1985); La  educaci’on en la sociedad  informatizada; La era teleinformática; Rodriguez,G; Comp. Ed. Folios.

SLEEMAN, D; BROWN,J.S. (1982); Intelligent Tutoring  Systems. Academic Press, 1982, apud DE OLIVEIRA, U. S. de (1988) Um Sistema Tutorial Inteligente; Tese de Mestrado, I.M.E., Rio      de Janeiro.

TAYLOR, R (Ed.) (1980); The  Computer  in  the School: tutor, tool, tutee. Teachers College, Columbia University.

TENNYSON, R.D. (1987); FERRARA, J., Introduction  to  special  issue in Artificial Intelligence in Education, Educational  Technology, p. 7-8.

TENNYSON, D.T. (1987); MAIS: an educational alternative of ICAI;  Educational Technology.

THIOLLENT, M. J-M; (1989); Personal Communication.

TRAPPL, R. (1985); Impacts of Artificial  Intelligence,  (Ed.), North-Holland.

VAIDEANU, G; (1987); La interdisciplinariedad en la enseñanza:  ensayo de síntesis. Perspectivas (UNESCO), N.64.

WHITE, M.A. (Ed.) (1987);  What  curriculum  for the information age?, Teachers College, Columbia University; Lawrence  Erlbaum Associates Publishers.

WINN, W.; (1982); Visualization in learning and instruction: A  cognitive approach. FCT, Vol 30, N1, p 3/25. spring.

WINSTON, M.E.; CHAFFIN, R. e HERRMANN, D. (1987); A taxonomy of  Part-Whole Relations. Cognitive Sciences, 11  p. 417-444.

WITTGENSTEIN,  L. (1982); Last Writings on The Philosophy of Psychology, Vol. 1; Wright & Nyman (Ed.), The University of  Chicago Press.

WITTGENSTEIN, L. (1979); Notebooks 1914 – 1916; Wright &  Anscombe, (Eds.), The University of Chicago Press.

WITTGENSTEIN, L. (1981); Tractatus Logico-Philosophicus;  Routledge & Kegan Paul LTD.

WOHZ, A.; CHAVA, E.T.; (1988); Learning to view view to learning: A macro and micro integrative teaching system of T.V. viewer’s. Tech Trends, January.

Deja un comentario

Tu dirección de correo electrónico no será publicada. Los campos obligatorios están marcados con *