Notes from the Author
This work has alrerady been a previous small portion of a thesis dissertation at Universidade Federal Do Río de Janeiro (1989). It was then presented in Argentina, Germany to finally be published at Anglohigher.com in the UK. This is a slightly modified version so as to be introduced into this blog. Thanks. The author.
“If within the human species the word is a privileged channel for defining, objectifying and constructing reality, nevertheless reality cannot be exclusively defined by means of the word : images, sounds, conducts, rites… are other ways of generating and communicating “multiform” aspects (not necessarily complementary and, in some cases, antagonistic) of social representations.” De Rosa and Farr, (2001b)
The concepts of Full-Content-Image-based Interfaces, Iconic Function and their components are presented. The approach gained quantitative support in the theory of information ABRAMSON (1981), whereas qualitative enhancement was found in some premises from “An Image-Based Method to develop Educational Software”, MANGIONE (1989).
This approach, is based upon an integrative vision of reality which for educational purposes, involves the Human, Social and Technical Factors. In some other applications such as mass media communication, primitives may well be defined according to the domain of the problem.
This work presents A first-approach method for image reading and writing which is built upon the concepts of Explicit and Implicit Primitives. The approach is highly structured and clearly defined since it is built upon a set of Primitives, Macros and basic iconographic elements. These components are tightly vinculated through a Mereological Theory; BUNT, (1985), and The Whole-Part Relations (-a branch of the Inclussion Semantic Relations-), WINSTON, CHAFFIN & HERRMANN, (1987).
There are also two basic taxonomies which have been used so as to provide this approach consistency and further projection to computational, educational and diagnosing applications: A taxonomy for Visual Events, COBUM, (1975) and the taxonomy of Iconic Substitutions, DURAN, (1973). Examples are given on how to read and write different kinds of images and the way they could be coded making use of the methodology, techniques and tools developed herein.
The approach is entirely semantic since Whole-Part Relations are used into a tight conjugation with all kinds of visual events and a group of valid icons substitutions. The scope of the proposal as a whole is the study of Imagery and Social Representation, under the latest conceptions of linguistics and imagery.
As it is indicated by the title this is a method: it is neither a authoring system nor a Computer-Aided-Instruction program. It is an epistemic approach especially developed to show how imagery can be treated as a reading/writing object. It is important to make clear that this iconographic approach is complementary to the existing ones since it privileges intellectual work based upon visual perception. Regarding some unfeasible abstract or logical applications within an iconographic approach, we suggest physical and conceptual modeling when limit situations appear. It is then important to see the boundaries of iconographic treatment.
I.- AIMS AND HYPOTHESES:
THE ORGANIZING POWER OF IMAGERY
Assuming that the two basic concepts of WHITE, (1987) regarding information technology are true, i.e., increasing amount of information and information selection criteria, we shall introduce this section by discussing some of the most important aspects of text, imagery, opposite and complementary tendencies.
TEXT AND IMAGE-BASED INTEFACES: Handling graphics, meaning words.
Far from our purposes is the intention of uselessly critize the real and well recognized merits of text-based interfaces. Most of what we have in this life is through language and its sentential representation. As the reader can see , we are not blaming text when it is, so far, the only means we have to talk about images and besides that, it is so powerful that it can talk about itself.
Although we have particularly nothing against text, we are advocated to the task of writing about images in addition to compensate some of the text-abusing problems. Imagery, in this case, will be treated as a special topic. It is certainly our main concern and we are definitely trying to enlarge its field, to enhance its semantic domain and to expand its information organization power up to the point where text has come. Nevertheless, there is no reason to assume we would not be accepting the contribution of both text and image-based interfaces.
Finally and according to De Rosa and Farr (2001b), “if the study of the ways of communication allows the reconstruction of the nature, structure and dynamics of social representations, then the scholar’s duty is to consider all the possible channels of communication and not only the verbal one.”
IMAGERY AND THE TRUE VALUE:
WITTGENSTEIN, (1981), was a critical observer of images and its attributes. He said that an image may or may not transmit what the reality is. Then, the character of an image as a true-value transmiter, will be under discussion. He said that an image may transmit both what the reality is and what is not since image is only a reflection of the reality and it is “used” to show how the reality is but it does not prove to be true.
Some contemporary authors are very concerned about the final translation of images into propositions, WINN, (1982). From a semantical point of view, we assume that there is a close relation between what we called Iconic Functions and logic thinking (which is basicaly propositional), otherwise there would never be perceptions PRIETO CASTILLO, (1988), equivalent to awareness, percepts, WINN (1982), as a previous stage for concepts, indeed, thinking activity from visual experiences.
The organizational power of image is best defined by WHITE, (1987) when she says that we have been trained to learn how the reality is through text. That is, from left to right and from the top to the bottom which is certainly poor if compared to the amount and quality of information that may be processed at a glance (spontaneous time-space distribuition is acquired), with a simple eye movement. Search and Recognition are always linear in sentential representation being more complex but flexible and integrative under visual treatment.
Inference is largely independent of representation (provided the rules are applied to the same contents). Search and Recognition are quite different in text and image representations. The balance of cost among Search, Recognition and Inference is mainly on the Recognition stage because it is the major efficiency difference between the sentential and the iconic (-diagrammatic in the original-) representation. A visual (iconic) system provides, at virtually zero cost, all kinds of perceptual inferences, LARKIN & SIMON, (1987).
WHAT A FULL CONTENT IMAGE IS:
A Full-Content Image is a graphical representation of the reality. This representation may both have analogical and logical correspondence with the reality where for educational purposes, Human, Social and Technical Primitives are present, either explicit or implicitly.
An Explicit Primitive denotes the reality through a graphical, symbolic or analogical representation. On the other hand, an Implicit Primitive is the one which arises from a deeper level of ” iconic reading” because it was never “written” onto the image. It has been connotated within the contents, being “pure meaning” but not the “word” itself.
It should not be confused what is not represented (because it simply does not exist) with what is implicitly represented. This means that some of the Primitives may not appear explicitly but they may have been actually taken into account under a different representation figure, that is, they were connotated into the image.
We can then say that an Explicit Primitive (Ep) is the one which holds a complete matching between the agent’s external and internal representation. On the other hand, an Implicit Primitive (Ip) is the one the agent can detect because of his semantical connections rather than by an analogical or somehow graphical correspondence with the reality.
WINN (1982) makes reference to concepts like these as follows: “…the correspondence between visual information and reality is not important but that the correspondence between the internal and external forms of representations…”
We define an Iconic Function as a set of Primitives (a Full-Content Image) semanticaly connected to produce a true-value representational entity which reflects both by analogy and logic icons either the real or formal world. Internal coherence is an essential requirement to satisfy the true-value condition.
As a consequence, it will be finally necessary to have a one-to-one correspondence among the Primitives of our function and the elements from reality. Indeed, the mutual correspondence between the Representational Entity with the real or formal world as a whole and the corresponding true-value relations among different parts of that picture. Otherwise, we would only have a full-content image but not an Iconic Function.
ATRIBUTES AND GENERAL FEATURES OF AN ICONIC FUNCTION: A qualitatiive point of view.
Every Primitive participating into an Iconic Function will always have some attributes and features which specificaly let the agent communicate. Some of these attributes and general features are, MANGIONE, (1989):
i.- Shapes: made up of lines, squares, circles, points as well as some other irregular or alternative elements;
ii.- Size: which may be represented by a pixel, a letter box or a full size screen. Size will depend on perceptual faculties especially on distance comprehension from the agent’s point of view, ARHEIM, (1985).
Sizes are highly dependent on the dimensions of the screen and more than five different sizes are not advisable. Shapes should be treated in addition to their nature of geometric or pictorical where the latter requires – for the same resolution-, as much as double amount of pixels for the desired final presentation, HEPP & ALVAREZ (1988).
iii.- Chromaticity: (depending on the technical attributes of the machinery) it will be of a big help for further applications in diagnosing and prescriptions.
As long as chromaticity is important, bright colors should be treated with caution since they provoke a persistence effect on visual perception mechanisms. Moreover, use of complementary colors should be well administrated since they drag to a vibration effect, HEPP & ALVAREZ (1988).
iv.- Text, Windows and Special Functions: (such as rotation, translation, scaling and some other functions to work on the shapes once we have got them done).
As in many other fields related to structured methodologies, we will based our Primitives on the concept of Macros as subsidiary procedures of Primitives. We will be calling Macros those Structured Graphical Objects with some or most of the attributes and features mentioned above. They will be easily called up by their names just as we do in structured programming.
Within this morphological approach, an Iconic Function may be specified as follows:
Iconic Function (I.F) = F(X,Y,Z … )
A Primitive for Human Component X = f(r, s, t, u)
A Primitive for Social Component Y = g(r, s, t, u)
A Primitive for Technical Component Z = h(r, s, t, u)
are the Global Variables or Primitives and,
r = shape
s = size
t = chromaticity
u = special functions
are the local variables of the problem.
As WHITE, (1987) has warned us about emerging problems of information overloading, HEPP; ALVAREZ, (1988) recommended not to use more than seven concepts on the same screen. Exceeding this limit may cause short term memory to fail easily.
IMAGERY AND THEORY OF INFORMATION: A quantitative approach.
We will present some comments related to a comparative study, matching both text and imagery information transmission power. It should be noticed that the survey does not either make a proposal for meassuring up the amount of information understood by the agent or what should be called “information”. We know that information varies from one subject to another but we are interested in the quantitative aspects of transmiting coded information so as in any Mass Media.
The quantitative approach we are talking about is the originally given by ABRAMSON, (1981). He made some Radio and Television comparisons in terms of capacity for information transmition. We now make some adaptations on this calculus to apply it to some text and images coded messages always into a computational enviroment, in this case, a simple computer.
Making some figures, we can say that the probability P of an E event to accur is P(E) so when it occurs we have transmited
I(E) = log (1/ P(E)) information units
Let’s suposse now that we are not talking about two different means like Abramson’s radio and television sets but about the same means as a computer video screen with minimal graphic attributes. Let’s also assume that we are going to analyse the amount of information this very same screen is able to transmit both when using sentential representation (text) and imagery.
Just to take a simple screen we say that this one has 800 x 600 pixels and has only the possibility of generating black and white images. We are putting some restrictions to facilitate the calculus and to show the reader that it is not even necessary to add chromaticity to prove that even in a restricted environment, imagery proves to be superior than text in its instant information transmition power.
Suposse that the screen generates 10 different shades of gray. Thus:
10^(800×600) = 10^(480.000)
different images may be generated. Assuming that all of them are equally probable, the probability of one of them is:
Now, let’s consider another point of view. Assume we have:
i.- A set of 3,000 available words to describe what an image holds;
ii.- The standard screen is 80 columns and 25 rows what yields 2,000 box-letter positions;
iii.- We will work with 5 letters long average words.
Thus, 2,000 boxes in the screen and a five-letter length for each word, we have 400 possible words on the screen except that we need some blanks to separate them. So, let’s assume we take off one (blank) space every five letters. This yields about 320 word on the screen to explain the content of our image.
Amount of information given by an image is:
I(E)Image = 800 * 600 * ((log 10)/(log 2)) = 1.536.000 bits
Amount of information given by text is:
I(E)text= 320 * (log 3,000/log 2) =3,696.93 bits
And the ratio between them is:
Finally we can say that, in terms of instant transmition information power, each of our images is around a 400 hundred words. In spite of the fact that authors have some doubts about Abramson’s calculus and phylosophical questions have risen, his calculus has proven to be good enough in our approach so as to technically depict the advantages of using imagery for information transmition.
AWARENESS, BELIEF AND KNOWLEDGE FOR IMAGERY
Besides the morphological features previously mentioned, it should be remember that there exists a whole variety of psychological mechanisms vinculating external and internal representation. Awareness, belief and knowledge phenomena will be revised so as to introduce some interesting concepts into imagery.
According to FAGIN & HALPERN, (1986), there is a kind of belief based upon the intuition about certain real or formal P fact. This is called Implicit Belief (L). On the other hand, there is another kind of belief called Explicit Belief (B), which is associated to Awareness (A). Thus we say that,
B = LxA
Maybe the easiest way to refer belief to awareness is the following. Let’s suposse we have an Iconic Function, say Fi. As we have just anticipated, every Iconic Function can be divided into Primitives. We then say that the agent may have Explicit Belief of the Fi function if he is Aware of every primitive in the function. Thus:
A(p1) ^ A(p2) ^ A(p3) ^ … A(pk) –> B(FI)
A –> the modal operator for Awareness;
B –> the modal operator for Belief;
p1,2,3,k –> any primitive of the function;
^ –> logic conjuction.
The expression says that an agent explicitly believes in a function FI (in the sense of accepting it, being sure about its contents or to assume it like possible) if only he is aware of all the primitives in that function.
This point is extremely important when we are referred to instructional and educational imagery. The use of imagery has certain advantages, according to PARK, (1985), WHITE, (1987), MANGIONE, (1989) which are fully complentary with FAGIN & HALPERN, (1986) approach in addition to detect and diagnose what an agent perceives (-is aware of-) and the way he does it: explicitly, implicitly or by an harmonic semantic combination. Socially speaking, it is also advisable to take into account De Rosa and Farr, (2001b), when they introducie -in terms of belief-, the following concept: ”the code of iconic representation can be considered as a specific and privileged means of expression of beliefs and of “irresistible” representations, deeply rooted in our social memory.”
VISION, PERCEPTS AND CONCEPTS:
ARHEIM, (1985) states that the cognitive operations (namely, thinking) are not a privilege of mental processes beyond perception but ingredients of perception itself. There is something else to say about this subject especially regarding to mental spaces and visual perception operations. WITTGENSTEIN, (1982), for example, remarked that we observe the organization of an object or better, we observe something related to its organization, perhaps a characteristic of its organization. This is obviously achieved in terms of two basic operations: abstraction and generalization, ARHEIM, (1985). What an agent perceives are global qualities rather than singular features.
Needless to say those two operations are closely related to each and every single subject. Perception of an object is not transferable from one subject to another although it is possible to transmit the impressions that an image has produced over a particular subject, paraphrasing WITTGENSTEIN, (1982). Besides that, visual perception, percepts construction and concept building depends on personal experience (individual source). The richer the visual experience (internal representation), the more focalized the subject’s attention on external representation aspects. At the same time, a pattern arises from common features emerging from many objects and the integration of their attributes.
This activity of integrating common features from n objects is conducted by two essential cognitive operations, namely, Generalization and Abstraction. The former operation may be defined as the one consisting in the detection of the transmitted patterns by an object. Should the sample involve many objects, generalization also implies synthesis. On the other hand, Abstraction is defined as the operation consisting in recognizing (extracting) the pattern emitted by the object from the context.
The ability of perceiving an object as immutable is to produce the highest possible generalization. Abstraction is activated by generalization for a further reconstruction of the reality.
SOME ICONIC FUNCTIONS READING
We have to analyse one example so as to make clearer the concept of implicit primitives, inside and outside the picture. How would we analyse a written piece of music?
First of all we have to say that:
i.- The piece of music has been coded to be perceived by the sight sense;
ii.- It has been meant to be uncoded by the same sense but to be checked up by the ear sense and performed mainly with the hands;
It will also be of a big help to admit that:
i.- It is not a typical picture but;
ii.- It is a text-like representation entity since it uses symbols (whole, half, quarters and so on);
iii.- The reading implies that this is not a simple description. It has a descriptive component but it also has a performative component too. That is where the agent interacts with the image.
We can now begin our analysis in terms of primitves and the way they contribute to the picture as a whole. It should be recognized that:
i.- There is an Explicit Technical primitive being this read as the formula that coded music is and the instrument(s) it has been coded to be played on;
ii.- There is an Outer Implicit Primitive, namely, the Human Factor since he is the only one able to do this;
iii.- The Social primitive is the most complicated to be found and seen. How would we know whether the piece belongs to a Barroco author or to the Romantic period or to any of the possible ones?
The question has at least two answers: the literate musician knows just by the reading of the piece and the execution confirms it. The iliterate may be missing this primitive unless he developed a sense to distinguish intuitively one period from another.
As the reader can see, this keeps a close relation between what we have just seen about belief, awareness and knowledge. Let’s see.
For the literate, not only he intuitively believes the music belongs to a social, musical or artistical movement but also he is aware of it because he perceives it by the reading and checks it by playing the piece. So he has an intuition and some experience of what he sees and plays and that is formally written in our modal logic as:
B1 Brand.6 = L1 Brand.6 ^ A1 Brand.6
which is read as: the agent 1 has an intuition of Brandenburg Concert number 6 and besides that, he is conscious (aware) of the same piece. This yields an explicit belief of the musical piece.
On the other hand, an iliterate with all his talents may implicitly believe in the piece of music because he only knows it by ear but would not match reading with performing criteria. So he is based upon an intuitive believe (L).
in this case the formula is simply read as follows: the agent 2 intuitively believes in the piece of music. This yields that our piece of music is a “true-value representation” for the literate but it is not for the ilitarate.
SUMMARY TO THE AIMS AND HYPOTHESES
Summarizing, we can say that every image can be observed, analysed and interpreted under the light of this simple first-approach method.
We then suggest that:
1.- Amount and kinds of Primitives must be determined in terms of either Explicit or Implicit components;
2.- The domain of Implicit Primitives must be carefuly studied in terms of Inner Implicit Primitives (Iip) and Outer Implicit Primitives (Oip).
3.- The character of the primitives must be clearly detected in terms of Descriptive (Ds) and Performative (Pf) components.
Applying this to the examples developed in this work we finally say that:
A set of paraboles are a Non Interactive Iconic Function where there is nothing to be performed. It is a description. It is a graphical representation of a mathematical formula. The only primitives participating in such a picture are inside of the territory of the representation entity.
A Warning Traffic Signal and/or the piece of music are Interactive Iconic Functions:
i.- They are interactive because they demand man participation to be performed. Otherwise they would not satisfy the true-value condition.
ii.-They are also a description because they have already been coded too;
iii.- Some of the primitives will be inside the domain of the picture and we call them Inner Implicit Primitives (school in the signal). Likewise, there may well be some other primitives outside the domain of the picture and those will be called Outer Implicit Primitives (man and car in the same traffic signal).
Let´s analyse the complete meaning of this Traffic Signal according to our methodology. It is on the power point figure. (Let´s read!)
This is not the only topic we should have into account. There is also necessary to notice that, there are three main component or Primitives as they are:
a.- A Human Primitive,
b.- A Social Primitive and,
c.- A Technical Pritive.
Let´s take a deep look at this ireading:
As it is clearly seen in this picture, Social factor has been depicted.
And completng the other tow components, we finally get the Warning Traffic Signal!
II.-SOME ANSWERS TO THE QUESTIONS
WHOLE-PART SEMANTIC RELATIONS FOR SENSE-MAKING PICTURES
An Iconic Function would not be made just upon a simple aesthetic combination of components. It is also necessary a semantic foundation to support everything that may be constructed from Macros and Primitives integration. Otherwise, we run the risk of producing what was best called by THIOLLENT, (1989) “a sort of bricolage”.
We need some linking relations which are, in this case, a branch of the general semantic relations called Meronimic Relations. According to WINSTON, CHAFFIN & HERMANN, (1987), “mero” comes from a greek word meaning “part of”.
The consistency of Meronimnic Relations is of a definite relevance for both deductive and inductive iconic inference. Nobody would be able to produce meaningful pictures and images unless meronimic relations were actively contributing to achieve this goal. Within the piece of music or into a set of paraboles, there are semantic relations supporting the final coherence of the work.
As we can easily see from this taxonomy that there are six kinds of meronimic relations all of them of a great value if applied correctly. We will just mention some of the guidelines to apply the taxonomy to the graphic domain and what is the way they cooperate in a sense-making picture (namely, an Iconic Function).
i.- First of all it should be considered the fact that we have based our proposal on Elements, Macros and Primitives participation so that meronimic relations will be working in each of these levels and among them;
ii.- Besides that, it will also be of a big help if we kept in mind that a Full-Content- Image-Based Interface would exceptionally be based upon only one image so that meronimic relations will be linking not only the parts of a “graphic frame” but also, different frames so as to reach a global meaning (just as it is necessary to have sequences of images semanticaly linked).
iii.- Inference (either inductive or deductive) will completely depend on these semantic relations. Otherwise there will not be any possible Iconic Inference.
ARE ICONIC DIALOGS POSSIBLE?
They may get achievable in many ways. We propose three different ways of developing Iconic Dialogs: The first one which we call “a Whole-based” Iconic Dialog; the second, “the Part-based” Iconic Dialog and the “Natural-Language-Primitive-based” Iconic Dialog.
We will explain each of these alternatives since either one can produce the desire communication effect with the agent. Regardless the kind of Iconic Dialog we are interested in, Meronimic Relations are simply crucial to achieve any satisfactory iconic interaction.
The Whole-based Iconic Dialog
When we first presented the concept of Iconic Function, we said that it is the representational entity which completely reflects the real or formal world under study (i.e. the entity that holds true value).
What we are going to do here is simply to propose a way of semantic work which consists in dismounting the entity according to one or many of the modalities listed below. We will simply go from the Whole to the Part splitting up the Continent meaning as a whole, all the way down to the content meaning of every single primitve, macro or element. It is easy to see that this iconographic entity may well be semanticaly divided according to the following criteria:
II.1.1.- “Split up” according to the primitives: which are the dominant components in this approach;
II.1.2.- “Split up” according to the macros: currently components of the primitives as subsidiary procedures;
II.1.3.- “Split up” according to basic elements: meaning the maximum possible dismounting level.
II.1.4.- “Split up” according to similarities (or differences).
The Part-based Iconic Dialog
Related to the previous one, this is exactly the opposite way of establishing an Iconic Dialog. We will go from content meanings to continent meanings. We will develop a semantical integration activity all the way up to the global meaning of the scene itself i.e., we start up from the part being the Whole our final goal.
It is easy to see that the same kind of activities may be developed from this point of view only with a change in the kind of operation. Instead of “splitting up”, the logical operation will be now to “make up” different portions of the entity to finally get to the whole.
The Natural-Language-Primitive-based Iconic Dialog
We know man is not very used to establish an Iconic Dialog, mainly if he has got to “shoot” the first “word”. Remember this: word is not exactly written into a sentential representation. It is imagery itself. This is why we propose some alternative ways of facing this new problem according to human possibilities and at a low cost in terms of cognitive resources versus expected outcomes.
Creating an Iconic Dialog starting from the primitives of natural language is feasible under one of these three modalities:
II.3.1.- User-guided dialogs: which clearly speaks about the initiave of the agent to establish the iconic communication;
II.3.2.- Machine-guided Dialog: in which the beginnig of the dialog relies on the machine attributes to develop these activities;
II.3.3.- Man-Machine coordinated dialogs: where either the agent or the machine can establish and direct the whole course of the communication.
Considering the normal effort that Iconic Dialog may require from the agent, he will sometimes be allowed to use sentential language, but the method requires either the system or the machine or an heuristic environment to answer by icons. The agent may always have the chance to choose one of the two modalities which are merely proposed so as to facilitate this otherwise, hard task:
i.- The agent will feed the system or environment with words referring just nouns (or objects) to what he will receive either an explicit or implicit iconic primitive;
ii.- The agent will feed the system or the environment only with verbs, infinitive or conjugated but just verbs.
Both ways are meant to lead us to work in the frontier of iconic-formal-propositional representations. It should be noticed that the level of abstraction used in this image-word interactive dialogs is especially applicable to people with complete domain of the oral and writen (sentential) language. For children, it will have to be simplified so as to produce simple iconic dialogs based upon the restricted set of iconic primitives children know and their also restricted set of natural language primitives.
The ratio between the amount of recognizable set of images and words will have to be strongly marked in favor of the images since a (sentential) language matches a single code (word) with a single concept. On the other hand, there are not two identical pictures to represent the same concept ARHEIM, (1985); (there is one word to the concept of “lake” but there are thousands of pictures of lakes and they represent the same concept in different ways and with a whole variety of attributes).
It should also be remembered that an iconic representation (diagrammatic representation in LARKIN SIMON, (1987)), preserves explicitly the information about the topological and geometrical relations among the components of the problem, while the sentential representation preserves other kinds of relations like temporal or logic sequence. Iconic primitives meaning topological and geometric properties would not be the right answers to temporal or logic questions unless a thorough analysis is made by an interdisciplinary team; phsychologists, Visual communication experts, analysts, knowledge engineers should participate coordinately to keep expectations of success.
As in any other interactive way of communication, there must be an answer from the machine for a dialog to take place. This is why we are going to need some equivalences and substitutions so as to let the agent and the machine interchange icons, figures, pictures, indeed, meanings. This is why we propose a list of possible kinds of substitutions in addition to facilitate fluent man-machine interchange. This is what into a sentential representation is called synonyms and paraphrasing.
ICONIC SUBSTITUTIONS FOR ICONIC INFERENCE
There would not be any possible communication process unless we have a repertoire of images, namely icons, diagrams, figures, and the likes to produce a permanent and fluent interchange. This repertoire will frequently need synonyms and alternative figures to the same “word”. This is why we propose the use of a taxonomy for substitutions which will be developed as follows.
When sentential representation is used -as LARKIN & SIMON, (1987), likes to call text-, paraphrasing may well be one of the possible solutions to avoid ambiguity; but, what about working into an imagery environment? Do we have some techniques and tools to produce a similar effect into imagery?
This is a part of our proposal and it is based upon a taxonomy from DURAN, (1973) which accurately adjusted to the approach, facilitates Iconic Dialogs seeking Iconic Inferences.
Duran’s taxonomy holds four kinds of susbstitutions:
i.- Identical element substitution: from which, everybody knows at least, “emphasis”, (other than hyperbole and litote). See examples in DURAN, (1973), MANGIONE, (1989);
ii.- Similar element substitution: divided in “formal and content substitutions”;
iii.- Different element substitution: which may be one of the best suited to work with Inner and Outer Implicit Primitives. Different element substitution is divided into:
a.- Cause by effect substitution;
b.- Effect by cause substitution;
c.- Object by destiny substitution;
d.- Whole by part substitution.
iv.- Opposite element substitution.
If the question, “What are these substitutions good for?” is still in the reader’s mind, the answer is another question: How does the reader think the agent could ever produce an inference if he gets stucked up at the first or second image he is exposed to? Would he ever be able to understand every line of the emerging Iconic Dialog? Of course, not. The iconographic treatment deserves as many chances of well developed and meaningfull communication just as it can be performed through text. Synonyms, paraphrasing and some other usefull figures must also be translated and materialized into any iconographic approach.
To make these substitutions possible we have to make clear that there will be necessary some criteria in terms of equivalences between what is being substituted and the substituting primitive. We then say that:
i.- Two representations are informationally equivalent if all the information in the one is also inferable from the other and viceversa. Each could be constructed from the information in the other, LARKIN & SIMON, (1987);
ii.- Two representations are computationally equivalent if they are informationally equivalent and, in addition, any inference that can be drawn easily and quickly from the information given explicitly in the one, can also be drawn easily and quickly from the information given explicitly in the other and viceversa, LARKIN & SIMON, (1987).
A VISUAL EVENTS SUBSIDIARY TAXONOMY
COBUN, (1975) contributed to this work with his taxonomy for visual events. No matter how many substitutions there must be into an iconic communication process untill the agent gets the right meaning. What is really important is to lead him straight forward to where the concepts are.
In addition to produce very organized and highly coherent substitutions, we should also know that there are several kinds of visual events. Whatever we do when substituting primitives, macros and elements in the picture, we have to be aware that we will sometimes produce a change in the “kind of event” the agent is exposed to. Remember that there are events sharply related to the reality and on the other hand, there exists a rich variety of events clearly belonging to fantasy or figurative domains.
Any time you lead your agent’s attention to a special concept, make sure that (you are and that) he is aware he might be conducted through some alternative paths which are not strictly related to the reality and to the nature of the world under study.
We summarize Cobun’s taxonomy as follows:
i.- Documentary event: which is recorded as it occurs;
ii.- Reconstruction event: that one that could not get registered as it happened;
iii.- Contrived event: especially devised to show the agent how things would be “if…” some conditions took place;
iv.- Logical situation event: produced to show the agent predicted outcomes;
v.- Inductive situaton event: a typical “what if…” event;
vi.- Deductive situation event: especially devised to give the agent some elements for problem-solving activities;
vii.- Open-ended situation event: the one with a non deterministic solution;
viii.- Demonstration event: the one which has been designed to show a process step by step;
ix.- Expository event: ideally applicable to show from beginning to conclusions any event with closure;
x.- Participatory event: an essentially interactive event devised to present objective-oriented contents;
xi, xii, xiii.- Theatrical, Fantasy and Animated events: very apropriated in children applications;
xiv.- Data event: mainly related to statistics;
xv.- Programmed event: which may well involve most of the just mentioned events.
Under the light of this classification it is advisable before doing anything else to:
i.- Classify every scene within the iconic sequence according to the event it is involved in;
ii.- Make some “chapter-like” array (a semantic array in our approach) with the images within the sequence;
iii.- Give the whole sequence of images the name of the general characteristic event it is involved in.
These general hints will certainly make clear how the taxonomy is applied to a well organized iconographic treatment. Nevertheless, it should be remembered that according to De Rosa and Farr, (2001b); the construction of a theory of imagery categorization could well be as follows:
- “as a source able to activate social representations or favour the development of new social representations ;
- as a product of social representations, i.e. an iconic-symbolic synthesis, a condensed materialisation of one social representation, a direct expression of the objectification process, in other terms as itself as a social representation ;
- as a medium, a specific form of transmission, linked to differentiated channels (traditionally the visual arts, but also television, cinema, photography, new audio-visual technologies, internet etc.), through which new or pre-existing social representations are conveyed.”
ALGORITHMIC TREATMENT OR HEURISTIC APPROACH?
We have already developed both an algorithmic and an heuristic approach in MANGIONE, (1989). We will simply say that either one will hold an Image-based proposal for a First-Approach-Method for Image Reading and Writing as well as to develop Full-Content- Image-Based Interfaces.
DWYER, (1980) defines algorithmic approaches as being pre- determined sequences to achieve some goals. The algorithmic approach suggests to begin by the output specifications in terms of objectives to further devise the procedures to get to the desired outputs. An algorithmic procedure offers the possibility to give the problem structure and precision, contributing to encapsulate that precision into a reproduceable way. An algorithmic modality will always produce the same outputs when activated by the same inputs under the same process.
On the other hand, there is the chance of promoting an iconic heuristic environment. This would be somehow opposite to the behaviorist-fashioned-algorithmic approach. The basic support of this modality are the Cognitive Psychology and the heuristics.
i.- Heuristics favor an open-ended modality where microworlds are explored through some procedures meant to let the agent conduct the whole session.
ii.- Heuristics work locally since when there is a good one to solve a problem within certain domain, it temporarily loses the sense of the integrity it is involved in. Nevertheless, heuristics tend to satisfy the objectives for a global result.
iii.- Heuristics imply several procedures where the final state of the first procedure is the initial state of the second one and so on. Basicaly, heuristics do not say what to do but how we could do something.
FROM GRAPHIC MODAL OPERATORS TO FULL-CONTENT IMAGES.
We have developed a set of modal logic operators to give the approach the same formal treatment as we did with awareness, belief and knowledge. We gave them the name of Graphic Modal Operators. We have created this set of operators which are divided into categories closely related to what is expected to be the reality of human being levels for perceiving tasks and cognitive operations, such as: BONNET, (1988); LIVET, (1988);
i.- A Sensory level, which gives the first sketch of a visual activity; it is considered as being part of the stimuli itself;
ii.- A Perceptive level, where different configurations take place through structured representations;
iii.- A Semantic-Cognitive level, which deals with intermodal aspects catalyzing processes coming from the previous levels. It is in this level where modelling and meanings happen to take place.
In order to create an apropriate environment to develop our Graphic Modal Operators we have divided them into three categories such as,
i.- Psychologic and Perceptual level, with the Search (Sh), Visualize (V)‚ and Perceive (P)‚ operators where:
* Search is the visual inspection within the search space seeking structural elements of the picture;
* Visualize indicates the agent is able to localize the elements in the search space;
* Perceive indicates the agent is aware of what is in the search space.
ii.- Cognitive Level, which holds the Recognize (R), Abstraction (Ab), Generalization (G)‚ and Inference (I)‚ operators;
* Recognize indicates the coincidence between internal and external representation;
* Abstraction indicates the agent is able to extract the object from the context;
* Generalization implies the capabilities of the agent to extract the pattern from an object (class);
* Inference indicates the agent arrives to a conclusion having started out from one or more premises.
iii.- Instrumental level, which is also divided in:
a.- Graphic: with the Union (U)‚ and Division (D)‚ operators
* Union‚ says the agent is able to produce graphic-logical union;
* Division‚ means an action completely opposite to the previous one;
b.- Semantic: with the Makeup (M)‚ and the Splitup (Sp)‚ operators.
* Make Up‚ implies the capability to semanticaly integrate parts into a whole;
* Split Up‚ implies capabilities exactly opposite to the previous one.
SUMMARY AND CONCLUSIONS
The proposal we presented involves what we consider partially an answer to the emerging demands for Imagery Literacy. Educational, technical, psychological and social aspects were taken into account to develop this communication where semiotic, semantics and graphic representation participate in an integrative effort to conduct the human understanding to complementary and richer methods for a better comprehension of reality.
Man-to-man and man-machine communication processes were treated as been transparent considering either one like “the agent”. Obviously the difference is based on the quality and domain of application of this approach.
We presented in this work the concepts of Full-Content Images and Iconic Functions. We also promoted Iconic Dialogs to reinforce the Iconic Inferences. Within this approach, there are three useful taxonomies which let us produce this quantitative and qualitative graphic information interchange between men and for man-machine communication.
Artificial Intelligence, Ergonomics and Cognitive Sciences are plenty of methods, techniques and tools to produce consistent applications of this approach either for training and education topics as well as in knowledge engineering or in the emerging area of graphic interfaces.
The main contributions of the proposal are:
i.- The setting of the existence of Inner and Outer Explicit and Implicit Primitives;
ii.- Modular and Graphic-Object-Oriented Macros;
iii.- The mechanic for Iconic Dialogs and some hints for evaluating and diagnosing tasks closely related to visual thinking and perception activities.
We keep our hopes up that, in the same way we read and understood a whole lot of images in our efforts to bring light over this topics, the method is effective and the reader will be able not only to apply it on his own but it he will also be able to produce satisfactory applications in systematic and systemic training, education tasks and in the design of either conventional or intelligent man-machine Full-Content-Image-based Interfaces.
Mendoza, 1 de diciembre de 2018. 19:38´
ABRAMSON, N.(1981); Teoria de la Informaci’on y codificaci’on, Ed. Paraninfo.
ANDERSON,J.R. et alii; (1984);Cognitive Principles in the Design of Computer Tutors. Proceedings of the Sixth Anual Conference of the Cognitive Science Society. Boulder, Colorado.
ARHEIM, R.; (1985); El Pensamiento Visual; Eudeba.
BOLT, R. A. (1979); Spatial Data Management. Cambridge Massachusetts, M.I.T., 1979, apud HAWKRIDGE; D., New information technology and education. Johns Hopkins University Press.
BONNER, J. (1987); Computer courseware, frame-based or intelligent. Educational Technology,
BONNET, C. (1988); La Perception Visuelle des formes et du movement. Intellectica, 5-88-1; p.57.
BORK, A. (1980); Learning through graphics; The computer in the school: tutor, tool, tutee; Taylor, R. (Ed.), Teachers College.
BUDIN, H; TAYLOR,R; KENDAL, D (1987); Las computadoras y los estudios sociales. Rev. Informática Educativa, pp. 7.
BUNT, H; Mass Terms and Model-theoretic semantics.; Cambridge University Press, 1985.
CERRI, S (1985); The intersection of A.I. and Education: Investing in tough-intensive endeavors; TRAPPL, R. Impacts of Artificial Intelligence, North-Holland., T.C. (1975); Target: Rational Design; Instructional Media and Technology (Ed.) Sleeman, P & Rockwell, D.M., Halsted Press; John Wiley and Sons.
COBUM, T.C. (1975); Target: Rational Design; Instructional Media and Technology‚ (Ed.) Sleeman, P & Rockwell, D.M., Halsted Press; John Wiley and Sons.
CURRAN, S; CURNOW, R. (1984); Juegos, Imágenes e sonidos; ed. Gustavo Gili S.A., Barcelona.
CYRS, T.E. (1975); Modular Approach to Curriculum Design using the System Approach; Instructional Media and Technology (Ed.) Sleeman, P & Rockwell, D.M., Halsted Press; John Wiley and Sons.
DE ROSA, A.S., FARR, R. 2001b Icon and symbol: Two sides of the coin in the Investigation of Social Representations. In BUSCHINI F. & KALAMPALIKIS N. (eds.) Penser la vie, le social, la nature. Mélanges en hommage à Serge Moscovici. Les Editions de la Maison des Sciences de l¹Homme, Paris: 237-256.)
DURAN, J. (1973); Retórica Imagem Publicitaria; Analise das imagens. Vozes.
DWYER, T. (1980); Heuristic Strategies for using computers to enrich education, TAYLOR, R (Ed.); The Computer in the School: tutor, tool, tutee.Teachers College, Columbia University.
FAGIN, R; HALPERN, J. (1986); Belief, Awarness and Limited Reasoning: Preliminary Report; IBM. San José, California.
GOLDBERG, A; (1984) Smalltalk – 80 The Interactive Programming Environment, Xerox, Palo Alto Research Center; ADDISON- WESLEY.
HAWKRIDGE; D., (1983); New information technology in education. The Johns Hopkins University Press.
HEPP, P; ALVAREZ, C; (1989); Diseño de Interfaces Hombre-Computador con Multimedios. Anales IX Conferencia de la Sociedad Chilena de Ciencia de Computación y XV Conferencia Latinoamericana de Informática. Santiago, julio 10, 14.
HOFMEISTER, A. (1984); Microcomputer Application in the classroom CBS College Publishing.
KRATHWOHL, D; BLOOM, B; MASIA, B. (1970); Taxonomy of educational objectives; David McKay Co. Inc., New York.
LARKIN, J.H.; (1987); SIMON, H.A.; (1987); Why a diagram is (sometimes) worth ten thousand words. Cognitive Sciences, 11, 65-69.
LEGGE, D; BARBER, P. J. (1977); Informaçao e Habilidade; Organiz. de Herriot, P; City University, Londres; Zahar Editores.
LIVET, P. (1988); Image, Perception e Representation.; Intellectica, 5-88-1, p. 33.
MALONE, T.W. (1981); Toward a Theory of Intrinsically Motivation Instruction; Cognitive Science, 1981, 5, 130-145, apud PARK, O; SEIDEL, R.J.; Conventional CBI Versus Intelligent CAI: Development for the Future Systems.Educational Technology,, 15-21.
MANGIONE, J.C. (1984); Primer Curso de Inform’atica Educativa; Mimeografado, Escuela de Comercio Mart’in Zapata, Universidad Nacional de Cuyo.
MANGIONE, J.C. (1985); Capacitación de docentes en el nivel medio.Primeras Jornadas de la Computadora en la Educaci’on; CRICYT-ME, Mendoza, Argentina.
MANGIONE, J.C. (1986); Informática Educativa: Una Experiencia y una perspectiva. Primer Congreso Federal de Informática Educativa, Santa Fe, Argentina, Julio de 1986. Anales del Primer Congreso Federal de Informática Educativa; COFEIN.
MANGIONE, J.C. (1986a); Educaci’on Vial como factor de mejora de la conducta social. Primer Simposio Argentino de Seguridad y Educación Vial.
MANGIONE, J.C. (1986b); El computador como recursos didáctico. Primera Feria Local de Informática, Mimeo. Mendoza, Argentina.
MANGIONE, J.C.; OSES, M.R. (1986); Educación Vial y el replanteo de las normas: sustituci’on de la prohibición por la persuasi’on. Primer Simposio Argentino de seguridad y Educaci’on Vial.
MANGIONE, J.C.; VALDEMOROS, M.P. (1986); Informática y Educación Vial; Primer Simposio Argentino de Educación y Seguridad Vial, Buenos Aires, Argentina.
MANGIONE, J.C.; PRILUSKY, J. (1987); Redes computarizadasen la educación.; Rev. Informática Educativa., Setiembre, N6.
MANGIONE, J.C. (1988); Informática e Educaçao Nao-Convencional; Boletín de Informática rcii – UNESCO, Año 6, Suplemento 1.
MANGIONE, J.C. (1988a); Tendencias da comunicaçao em Linguagem Natural no Contexto Tecnológico, 40a Reuniao Anual da Sociedade Brasileira para o Progresso da Ciencia.
MANGIONE, J.C. (1988b); A imagem como meio cognitivo, a informática como recurso didático. Primeiro Congresso Internacional Imagem, Tecnologia e Educa,cao; Nucleo de Produçao da UFRJ.
MANGIONE, J.C. (1988c); Uma abordagem geral sobre crença e conhecimento no contexto social; VIII ENEGEP, Sao Carlos, p. 394-398.
MANGIONE, J.C. (1988d); Uma abordagem geral sobre crença e conhecimento no contexto social; XVII Jornadas Argentinas de Informática e Investigación Operativa. (Versao completa), p. 794-814.
MANGIONE, J.C., (1989); An image-based method to develop educational software; Master’s Dissertation, Federal University of Rio de Janeiro.
MANGIONE, J.C.; (1989a); Etica para la educaci’on del ciudadano en la sociedad informatizada. Primer Simposio Internacional de Etica para Situación Contemporánea. Universidad de Buenos Aires. Septiembre 4-8.
MANGIONE, J.C.; (1989b); Transferencia y Adaptación de Tecnolog’ía: una visión desde la Gestión de Recursos Humanos. Primer Simposio Internacional de Etica para Situación Contemporánea. Universidad de Buenos Aires. Septiembre 4-8.
MANGIONE, J.C.; CLEIMAN, D.F.; (1989) Redes de Computadoras y Procesos Cognitivos. Tercer Congreso Federal de Informática Educativa. Mendoza, Septiembre 4-6.
MANGIONE, J.C.; CLEIMAN, D.F.; THIOLLENT, M.J – M; (1989); Transferencia, Adaptaçao e Nacionalizaçao de Tecnología Educacional. IX Conferencia de la Sociedad Chilena de Ciencia de la Computaci’on y XV Conferencia Latinoamericana de Informática. Santiago de Chile, 10- 14 de julio.
MANGIONE, J.C.; CLEIMAN, D.F.; THIOLLENT, M.J-M.; (1989a); O computador na escola: a questao administrativa, a questao pedagógica. Primeras Jornadas de Informática Educativa. San Luis, Noviembre 24-26..
MANGIONE, J.C.; CLEIMAN, D.F.; THIOLLENT, M.J-M.; (1989b); video-jogos e programas educativos: análise e comparaçao. Primeras Jornadas de Informática Educativa. San Luis, Noviembre 24-26..
MANGIONE, J.C.; CLEIMAN, D.F.; THIOLLENT, M.J-M.; (1989c); Fundamentos Psicológicos, Comunicaçao Visual e Tecnologia Informática: Uma abordagem morfológica da Comunicaçao Visual homem-computador. Primeras Jornadas de Informática Educativa. San Luis, Noviembre 24-26..
MANGIONE, J.C.; CLEIMAN, D.F.; THIOLLENT, M.J-M.; (1989d); Inteligencia Artificial e Informática Educativa: Interseçao e Integraçao. Primeras Jornadas de Informática Educativa. San Luis, Noviembre 24-26..
MANGIONE, J.C.; (1989c);Tecnología, Universidad y el Autor Latinoamericano. Jornadas del Pensamiento Latinoamericano, Univ. Nacional de Cuyo, Mendoza, Noviembre 16, 18.
METZ. C. (1973); Além da analogía, a imagem; Análise das imagens, Vozes, p. 7-18.
NORMAN, D. (1987); Qué pasa en la mente del alumno?; Revista Informática Educativa, Año 2, n’umero 4.
ORTEGA, G.G. de; MANGIONE, J. C. (1985); La Computadora en la Escuela Secundaria; Revista Estrada, N 23, Año 6, Setiembre, p. 25-27.
PARK, C. S. (1985); Interactive microcomputer graphics; Addison Wesley.
PARK, O.; SEIDEL, R.J. (1987); Conventional CBI Versus Intelligent CAI: Development for the Future Systems. Educational Technology, 15-21.
PENINOU, G. (1973); Física e Metafísica da Imagem Publicitária; Análise das Imagens, Vozes, 60-81.
PRIETO CASTILLO, D; (1988); El Derecho a la Imaginaci’on. Comunicación. Ed. Paulinas. N8.
RICH, E. (1985); Natural Language Understanding: How Natural can it be?; Charles Weisbin (Editor); A.I. Applications. The engineering of knowledge-based system. Proceedings of the Second Conference. Miami Beach. Florida; IEEE press, North-Holland.
SCHMUCLER, H. (1985); La educaci’on en la sociedad informatizada; La era teleinformática; Rodriguez,G; Comp. Ed. Folios.
SLEEMAN, D; BROWN,J.S. (1982); Intelligent Tutoring Systems. Academic Press, 1982, apud DE OLIVEIRA, U. S. de (1988) Um Sistema Tutorial Inteligente; Tese de Mestrado, I.M.E., Rio de Janeiro.
TAYLOR, R (Ed.) (1980); The Computer in the School: tutor, tool, tutee. Teachers College, Columbia University.
TENNYSON, R.D. (1987); FERRARA, J., Introduction to special issue in Artificial Intelligence in Education, Educational Technology, p. 7-8.
TENNYSON, D.T. (1987); MAIS: an educational alternative of ICAI; Educational Technology.
THIOLLENT, M. J-M; (1989); Personal Communication.
TRAPPL, R. (1985); Impacts of Artificial Intelligence, (Ed.), North-Holland.
VAIDEANU, G; (1987); La interdisciplinariedad en la enseñanza: ensayo de síntesis. Perspectivas (UNESCO), N.64.
WHITE, M.A. (Ed.) (1987); What curriculum for the information age?, Teachers College, Columbia University; Lawrence Erlbaum Associates Publishers.
WINN, W.; (1982); Visualization in learning and instruction: A cognitive approach. FCT, Vol 30, N1, p 3/25. spring.
WINSTON, M.E.; CHAFFIN, R. e HERRMANN, D. (1987); A taxonomy of Part-Whole Relations. Cognitive Sciences, 11 p. 417-444.
WITTGENSTEIN, L. (1982); Last Writings on The Philosophy of Psychology, Vol. 1; Wright & Nyman (Ed.), The University of Chicago Press.
WITTGENSTEIN, L. (1979); Notebooks 1914 – 1916; Wright & Anscombe, (Eds.), The University of Chicago Press.
WITTGENSTEIN, L. (1981); Tractatus Logico-Philosophicus; Routledge & Kegan Paul LTD.
WOHZ, A.; CHAVA, E.T.; (1988); Learning to view view to learning: A macro and micro integrative teaching system of T.V. viewer’s. Tech Trends, January.