Graphics and Languages
For the Flexible Modular Framework
John F. Sowa
21 July 2004
http://www.jfsowa.com/talks/iccs04.htm
Human-Computer Interface
Than are dreamt of in your philosophy.
Different applications have different requirements.
Different people have different languages, cultures, preferences, and habits.
Different hardware devices and physical environments have different features and limitations.
Some points to note:
- Everybody knows at least one natural language.
- Most people have five senses, and different applications use different subsets of them.
- Many applications use special-purpose graphics, which are highly specialized for the purpose.
The HC interface should take advantage of human abilities, respect
human limitations, and accommodate any application requirements.
The Problem
How can you design a truly semantic system?
How does the semantics affect the human interface?
How do you provide semantic support for everything?
- Languages, graphics, distributed agents, databases, World Wide Web, software development, servers, desktops, handhelds, phones, games, TV, embedded systems,
- Trillions of dollars of existing hardware and software,
- And billions of existing people.
Semantic Web
Proposal by Tim Berners-Lee:
- Goal: Semantic foundation for the World Wide Web.
- Starting point: Syntax.
- But no guidelines for the upper levels, especially proof and trust.
Elephant 2000
I meant what I said, and I said what I meant.
An elephant's faithful, one hundred per cent.
Moreover,
An elephant never forgets.
Proposal by John McCarthy:
- Goal: Natural communication between humans and computers.
- Starting point: Logic, intentionality, and speech acts:
Assertion, question, judgment, request, response, permission, promise, commitment, accomplishment, verification. evaluation.
- Proof and trust are directly related to speech acts:
Some Observations
- Semantic Web has provided many useful components.
- Elephant 2000 goes further:
It shows how logic-based components can support meaningful communication.
- But more work is needed:
- How do the components communicate?
- How do they find other components to communicate with?
- How can the new components communicate and interact with legacy systems?
- How can people communicate with all of the above?
Flexible Modular Framework (FMF)
- An architecture for intelligent systems proposed by Sowa (2002).
- Based on message passing among interacting agents.
- Elephant 2000 as the agent language.
- Logic expressed in CGs, KIF, XML, CLCE, OWL, RDF...
- Linda blackboards for associative addressing.
- Logic-based wrappers for legacy systems.
- Graphics and controlled natural language(s) for the human interface.
Different Interfaces for Different People
VivoMind Software Layers
Distributed Agents
Agents are supported by the IX features of Prologix:
- Multithreading. Each agent runs one or more threads on one or more CPUs.
- Message passing. Agents communicate by passing messages among themselves or among their own internal components.
- Direct communication. Agents that know each others' identity can send messages directly to the addressee.
- Associative communication. Agents that need some service, but don't know the service provider, can send associative messages through a Linda blackboard.
- Multiple Linda blackboards. Support associative message passing within an agent, among agents, or across the Internet.
- Uniform protocols. Same interface to local agents and agents distributed across the Internet.
Linda
A message passing protocol designed by David Gelernter.
Messages are represented as tuples:
- Data. A data tuple represents a fact that is being asserted, a value that has been computed, or a response to some earlier message.
- Patterns. A pattern tuple has a variable marked with ? in zero or more of the slots in the tuple. When a process requests a data tuple from BB, it provides a pattern tuple. If the pattern matches some data tuple currently in BB, the variables in the pattern tuple are replaced with values from the corresponding slots in the data tuple.
- Executables. An executable tuple differs from a data tuple by containing an expression in one or more slots. For each expression, a new process is started asynchronously to evaluate the expression before the result is placed in BB.
In effect, each kind of tuple represents a kind of speech act.
Linda Blackboards
Six operators for sending and receiving messages to/from blackboard:
- Out. Sends a data tuple to BB.
- In. Sends a pattern tuple to BB and waits to receive a matching data tuple from BB.
- Read. Like In, but when a matching tuple is found in BB, a copy is sent to the requesting process, and the original remains in BB.
- Eval. Evaluates the expressions of an executable tuple and sends the resulting tuple to BB.
- InP. Like In, but if no matching tuple is found in BB, it sends a null tuple to the requesting process instead of making it wait.
- ReadP. Like InP, but if a match is found, it sends a copy of the matching data tuple to Response and leaves the original data tuple in BB.
FMF Tuples
FMF messages consist of Linda tuples with six components:
- Source. A character string that identifies the sender.
- Message Id. A character string generated by the sender.
- Destination. A character string that identifies the intended receiver, if known. For associative communications, this string is empty.
- Speech Act. A character string that states the speech act.
- Language. A character string that specifies the language, such as "KIF", "CGIF", "English", or "Deutsch".
- Message. An arbitrary expression in the specified language that states the propositional content.
Speech Acts
Austin's five categories of speech acts:
- Verdictives "are typified by the giving of a verdict,
as the name implies, by a jury, arbitrator, or umpire."
Examples: acquit, convict, calculate, estimate, measure, assess, characterize, diagnose.
- Exercitives "are the exercising of powers, rights,
or influence."
Examples: appoint, demote, excommunicate, command, direct, bequeath, claim, pardon, countermand, veto, dedicate.
- Commissives "are typified by promising or otherwise
undertaking."
Examples: promise, contract, undertake, intend, plan, propose, contemplate, engage, vow, consent, champion, oppose.
- Behabitives "are a very miscellaneous group, and
have to do with attitudes and social behavior."
Examples: apologize, thank, deplore, congratulate, welcome, bless, curse, defy, challenge.
- Expositives "make plain how our utterances fit into
the course of an argument or conversation, how we are using words,
or, in general, are expository."
Examples: affirm, deny, state, assert, ask, identify, remark, mention, inform, answer, repudiate, recognize, define, postulate, illustrate, explain, argue, correct, revise, tell, report, interpret.
Controlled Natural Languages
A stylized version of a natural language with syntactic and semantic restrictions for some special purpose.
Aristotle designed the world's first version of controlled NL
for expressing syllogisms.
| Type | Name | Pattern |
|---|---|---|
| A | Universal affirmative | Every body is a material substance. |
| I | Particular affirmative | Some body is animate. |
| E | Universal negative | No mineral is animate. |
| O | Particular negative | Some body is not animate. |
Aristotle's version of CNL was the most widely used formal logic for over two thousand years.
And it's much better than many notations in use today.
Common Logic Controlled English
Examples of CLCE translated to predicate calculus and Conceptual Graph Interchange Form (CGIF):
- CLCE: If some person x is the mother of a person y,
then y is a child of x.
PC: ~(∃x:Person)(∃y:Child) (Mother(x,y) ∧ ~Child(y,x)) CGIF: [If: (Mother [Person *x] [Child *y]) [Then: (Child ?y ?x)]] - CLCE: Every ancestor of a person x is either
a parent of x or a parent of an ancestor of x.
PC: (∀y)~(∃x:Person)(Ancestor(y,x) ∧ ~(Parent(y,x) ∨ (∃z)(Parent(y,z) ∧ Ancestor(z,x)))) CGIF: [(Ancestor [@lambda] [Person:*x]): @every*y] [Either: [Or: (Parent ?y ?x)] [Or: (Parent ?y [*z]) (Ancestor ?z ?x)]]
Conceptual Graph Display Form
CLCE: Mary gives a dog a very juicy steak from a hot pan.
PC: (∃x:Dog)(∃y:Steak)(∃z:Give)
(∃w1:Juicy)(∃w2:Pan)(∃w3:Hot)
(Person(Mary) ∧ Agnt(z,Mary) ∧ Rcpt(z,x)
∧ Thme(z,y) ∧ Attr(y,w1) ∧ Very(w1)
∧ Srce(z,w2) ∧ Attr(w2,w3))
CGIF: [Give *z] (Agnt ?z [Person: Mary]) (Rcpt ?z [Dog])
(Thme ?z [Steak *y]) (Attr ?y [Juicy: *w1]) (Very ?w1)
(Srce ?z [Pan *w2]) (Attr ?w2 [Hot])
Mapping CLCE to Logic
- Mapping language to logic requires a lexicon and an ontology.
Ontology defines the concept and relation types.
For each word, the lexicon specifies the syntax and the associated concept or relation type(s).
- Additional information is necessary to map the resulting logic to some database or programming language.
- A general-purpose lexicon and ontology may be helpful, but a lot of domain-specific information is also needed.
Database Example
Two structures represented in a relational database
CLCE Declarations
Declaring syntax, logical relations, and SQL mapping:
Declare object as noun
from SQL("SELECT ID FROM OBJECTS"),
shape as functional noun (x1 shape of x2)
from SQL("SELECT SHAPE, ID FROM OBJECTS"),
color as functional noun (x1 color of x2)
from SQL("SELECT COLOR, ID FROM OBJECTS"),
supporter as noun (x1 supporter of x2)
with relation(support)
from SQL("SELECT * FROM SUPPORTS"),
supportee as noun (x2 supportee of x1)
with relation(support);
Declare pyramid, block as name of shape,
red, green, yellow, blue, orange
as name of color,
A, B, C, D, E, F, G, H
as name of object.
These declarations are sufficient to translate CLCE sentences
that use the declared words (plus the built-in vocabulary)
to SQL and to Common Logic in any notation.
Database Updates in CLCE
With those declarations, SQL updates can be generated from the following CLCE sentences:
The shape of A is pyramid; the color of A is red; A is a supporter of D. Pyramid is the shape of B; green is the color of B; a supporter of D is B. C has pyramid as shape; C has yellow as color; D is a supportee of C. D is an object that has block as shape; the object D has blue as color; a supporter of the supportee E is the supporter D. The shape of E is pyramid, and the color of E is orange.
These sentences illustrate various syntactic paraphrases that are
translated to logically equivalent statements in Common Logic.
Database Queries and Constraints
Query stated in CLCE:
What is the color of every supporter of the object D?
This question can be translated to an SQL query or to a goal
in Prolog and other logic-based systems.
Constraint stated in CLCE:
For every object x that has orange as color, every supporter of x has block as shape.
This constraint can be translated to an SQL constraint or to an
axiom in various logic-based systems.
Attributes and Characteristics
Declaring "orange" as both a name and an adjective:
Declare orange as name of color,
orange as adjective
from SQL("SELECT ID FROM OBJECTS
WHERE COLOR='orange'").
Such declarations can simplify CLCE statements:
Does some object that has orange as color have pyramid as shape? Does some orange object have pyramid as shape? Is some pyramid orange?
Double declarations of words such as orange and pyramid could be reduced with a general ontology of attributes and characteristics and their expression as adjectives and nouns.
Unrestricted Natural Language
- In general, still an unsolved problem.
- But for special-purpose domains, a great deal can be done.
- When sufficient background knowledge is available to the NL processor, the difference between unrestricted and controlled NL begins to blur.
- For further discussion of this issue, see the references at the end of this talk.
Conclusions
The semantic issues of language, logic, and ontology are intimately interconnected.
Trying to build semantic systems by piecemeal hacking won't work — people have been doing that for over 40 years.
Systems based on a broader vision, such as Elephant 2000, have a better chance of success.
Best strategy is to hedge your bets:
- Design a modular system.
- Make it flexible enough to accommodate anything.
- Focus on the fundamentals: logic, intentionality, and speech acts.
- Design the human interface to take advantage of familiar patterns in ordinary language.
References
The publication that corresponds to this lecture:
Sowa, John F. (2004) "Graphics and Languages for the Flexible Modular Framework," in K. E. Wolff, H. D. Pfeiffer, & H. S. Delugach, eds., Conceptual Structures at Work, LNAI 3127, Springer-Verlag, Berlin, 2004, pp. 31-51.
For more detail about the Flexible Modular Framework (FMF):
Sowa, John F. (2002) "Architectures for intelligent systems," IBM Systems Journal 41:3, pp. 331-349.
John McCarthy's paper on Elephant 2000.
Specifications for Common Logic Controlled English (CLCE).
A paper that describes the VivoMind Analogy Engine and the Intellitex parser:
Sowa, John F., & Arun K. Majumdar (2003) "Analogical reasoning," de Moor, Lex, Ganter, eds., Conceptual Structures for Knowledge Creation and Communication, LNAI 2746, Springer-Verlag, Berlin, 2003, pp. 16-36.
A paper on the model-theoretic foundations of CGs with nested contexts, which could be used to define the semantics of speech acts:
Sowa, John F. (2003) "Laws, facts, and contexts: Foundations for multimodal reasoning," in Knowledge Contributors, edited by V. F. Hendricks, K. F. Jørgensen, and S. A. Pedersen, Kluwer Academic Publishers, Dordrecht, pp. 145-184.
A paper that describes the application of VivoMind software to the problem of legacy re-engineering:
LeClerc, André, & Arun Majumdar (2002) "Legacy revaluation and the making of LegacyWorks," Distributed Enterprise Architecture 5:9, Cutter Consortium, Arlington, MA.