Choreographed Intelligent Agents: Empirical Research, Socio-Cultural
Effects, and Related Ethical Issues
- Thomas Cooper, Ph.D., Emerson College (bio)
The incoming generation of technology features choreographed
intelligent agents as a key cog
in the software/hardware convergence infrastructure. Although
elementary agents such as search engines (cf. ‘Bots’) have
been commonplace, using new agent types in fascinating combinations
– i.e. choreographing them – will be much more of a standard
feature of the next generation of technology.
This paper introduces agent choreography, as well as empirical
research about agents and relevant empirical research literature. Recommendations regarding empirical research
about the implementation and socio-cultural effects of choreographed
agents are provided. In a related vein an analysis of the likely
socio-cultural effects and germane ethical issues introduced by
agents and by multiple agent interaction (cf. choreography)
is presented based upon empirical agent research to date, the
findings of empirical research in other disciplines, and
research conducted about related technologies.
3)an agent conducting
research on Lady Diana Spencer to help an author write a book gathers
confidential information from data bases about another Diana Spencer
living in Australia because it cannot distinguish between the two
people or respect confidential boundaries. The author publishes the
latter materials about Diana Spencer’s health conditions without realizing
the mistake and lawsuits regarding invasion of privacy, defamation,
and violation of confidentiality are pursued.
The arriving generation of technology increasingly features
choreographed intelligent agents, also known as multi-agent or
interactive agent systems. This
essay will introduce agents, explain choreographed agency, discuss
what is known empirically about agents and what ought to be known and tested from an empirical standpoint, and finally
make recommendations. An analysis of likely socio-cultural effects,
and especially of some possible ethical issues associated with
implementation of agent technical systems, will be presented. Some of these likely effects and potential
ethical issues seem closely linked.
What are Agents and Choreography?
Millions of people have been introduced to agents by using
Internet search engines and other
automated software devices which replace, enhance, and accelerate
human processes. The Foundation
for Intelligent Physical Agents has roughly defined such an agent
an entity that resides in environments
where it interprets “sensor’
data that reflect events in the environment and executes
‘motor’ commands that produce effects in the environment (FIPA, http://fipa.telecomitalialab.com/
In less technical terms, an agent is smart software that autonomously
and often repetitively performs computer or computer-like tasks
on a person’s behalf. An
agent might automatically send e-mail birthday cards on the correct
dates of each year, locate new research about a topic as it becomes
available on-line, book travel to Tokyo or London every month,
or launch missiles at designated targets whenever specific airspace
is invaded. Potentially,
sophisticated agents may act as an individual’s concierge, para-legal,
operations director, secretary, assistant, travel agent, researcher,
and similar functions.
Choreographed or interactive agents
are entire systems of linked agents that can perform more complex
assignments such as functioning as a mailing house for a company. Such a multi-agent mailing system could simultaneously
mail bulk messages or publicity, send auto-replies to incoming
messages, selectively forward replies, and at the same time add,
drop, and correct addresses.
Choreographed systems might also serve as the nervous system
for an Artificial Intelligence complex or robot, sell and distribute
products on-line, provide multiple options for someone who is
“lonely” on-line, write hourly news updates, and eventually coordinate
elaborate power grids, weapons systems, and internationally interactive
and Cooper, 1998)
Agent Expansion and Cultural Impact
Without knowing it many publics worldwide have begun using
intelligent agents. For
example, search engines
are all but routinely provided to those who purchase access
to the Internet. However, little if any thought seems to be
given to the impact that choreographed agents might have upon
the cultures into which they are introduced. Yet it is widely known that the previous introduction of communication technologies into specific cultures
and regions has had marked
Consider, for example, the specific case of the Pacific region. Empirical research, especially
that published in PTC Proceedings and the Pacific Telecommunications
Review, and field work, such as by Ogden (1991), Varan (1993),
Plange (1993) and in documentaries by O’Rourke
(1987), and Gardner (2000)
suggest that technologies introduced into the Pacific may have
both expected and unanticipated socio-cultural effects.
For example, Ogden (1991)
noted that the impact of television in Palau and The Marshall
Islands led to instances of young people disrespecting and ignoring
their elders in ways that disrupted traditional society.
O’Rourke (1987) observed that a tiny island such as Yap,
after the introduction of television, was socially influenced
to depend upon unnecessary consumer products, and Varan (1993)
noted that students in the Cook Islands who watched late night
TV fell asleep in classes, that violent videogames replaced peaceful
traditional childrens' games, and that island girls began to aspire
to unattainable and cosmetic ideals of female beauty.
If there are studied effects (too
many to be fully listed here) of television, radio, VCRs, video
games, and the Internet within the Pacific, it seems both likely
and logical that choreographed agents will have Pacific impact
as well. Moreover, as
early as 1998 the Foundation for Intelligent Physical Agents was
hosted in Osaka, Japan, with participants from Comtec, Hitachi,
Fujitsu, JVC, Matsushita, NHK, NTT, and Tohoku University in attendance
(http://fipa.telecomitalialab.com/osaka/ ). There can be no doubt that there are, on the one hand, key
players in the development and implementation of agents in the
Pacific as well as, on the other hand, entire countries, villages,
and islands that are unaware of the possible pending effects of
However, this is but one example. Research seems non-existent about the possible effects of choreographed
agents in, for example, Latin American or Africa. Nevertheless, such technology is being developed
and introduced wherever there is access to the Internet. However, intelligent researched human
thinking about whether Artificial Intelligence embedded
within computers will transform specific societies in similar
or different ways is difficult to obtain.
is Known Empirically About Agents?
It is far better to have empirical and scientific knowledge
about a new technology than merely anecdotal or undocumented reports. Fortunately, a growing scholarly and technical
literature has emerged to surround agent development, although
not around socio-cultural effects.
Layton and Isbister have provided a sizable literature
review of agent research in 2000 (http://research.microsoft.com/research/ui/persona/isbister.htm ). Their analysis of the literature concludes that there are primarily
two types of substantial research, one concerning the more publicly
widespread “filtering” agents, including search and other autonomous
engines for users. The second type covers a wider range of agent
bridge functions that might assist users to obtain any large number
of goals. It is within this second literature that the expressive qualities
of “characters”, and thus the thin line between “intelligent”
software and human behavior are being discussed and debated. Isbister and Layton found that an important
list of issues, which include trust, user control, customization,
repair, security, privacy and especially anthropomorphism, are
being discussed within the mushrooming literature.
Scholarly literature reviews are more important when their
analysis reveals a summary overview. One of the best empirical overviews of agent
research is presented by Nwana and Ndumu(1999) within their eye-opening
paper, “A Perspective on Software Agents Research”
After assessing research from the final five years of the twentieth
century, the authors conclude that despite much progress in the
development of agents, serious problems still remain in the advancement
of multiple agent systems. In short they conclude that
insufficient work has been dedicated to the problems of ontology,
legacy, and the context by which agents interact and fully understand
each other. In other words,
Nwana and Ndumu note how much research addresses the trees but
not enough addresses the forest.
Therefore, far more attention needs to be paid to holistic
or systemic approaches to agent choreography if agents are to
function in concert. Refinement of agent “teamwork” must become
a theme of the current decade.
Ought to be Known About Agents?
Research to date seems far more focused upon the functionality,
design, tasks and technical requirements for software than upon
the possible socio-cultural effects of agents.
Hence, after a review of existing literature and of research
overviews it is the conclusion of this essay that not only are
Nwana and Ndumu’s recommendations valid – more systemic or holistic
research is needed – but also a campaign for predictive and preventive
socio-cultural empirical research should be undertaken as well.
Moreover, a specific subset of these effects raise discrete
ethical issues (to be discussed later), issues warranting a large
number of studies and significant body of literature.
Before elaborating upon how the black holes in agent research
might best be filled, it is important to note that there are already
a few initial steps in that direction.
For example, the entire discussion about whether or not
Artificial Intelligence will ever be “alive” often considers the
possible societal and moral consequences of having machines perform
as humans. Moreover, there are scholars investigating
the agent field such as Abe Momdani and Jeremy Pitt in England
who are fully aware that intelligent agents are not autonomous
but are owned, and consequently must be studied as part of society (see Pitt’s Alfebiite Project
). Such initial steps will beget others but are currently the
exception, not the rule.
Given the dearth of agent effects research, it is valuable
to study “next-of-kin” empirical research and analysis for its
possible relevance if applied to new and relatively untested technologies. For example, although the effects of combining agents are largely untested, there have nevertheless been
important breakthroughs in testing interactivity or combination
processes in such fields as toxicology and bio-chemistry.
In such studies, as with Heisenberg’s Uncertainty Principle
in physics, it has been demonstrated that interaction (or, in
Heisenbeg’s terms, observation) of itself may produce discrete
effects. For example, U.S. toxicologists have perceived:
A step toward recognizing multiple
agents. The National Academy
of Science Committee notes that two or more additives can act
synergistically to promote high-tumor yield in animals, whereas
separately they do not (Hall, 61).
Similarly, in his landmark research text Food for Nought (Harper & Row, 1974), Canadian
bio-chemist Ross Hume Hall concluded
that chemical additives used in food production might have one
effect upon food if tested separately,
but when tested in combination
with other chemical additives might behave entirely differently.
(Hall, 62) What is important about such research is the
importance it gives to interaction.
Two important bodies of literature exist about the history
of technology and its effects. The first, which includes works by Lin White, Seigfried Giedion, Lewis Mumford,
Harold Innis, Marshall McLuhan, and others, has already suggested
that technologies have specific effects upon their environments
and audiences. Based more upon qualitative, theoretical, and
historical research, this quasi-deterministic and sometimes fully
deterministic literature suggests that specific technologies such
as the printing press and parchment scrolls have transformed particular
societies and altered the senses (and their ratio)
used to communicate.
The second literature, as typified
by George Gerbner, Larry Gross, Ellen Wartella, and many
others, depends far more upon specific empirical quantitative
studies which test the effects of a specific communication technology
– typically television – upon modern audiences.
However, within both these literatures the combined
effects of interactive media are minimally discussed. For
example, if some societies in remote islands and impoverished
countries had no electronic media until the 1980s and 1990s
(and some are still totally without), what does it mean for an
island or country to now encounter multiple technologies – television,
the Internet, agents, the VCR, the CD, satellite, the DVD, fax,
and others – simultaneously?
The interaction of multiple media is important as it may
create combined effects such as noise pollution, Wurham’s
“information anxiety” (the feeling of not being able to keep up
with information), conflicting perceptions of truth, sensory ratio
shifts, confusion, multi-cultural conflict, global connectivity,
commercialization, social “progress”, business competition, and
other effects, some positive, some negative, some neutral, some
mixed, and some unknown. These combined, often hidden effects, arrive
almost as drastically in post-modern, industrialized societies
as well since the accelerated introduction of untested
communication technologies resembles a perpetual motion machine.
agents present a web of software interactivity within a larger
complex of multi-media interaction. Therefore it is critical that not only the effects of agents be
empirically tested, but also the effects of combined or
interactive systems of agents be scientifically studied
within the larger context of both a multi-media and a multiple
media environment. Studies
of effects and ethical issues spawned by new media such as by
Cooper (1998) exist but
the testing of how such new media will interact within various
environments is a necessity of the near future. Moreover, in an
age of globalization, Westernization, and the digital divide,
the further development of the “www”-world and “e”-environment
in sophisticated and hidden interactive ways raises questions
about further Western technological dominance and about the future
homogenization of culture into a single accent.
Socio-Cultural Effects and Ethical Issues
Many positive effects of agents are inherently obvious and
are the very reason that such new technologies are being implemented
wherever practical. These positive effects include 1) saving time,
2) saving money after initial purchasing costs, 3) saving labor,
4) making task accomplishment comprehensive, 5) automating mindless
tasks, 6) increasing global connectivity and 7) enhancing productivity.
However, it should be noted that each culture is different
and some cultures may not wish to increase global connectivity
nor to speed up production nor disrupt ancient traditions.
Indeed introducing agents of speed-up which bring other
cultures and advertising more readily into a remote or traditional
culture often introduces the ethical problems associated with
imposing external products, rhythms, and lifestyles which threaten
or divide local populations. Moreover, the implementation of new technologies,
like new drugs, creates hidden or unknown effects, some of which
later prove negative, such as the radiation danger of computer
monitors to pregnant women, the birth defects stemming from thalidomide,
and the increased aggression in children who are heavy users of
Although no crystal ball is available to observe the full range
of agent effects in each country, there are specific methods for
anticipating possible effects that are more scientific than others.
To some degree new technologies may be tested before, during,
or shortly after their implementation, as was once the case in
the United States by the Office of Technology Assessment.
Associations such as Future and Emerging Technologies,
which exist in Europe, may also be established for professionals
and academics to compare notes (http://www.cordis.lu/ist/fetintro.htm#what
). Professional organizations such as FIPA also allow professionals
to establish over-arching standards and ethical principles among
those developing agents. Additionally, interest groups within
academic and professional organizations such as the Pacific Telecommunication
Association in the Pacific and the National Communication Association
and International Communication Association further east could
be created to discuss the possible social effects of agents and
of other new communication technologies.
Among scholars there are two specific approaches to anticipating
possible socio-cultural effects which are grouped under the coined
term “presearch”, or “preventive research”.
The first of these approaches might be called “next-of-kin”
studies, that is, previous tests upon similar technologies and
tests of similar phenomena in other disciplines such as those
already reported in toxicology and biochemistry.
The second approach might be called “attribute-based” predictive research, which
means that if one may properly determine the essential attributes
or functional specifics of a new technology, then one might predict
possible socio-cultural effects by isolating and sometimes exaggerating
that attribute. For example, one attribute of agent technology
is that of autonomy. Autonomy declares that an agent may
accomplish a task repetitively without further instruction.
Such autonomy can potentially lead to a Sorcerer’s Apprentice
effect in which a process, once turned on, is not successfully
turned off. For example, if the boss in the office sends out electronic
birthday cards to all employees, after the boss dies the cards
will continue to be sent via e-mail, which would signal to everyone
that an agent rather than the boss had sent the cards. If an agent autonomously broadcasts warning messages to enemy ships
which invade local waters, how will the agent know when countries
have shifted their allegiance, have permission to trade goods,
or are in negotiation with the host country.
Hence one may predict that vendors, manufacturers, governments,
and policy experts will need to anticipate socio-cultural problems
based upon the attribute of autonomy and will need to constantly
reprogram, test, and monitor agents.
According to the combined research of FIPA scientists and this
research, other attributes of intelligent agents include 1) social
ability, 2) reactivity, 3) pro-activeness,
4) adaptability, 5)
mobility, 6) temporal continuity, 7) transparency, 8) auto-replication, 9)
self-maintenance, 10) auto-destruction, 11) sustainability,
12) multi-cultural presence and 13) camouflage/impersonation (http://www.fipa.org). Unless there is sufficient research, the combination of these attributes
mingled with the choreography of intelligent agents could lead
to predicaments which have not only unforeseen effects, but also
attendant ethical and legal issues which could be difficult to
unravel. Consider these hypothetical cases below which are composites and
projections of problems already encountered with agents or “next-of-kin”
1) the Stan McGregor
case in which an individual says “I am lonely” to the computer provided
in his hotel room.. Immediately choreographed agents simultaneously
provide Stan with lists of contact information about religious leaders,
psychologists, local arts and amusements, best friends, family,
escort services, and others who may attempt to soothe, counsel,
postpone, or eliminate his loneliness
Who is accountable if a pop psychologist or cult leader,
listed on-line in a manner so as not to be distinguished from all
the reputable counselors, gives bad advice leading to suicide or
if the escort service leads to death by AIDS or to criminal activity?
At this time agents cannot distinguish “ethical” and “accountable”
from “unethical” and “irresponsible”
(See Cooper et al, 1998, for debate about agent liability).
2) a listener
writes an evangelist who uses choreographed agents to filter and
answer audience mail with several different form letter replies
selected by topic. Since agents cannot distinguish “tone” and hidden
messages in the letters, but only respond to key words and phrases,
the already disturbed listener, seeking advice from a trusted authority,
receives the wrong generic letter, based upon using figures of speech,
interpreted literally by the agent, and has a nervous breakdown
or worse. The author writes
“If I get one more form letter back from an evangelist about turning
my money over to God and letting Him handle my financial problems,
I’ll commit suicide”. The
evangelist’s agent “reads” the key words “money” and “financial
problems” in the letter, and routinely sends him the financial form
choreographed agents, trained to shoot missiles at hostile targets
when a host country is attacked, fire instead at their own cities and people because the combination
of agents creates an unanticipated (Hall) effect or because enemy
hackers cleverly change
the target sites.
The attributes of agents make these and many other unexpected
scenarios possible unless sufficient preventive research (presearch)
is conducted. Moreover
agents created in one culture may have additional unknown effects
when introduced to other cultures. Hence the complexity is thrice
increased by the multiplicity of agents, of interactions, and
the implementation of agents, especially when combined as multi-agent
systems, and in countries where the Internet itself is a recent
immigrant, needs to be closely studied. Finally, a series of recommendations are presented based upon the
conclusions drawn from available empirical research, literature,
technology studies, next-of-kin research and from attribute-based
In most parts of the world most people are unaware of what
intelligent agents actually are, even if some populations are
using an agent prototype called the Internet search engine. It
is important that people be made aware of the nature and functions
of agents so that they will know, for example, if a response received
from a celebrity or government official came from a real person
or was generated by a machine. Wherever agents are unknown, education
about the nature and use of intelligent agents is important in
and beyond the classroom.
Moreover, it is vital to conduct empirical studies to test
the effects of linked agents in specific countries. There are hidden and long-term effects of each technology that are
often only discovered after the fact – once law suits and problems
have occurred. Scientists
conducting presearch can take steps toward minimizing possibly
damaging and as yet unknown effects.
Councils of scientists, business representatives, parents,
community leaders, government liaisons, and others, depending
upon the culture, may wish to be assembled to consider new technologies
prior to their implementation to make recommendations to engineers,
manufacturers, and policy groups prior to the widespread use of
Many positive and neutral effects may be anticipated from intelligent
agents, primarily in the areas of eliminating repetitive tasks
and in refining and speeding up the processes by which such tasks
are accomplished. Thus, differing cultures may value or devalue
agents to the degree to which each values authenticity, speed-up,
physical and mental labor, surrogate or substitute automation,
natural rhythms, cultural and individual privacy, and technical
innovation. Each culture
will have differing sensitivities to the degree that a technology
might contribute to the preservation or erosion of traditional
culture and language. For
example, one might imagine the differing response to and ethical
concerns about the ringing of an agent-activated portable cellular
phone/e-mail/pocket calculator in a cemetery burial in Italy, a mosque worship service
in Iran, a tea ceremony
in Japan, and an academic
seminar in England.
There are a growing number of resources available about intelligent
agents. Perhaps the best introduction to a wide variety of scholarly
and technical sources, many of which are listed in the endnotes
references of this article, are listed at the UMBC comprehensive
It is important that such sources and research grow in quantity,
quality, and diversity.
It is crucial that available findings about the effects of
solo and choreographed agents be quickly published and posted
so that an unknown quantity may be better tested, adapted, and
understood by the information rich and poor alike. Ideally, choreographed
agents may perform numerous positive and important functions.
However, potential negative and unknown effects cannot
be ignored and must be carefully investigated through widespread
and multi-cultural empirical analysis.
A balanced and thorough inspection is essential.
Bradshaw, J. (ed.) (1997). Software Agents. Cambridge: MIT Press.
Brislin, R. (1984).
Cross-Cultural Encounters. New York: Pergamon Press.
Christians, C. and T. Cooper (eds.) (1998). The Journal
of Mass Media Ethics. Special
Issue: The Ethics of
New Technologies. 13,2.
Cooper, T. (1998). “Cases
and Commentaries: Marketing
New Software and
Choreographing Intelligent Agents”
The Journal of Mass Media Ethics.
13, 2, pp. 124-136.
Ellul, J. (1964). The
Technological Society. New
Ferber, J. (1999). Multi-Agent
Systems: An Introduction to Distributed Artificial
Intelligence. New York: Addison-Wesley.
Gershenfeld, N. (1999). When Things Start to Think. New York: Henry Holt & Co.
Goldberg, D., D. Nichols, B. Oki, and D. Terry. “Using Collaborative Filtering to
Weave an Information Tapestry.”
Communications of the ACM.
Hall, R. (1974). Food for Naught: The Decline in Nutrition. Philadelphia: Lippincott.
Hoffman, U.and J. Stamos-Kaschke. (1998). “Mission Impossible? User Meets Agents.” Proceedings: Practical Applications of Intelligent Agents
Technology. London. pp.
Innis, H. (1951). The
Bias of Communication. Toronto. University of Toronto Press.
Isbistir, R. and T.
Layton (1996). “Intelligent Agents: A Review of Current Literature.”
Lai, K., T. Malone, and K. Yu (1988). “Object Lens: A ‘Spreadsheet’ for Cooperative
Work.” ACM Transaction
on Office Information Systems.
McLuhan. M. (1964). Understanding
Media. New York. Signet Books.
Maes, P. and R. Kozierok. (1993). “Learning Interface Agents”, Proceedings of the
AAAI 1993 Conference. Cambridge: MIT Press.
Moravec, H. P. (1998).
Robot: Mere Machine to Transcendent Mind. Oxford:
Oxford University Press.
Ndumu, D., J. Collis, and H. Nwana (1998). “Toward Desktop Travel Agents”.
B.T. Technological Journal.
16, 3, pp. 69-78.
Nwana, H. and D. Ndumu (2000). “A Perspective on Software Agents Research.”
Nwana. H.and N. Azarmi.(eds.). (1997)”Software Agent and Soft
Toward Enhancing Machine Intelligence.”
Lecture Notes in Artificial
Intelligence. New York:
Ogden, M. (1991). “Preliminary
Report on the Social Impact of Television in
Palau and the Marshall Islands”.
PTC Thirteenth Annual Conference
Pacific Telecommunications Council, pp. 845-51.
Ogden, M. (1993). “Foreign
Influence, Local Choices: The Social Impact of
Television in Micronesia”.
Pacific Island Communication Journal.
Pavlik. J. (1998). New
Media Technology: Cultural and Commercial Perspectives.
Needham Heights, MA: Simon and Schuster.
Plange, N. (1993). “Video, TV, and Fiji’s Society: Fast Forwarding
into the Future
or Pausing for a Closer Look”. Pacific Island Communication Journal. 16, 1,
Rao, A. and M. Georgoff (1995). “BDI Agents: From Theory to Practice.”
Procedures: First International Conference on Multi-Agent
Francisco, CA., pp. 312-319.
Russell, S. (1997). “Rationality
and Intelligence”. Artificial
94, 1-2, pp. 57-77.
Tan, Y-H. and C. Castelfranchi. (2001). Trust and Deception in Virtual Societies.
New York: Klewer Academic Publishers.
Varan, D. ((1993). “Introductory Television: Seven Lessons
Learned from the Cook
Islands”. Pacific Islands Communication Journal. 16, 1, 29-62.
Wedemeyer, D. and Savage, J. (1994). PTC Sixteenth Annual Conference
Pacific Telecommunication Council.
Wilson, R., F. Kiel, (Eds.). (2000). The MIT Encyclopedia
of the Cognitive
Sciences. New York: Bradford Books.
Wooldridge, M. and Jennings, N. (1995). “Intelligent Agents: Theory and Practice.” The Knowledge Engineering Review.
10, 2, pp. 115-152.
Yokoo, E. et al (2001).
Distributed Constraint Satisfaction: Foundations of
Cooperation in Multi-Agent Systems. New York:
http://fipa.telecomitalialab.com/ and www.fipa.org
Singh, M., R. Falcone, and Y. Tan, (2000) “Deception, Fraud
and Trust in Agent Societies.”
Alfebiite Conference, Barcelona, Spain.
Gardner, R. (2000). Telephone.
Elizabethtown, Pa./ Cambridge, Ma.