Introduction
This was an interview conducted for the same The Machine That Changed The World documentary series featuring the interviews with Larry Tesler and Steve Jobs I recently transcribed and published here. This interview took place over two days in July 1990. Only portions of it were featured in the documentary series.
I’ve always been an admirer of Alan Kay and his work. As I was watching this interview I realised there were so, so many things worth taking note of, and worth sharing, that I decided to carry out a full transcription of it. As you can see, it was an energy-draining, time-consuming task, but I’m happy to have done it. There’s so much food for thought here that it’s a veritable banquet.
For a comprehensive look into the series, I recommend checking out the excellent work by Andy Baio in 2008 on his waxy.org website.
About the interview
The video of the interview can be watched here. [Update, March 2022 — The original YouTube link doesn’t work anymore. You can watch the interview here instead. Note also that the video is accompanied by a transcript on the WGBH website, but it doesn’t look accurate in certain places; maybe it is an automated transcription?]
This is the full transcript of the interview. As mentioned above, the interview was recorded over two days, so it’s quite long (about 2 hours and 45 minutes in total). Therefore I thought it was best to split it in two parts, one for each day. This is Part 2.
I’ve applied gentle editing in some places to make the context of certain questions, and the meaning of some convoluted passages a bit more understandable. Understanding how the interviewer formulated his questions was sometimes hard, due to the low volume (he didn’t sound as he was miked), and to the fact that his remarks could be somewhat meandering.
Topics include the evolution of the computer and its role in society (past, present, and future), user interfaces, Doug Engelbart’s famous demonstration, the FLEX machine, the computer as a medium (including parallels with, and excursions on, the evolution of the book), the experience at Xerox PARC, Steve Jobs, Apple, the Macintosh UI in relation to what was pioneered at PARC, the Alto, the Dynabook concept, working with children, the stages of development and the ways to learn about the world, interfaces and application software, the universal machine, the computer interface as user illusion and the concept of virtual machine, the future of computing, virtual reality, and general considerations on the evolution and the revolution of the computer. Summarising these is a bit difficult, because all topics keep surfacing and returning throughout the whole flow of the conversation.
Disclaimer: I have done this transcription work and chosen to publish it here in good faith, for educational purposes. I don’t make any money from my website, as it is completely ad-free. In any case, should any copyright holder contact me requesting the removal of the following material, I will certainly comply.
Enjoy the conversation.
The interview - Day 2
Interviewer: All right, now, if you can recollect where we were yesterday — we were talking about Xerox PARC. Do you look back on it as sort of a golden age in computing? That so many people were gathered for a period together.
Alan Kay: I know I sort of feel it would be a little bit presumptuous to declare it a golden age in computing. It was certainly a golden age for me. I think the five years between 1971 and 1976 were the most fun I’ve ever had doing research. And I think that there’ve been five or six critical places in computing over the last forty years that have affected the way people think about things, and PARC was one of them.
Int: Now you weren’t the run-of-the-mill Xerox executives, and Stewart Brand wrote an interesting article about you, didn’t he?
AK: [laughs] Oh yes! That was a hilarious little side excursion. Stewart Brand was the person who put the Whole Earth Catalog together and he got interested in computer people because he lived right across the street from Engelbart’s work at SRI. So he decided he’d do a piece about the culture. And one of the places that was just starting up then in 1971 and ’72 was Xerox PARC, and so we all knew him and he was a friend and we invited him in. He wrote this great article which was published in the Rolling Stone, which was really considered to be a rag back then, especially by the East Coast establishment.
Int: It describes you people as sitting on bean bags and…
AK: Yes, the whole culture; and we were photographed — the photographs were all taken by Annie Leibovitz and, you know, it was a very ‘Hollywood, California’ type scene, and the Xerox people did not like it at all. They had an enormous reaction; so large a reaction to it that when Stewart republished the article in his book, they forced him to not use the name Xerox, and so he referred constantly in this article to “Shy Corporation”, which is what he called it.
Int: Now, you were talking yesterday about having done all this work on the Alto, the questions of whether you really wanted to get it out there, and so forth; and you made lots of demos, you said. Now, one big effort in particular was made at a sales meeting in Boca Raton, wasn’t it?
AK: Yes. That was rather late in the game, though. That was quite a few years after the Alto had been around, but there were constant recirclings… This wasn’t so much trying to get the Alto out as a product, although there were people who were very interested in it. A lot of these meetings, particularly the Boca Raton one, had to do with just getting Xerox at the top and its major executives to have some sense of how the world was changing and what their future was likely to be.
Int: Did you feel kind of disappointed — not so much, as you say, that they might have had commercial reasons for not backing it — but did you feel that they really understood what you’ve done, that they got it?
AK: No, I don’t think they got it. But I think the important thing about the Xerox deal is, what they did promise to do is to give us relatively unsupervised support for a period of ten years. And that is what they did. We had our skirmishes with them, and there were some people who had to sacrifice their body and soul to keep the umbrella over the group, but in fact Xerox turned out to be a very good funder. And there was no promise or even any intent that I ever heard in the beginning that Xerox was going to turn these into products. Most large corporations use their long-range research more as a hedge against future disaster than as pure product development.
Int: So in a sense you were particularly surprised then.
AK: Yeah, I was surprised! [laughs] Sure, I mean, it’s one thing for them not to make any representations about doing it as a product, but the whole setup was so obviously the way it was going to be that I was surprised, sure. I was amazed.
Int: Now while this had being going on, Ted Hoff had done the work on the microchip, and so forth, and hobbyist machines started to appear. What did you guys make of the hobbyist machines?
AK: Well, there was something that… first as a 4‑bit-wide chip and then an 8‑bit chip, and so forth, and they were sort of inevitable that they would appear. For me, I was surprised that so many people were interested in them. I realised that an enormous number of people felt disenfranchised from computers completely, and these were a way of getting in — I can touch the new technology and so forth. But the kinds of software you could run on them was really limited. And so there were various opinions at PARC. Larry Tesler was much more interested in the 8‑bit machines than I was, as an example. My feeling was is that you had to have at least a 16-bit machine that went so and so fast in order to do all of this stuff, and in fact that’s exactly the way it worked out. The Macintosh was a 16-bit machine that went thus and so fast, and you had to go that fast in order to do all the things that we were doing at PARC. And so, from my standpoint, I would have been just as happy if no machines had been built up until 1984 or so. Just from the standpoint of, you think of all of the unfortunate standards that were set, like MS-DOS and so forth, that are holding back the world even today. I think it would have been… But you never know!
Int: Yes, but on the other hand, as you say, they released this pent-up sort of mass of disenfranchised people. I mean, that was what was surprising even though, as you said, they were toys in one sense.
AK: Yes, yes, and I think that, depending on how you look at it, certainly one great thing was done on an 8‑bit micro and that was VisiCalc. That was one of the best things that’s ever been done on a personal computer as an idea. The reaction to that at PARC was both admiration and shock. You know, we couldn’t believe that we hadn’t seen it. That’s how arrogant we were. But I think that, aside from that, almost everything else done on the 8‑bit micro was a sort of a reversion back to the early 1960s when machines were very weak. Most of the 8‑bit micros had either no operating system or terrible operating systems. It’s not clear which is worse. And many of those operating systems are still hanging around today.
Int: Now, what you saw happen after this work was done at PARC, in the next years ahead this was going to be exported from a company like Xerox — that wasn’t really a computer company — into one of these new fledgling populist hobbyist companies. And that was a remarkable transition, wasn’t it?
AK: Yeah, I actually thought that Xerox was the right place to launch from since it wasn’t in the computer business. It really didn’t have any history to give up there. Whereas, I think it was more remarkable for a company that was deeply wedded to a certain kind of 8‑bit computing, like Apple was, to be willing to throw it all out and start in a completely new way. But that is very much one of Steve Jobs’s biggest strengths.
Int: Do you have any recollection of the events leading up to Jobs’s visit? Were you involved at all in that?
AK: No. No, I was at the famous demo that Larry Tesler gave Steve Jobs and some of the other people from Apple, but…
Int: What had you heard? Had you heard that Xerox was interested in buying a stake in Apple? Was that known?
AK: No, Xerox had a stake in Apple by then. There’s a company called Xerox Development Corporation and they had stakes in various companies. I forget how much it was, 10 or 20 percent or something like that. But, you know, of course at PARC we thought the Apple was the better of the 8‑bit micros that were around and so forth. But it was not unusual to have– we gave many, many demos. So it was not unusual to have somebody like Steve Jobs and other people come over.
Int: Do you remember talking to Jobs that day?
AK: Sure, sure…
Int: Did he get it then?
AK: He got it right away. I mean, there are two kinds of demos. There are ones where you are struggling to get the person to see what’s going on. We found for many people [that] they weren’t used to looking at a computer screen, so we had to put our finger on the screen and get them to watch our finger, and then see if they could focus in another quarter of an inch, and stuff. And then there are other people who tell you what is going to happen next. I mean, they are so on top of the demo that they know: “And now you’re going to show me…”, “Yes, here it is…” — you know. Those are great demos. I had the great pleasure of showing Smalltalk to Kristen Nygaard, who is the inventor of Simula, which is one of the major influences on Smalltalk. And it was that kind of demo: he just knew everything I was going to show him. It was stuff that he had been dreaming about for years, and here it was. And we’ve been friends ever since.
Int: Now, what do you think Steve Jobs’s greatest achievement is, then? He took this, he got it, and he took it back.
AK: I think, what he took back was an idea that things could be done a different way. And particularly Steve, who is such a visual person — very sensitive to how things look and appear, and stuff. And the whole notion of the graphics interface was something that really captured his imagination. And eventually some people from PARC went over to Apple — Larry Tesler was one of them. But what happened there was that they pretty much took what we had done as a departure point, and did a completely new design. And I remember I didn’t see any of it until the Lisa came out in 1983. And when I saw it I thought it was just the best thing I had ever seen. It was just incredible what… The Lisa was beautiful.
Int: Okay, but the Lisa was still too expensive. So really the Macintosh was the threshold machine which really changed…
AK: Yeah. I mean, the Macintosh in many ways is not a good a design as the Lisa, but it was a severe compromise. And the thing that was great about it is that it used the weak power of the then 16-bit [Motorola] 68000 to great advantage. Where the Lisa couldn’t quite make it with all the things it was trying to do. So in many ways you can think of the Macintosh II as Apple’s return to the Lisa.
Int: Now, when the Macintosh came out — and certainly the history of the computer since then has been very much a sort of a vindication of everything you did at PARC, isn’t it? — In a sense, hadn’t the main victory been won by that point? People had realised computers at least could be different from what they were…
AK: Well, I don’t think any of us at PARC were fighting a war, so it wasn’t clear who we were victorious against. But if it meant getting lots of people convinced, I don’t think that it was won, because the majority of people who compute today still don’t interact with a computer in that way. […]
Int: So you were saying that while it introduced a lot of people to this technique, it hadn’t…
AK: Yeah, I think that the important thing is, what we did at PARC was not an ultimate answer to the problem of interacting with computers. I think that a lot of people are going to be interested in it more than the millions that are now, and what will inevitably happen is that people will continue to be interested in it long after it is worthwhile interacting with computers that way. So you have this thing where people have this tendency to, once they like something, they get religious about it, and it hangs on and hangs on long beyond its actual use.
Int: Now, at the time Macintosh came out, Apple essentially was betting the company on it, right? And there was this very popular standard MS-DOS in existence on millions of computers. What are the reasons why most people find one more intuitive and easier than the other? And I want you to talk about some of the things you told me yesterday.
AK: Yeah, well I don’t think we ultimately know what the reasons are, but we certainly were guided by some theories that we have different ways of knowing about the world, and only one of them is through language. We have a kinesthetic way of knowing about the world through touch. We have a visual way of knowing about the world. The kinesthetic and visual ways seem to be more intuitive for people. They’re less crisp. They deal more with analogy, and the touch thing makes you feel at home. You’re not isolated from things when you’re in contact with them — you’re grounded. And so I think that, for me, the major reason the Macintosh works is because the mouse gives you just the tiniest way of putting your hand in through the screen to touch the objects that you’re working on. The screen gives you a theatrical representation of something much more complex, something that computer designers don’t even want to think about as a computer executing two million instructions per second with thousands and thousands of instructions in there. And then finally the least successful thing that we did, that we’re still working on, is a symbolic underpinning for all of this that allows you to express back into this little pocket universe.
Int: Certainly the image you have — you have a film where you show a two-year-old child — the image of the computer underwent quite a transformation.
AK: Yes, I think so. That film of the little girl, 22 months old, using a Mac very confidently — she’d already been using it for about six months or so — strikes a lot of people in a way that words don’t. Because they see here’s this little child, about 70% literate in the access part of the Macintosh user interface. She can start up applications, and do things in them, save them away, and all of those things. That’s what we were trying to do. To extend this thing from being something like a specialised tool or a car, to something that is more like media that extends down into childhood and up through the elderly.
Int: I want to turn a bit now to the practice of writing both interfaces and application software itself, and some of the problems with the differences between software and hardware. Basically, how would you characterise the difference between software and hardware? Because the word software, as far as I can see, only grew up about 1960. It was something new in a way.
AK: That’s true, although the notion of the stored program goes back a long way. But I think for me there isn’t any real difference — the hardware is just software that’s crystallised early. Because basically what you always have is something that you can remember in terms of some medium that can take on markings. And there are different kinds of markings. Then you have to have something that allows you to change those markings, and read them, and change them. And the simplest kind of computer you can build is one that only has a couple of states in it, and it’s all memory. It’s practically a clock and everything is out on the memory. So there’s almost no hardware there at all. And the main reason that there’s a fair amount of bits in the hardware of today’s computers is there are a lot of functions you would like to compute rapidly, and so there are special-purpose little pieces of logic in there for doing fast arithmetic and so forth.
Int: And that grew out of this historical accident that the first functions people wanted to compute were computational. So it made sense to put in special circuits to do…
AK: To do arithmetic and stuff. Yeah, I think it definitely is true, and there have been computer designs and computers built that look at their memory in a completely different way, of doing content-addressed storage and having many thousands of processors looking at storage at the same time where hardly anything resembling arithmetic is done most of the time. But in fact, it doesn’t matter, because arithmetic itself is just a manifestation of a particular way of putting the logic elements together.
Int: So at its very basic level — the way Turing might have thought about this — this is a general-purpose manipulator of symbols. It takes some markings, transforms them, and puts out other markings.
AK: Right, right, and there’s a trade-off between the number of different kinds of markings that you want to have the memory store, and the amount of logic that you need to be able to carry out computing functions.
Int: If we take an average computer, its hardware can carry out, what, just about a hundred basic functions?
AK: Sometimes, yeah. These days they’re usually up in the range of 300 or so. I think for most people trying to understand how a computer works down inside, it’s actually mostly memory, a vast amount of very undifferentiated stuff that is just for holding markings. Then there’s a very small amount of logic that carries out a few basic instructions. And you can think of all of the rest of the hundreds of instructions as things that the basic instructions could carry out but have been encoded especially for speed.
Int: And then what the programmer has to do, is using these available facilities it has to instruct it to do other things…
AK: Yeah, well most good programmers are relatively lazy, so the last thing they want to do is spend a lot of time grubbing around down in the machine code of the computer. So, usually what they do is, they write a couple of pieces of software, one is called an operating system, another one is called the programming language. And often they are both the same, as Smalltalk was. And what that piece of software does is to create a virtual machine which is much more hospitable. And so it’s a machine simulating a much nicer machine, and all of a sudden life is much more fun. And often you will use that machine to simulate an even nicer machine. Eventually things slow down so that the most wonderful machine you could have might run too slowly to be interesting, but some people still like to program in terms of these highly idealised machines, because maybe they’ll be built one of these days.
Int: So in a sense are you telling us that what we see on our Macintoshes is a much more pleasant machine…?
AK: Yes, in fact the user interface you can think of as the last layer that is designed to further sweeten the realities of all these different layerings that are going on. Most important thing is, when you’re baking a cake in the kitchen, you don’t have to worry about the details of organic chemistry, because they’re already subsumed by the ingredients that you have, and the recipes are there to make sure that what you have converges on a cake rather than a mess. And for exactly the same reason people like to confine where they’re working into an area where they know what roughly is going to happen, and if that area doesn’t work out well, then they’ll go to a deeper lever and make a few changes.
Int: Now, all of this of course depends on the computer being able to carry out instructions fast? Otherwise we wouldn’t be having this conversation…
AK: Simulation would not be that much fun if it were really slow. As it was when people had to just calculate simple trajectories of shells in World War II, they’d have 300 or 400 people on desk calculators just calculating a simple ballistic trajectory. That was a simulation, and in wartime it was deemed important enough to put these 400 people to work for days doing these things.
Int: Now, before the computer was built, Alan Turing, approaching the subject from a different point of view, spoke of a very simple machine which could imitate all other machines. The digital computer is an example of one such sort of machine, isn’t it? What does it mean then in that sense to call it a universal machine?
AK: Well I think it’s one of the niftiest things to wrap your head around. And that is that, regardless of what kind of thing you have — if it has a few basic properties like a memory and the ability to put symbols into memory and take them out and make changes, and make a few tests on them — that is enough machinery to enable you to simulate any computer that has ever existed, or any computer that will ever exist. How interesting the result will be depends on how fast the simulation runs. But, in fact, people sometimes think that a little machine on a desk or even a Sharp Wizard calculator is a different kind of thing than a big Cray computer. But in fact, they are the same kind of thing. You could calculate one of the greatest pieces of 3D graphics on a Sharp Wizard, given enough centuries to do it and enough external memory to put parts of the calculation.
Int: So this machine, even though it was built mainly to do scientific calculations, some people realised right from the beginning that it had just enormous potential.
AK: Yes, in fact the first person who seemed to realise it was Ada Augusta Lovelace, who was the sidekick of Babbage and maybe history’s first programmer. And she wrote in one of her papers that “The Analytical Engine weaves algebraic patterns just as the Jacquard loom weaves [flowers and leaves]”. And she understood that the generality of representation was the same kind as you can represent in books, which is namely the kinds of things that we can talk about. It wasn’t restricted to just numeric calculations, but it extended into the realm of general symbolisation of models.
Int: This amazing capacity has a very seductive quality about it.
AK: It is to us!
Int: I’m thinking of, say, in the late 1950s at MIT, some of the early hackers… Once people who realised the potential that they could make this machine do this, that, and the other… or Ivan Sutherland or whomever you mentioned…
AK: Yeah, I think two things happened. The Turing thing was there, and most of the early computer people either were mathematicians or had a lot of math in their background. And there were other formulations like that of Turing. Gödel’s theorem also was a way of building a universal machine. And there is a thing called Post’s production systems which is similar… they’re all studying mathematical problems, but they translated well to computer machinery. And most computer people were aware of them.
Then the second thing that had to happen was that there had to be some way of believing that the computer could get big enough and fast enough, so that anything that Turing said made any practical sense. You know, so you have to have both of those things. And what’s happened is that very early on — even in the 1960s but especially in the 1970s — people regularly would build very simple computers, and then use those simple computers that ran very quickly to simulate the kind of hardware they really wanted.
That’s what we did at Xerox PARC. Chuck Thacker who did the Alto, built a machine with almost no logic for that period of time. I mean in 1972 it had about 160 chips, two boards worth of chips. So that was very, very few gates. But it ran extremely fast. And it ran about five times faster than its main memory. And because of that the Alto could be many different machines. It was a Smalltalk machine when we ran Smalltalk on it. It was a Mesa machine when they ran Mesa on it. And it could take on different personalities. And I’ve always thought that was a really good way to go. Until you actually know what the biblical truth is on what your computer architecture should be, why not have a computer that you can mould into the kind of virtual machine that you want, right at the lowest level?
Int: This tractability, this mouldability gives the computer a romanticism that other machines lack, doesn’t it?
AK: Well, I agree on the romantic part, but of course, I’m fairly romantic about musical instruments. You know, most musicians are. Most musicians adore their instruments. So I think that, from my standpoint my romance is very much connected to the same way I think about other marvellous contraptions that we’ve made, including musical instruments, but [also] sailplanes, the kind of marvellous inventions that Paul MacCready makes, and so forth.
Int: But they don’t have this capacity of building a private universe do they?
AK: No.
Int: This is special to the computer…
AK: With a musical instrument you can build a private universe for somebody else. That is, you can make one, but it doesn’t have the tangibility that the computer has. On the other hand, it’s worthwhile remembering though that no matter what we build on the computer, and no matter what appears on the screen, it doesn’t have any particular sense to it unless some human is there comprehending it. So, it does have something in common with building a beautiful piece of music; that there has to — ultimately there is a listener, it might be just the composer, but what comes out has to, in some way, come back into the human sensorium and be understood.
Int: Now, you use a metaphor which is quite helpful in understanding programming: [the metaphor] of a puppet theatre. I wonder if you could give us that.
AK: Yeah, I think I got talked into that by some magazine. One of the traditional ways of programming on a computer is to think of the memory part of it as being inert like ingredients in a kitchen. And to think of the programs that you write as being like the recipes, and then the central processor is kind of like the cook who’s looking at the recipe book, and then goes over and stirs the ingredients and, if you’re lucky, you wind up with something good to eat. Another way of thinking about that is that’s like a puppet theatre, because the puppets are all inert and there are puppet masters going around. And the computer is a very energetic puppet master because there’s one, generally, [that] goes around twitching all of the strings of all of the puppets fast enough so that it seems like something real is going on. But another way of looking at programming is to say, “Well, why not let the puppets pull their own strings? So, we’ll let each puppet have, in effect, its own little computer within this larger computer, and we’ll let them be much more self-contained. There won’t be puppet masters from the outside”. And that’s called object-oriented programming. And the benefits of it are simplicity and ease of writing the programs.
Int: In procedural programs, if I want to tell my puppet master what to do, I have to list everything absolutely in the right order. In object-oriented [programming], I build my objects, give them behaviour and…
AK: Yeah, well, I think there’s a continuum from totally lockstep proceduralism to trying to deal with multiple processes, to having objects which have many of their processes inside of them and act much less sequential and so forth, to what’s coming in the future, which is something called agent-oriented programming or modular control programming — it hasn’t got a good name yet. But of something where the elements are much more like biological cells, that they’re quite self-contained. They may spend 90% of their energies just maintaining their own equilibrium, and maybe only 10% of them contributes to the larger system as a whole.
Int: And these are presumably even higher levels of virtual machine, as it were?
AK: Yes, that’s a very good way of thinking about them.
Int: Every time we go further up to a higher level of virtuality that suits us, it generally means that the computer has to work even harder.
AK: Yeah, oh, I don’t think it’s… The computer is only working at one level. Any given computer. It’s always executing two million instructions per second. So you can’t make it work harder. But it’s like, if you have a 5 horsepower go-cart and you put it up various grades of hills, there will be a hill eventually that it won’t be able to climb. It’s always putting out at 5 horsepower and it needs 10 to get up that particular hill. And that’s what happens: things start slowing down as you put more and more burden on an old-style processor.
Int: Can you give me an idea of what’s going on if I’m looking at a screen and I want to do something like open a file on the Mac desktop… Some notion of how many operations might be involved?
AK: Well, you can figure it out because the typical Mac these days executes about two million instructions per second. Well, say, opening a file it has to go out to the disk, so it’s complicated because now it depends on the speed of the disk moving stuff back and forth. But suppose you’re just in the Multifinder and you’re in this window doing something, and you put the mouse in another window, and the window comes up to the top of the screen, you can easily [figure it out]; you just get out a stopwatch — if it’s a big, lumbering thing and it takes about a second to rebuild the screen, then two million instructions have been executed. If it takes two seconds then four million instructions have been executed. And it’s always executing that many instructions per second even when it’s just idling.
Int: There’s immense, monumental complexity going on underneath these simple things…
AK: Yeah, and often the complexity is something that is a by-product of the way the system was programmed rather than being intrinsic.
Int: I want to move on now — I’m going to move on to your current work with the schools, but I want first for you to talk a bit about the idea of tools and agents, historically, and so forth.
AK: Okay. The way I think about tools and agents is, you need some sort of perspective to think about them. And I think about them in terms of the way we’ve extended ourselves over the last several hundred thousand years. And when you say “extension” to somebody, they almost always come back and say “tools”. And indeed there have been levers and wheels and physical tools, but there have also been mental tools like language and mathematics. And I think of tools as being extensions of the gesture, as a way of manipulating things. You’re manipulating symbols when you’re dealing with language. You’re bringing things that are hard to deal with into your power via your hand or something like your hand. So the ‘M‑word,’ to me, for tools is manipulation. And tools are things that you watch while you manipulate them.
And then the other main way people have extended themselves is by getting other people to take on their goals. Mumford called this making megamachines. He said that for most of human history most machinery made by humans has had other humans as its moving parts. So we make cities and cultures, and there are groups trying to do this and groups trying to do that. And, there are fewer goals in those kinds of groups than there are people. They’ve taken on each other’s goals, traded off, in one way or the other, and they are communicating. And the kind of entity that can take on your goals and act in your behalf, we call an agent. So an agent is something that watches you, and you manage it. So the ‘M‑word’ is management for agent, and manipulation for tools.
Int: You’re saying that agents have generally been people up to now, whereas tools have been…
AK: Yeah, you could say that a shepherd dog, or maybe a horse, maybe a thermostat is something you have to work really hard with… to build a thermostat to get it to take on the goal of what temperature that you want. But, by and large, they’ve been people up to now. And the interesting thing about computers, when you’re building agents on them, is the agents don’t have to be as smart as people, just like a thermostat does not have to be as smart as a person to be useful. The main thing you want it to do is to be able to take on some collection of goals, and to be able to deal with those goals while you’re not around.
Int: So an agent has to have some artificial intelligence in it?
AK: Yeah, if you like to use that term. You could call it flexible competence and make it sound a little less loaded.
Int: So how would this work? You feel that the computers of the next 10–20 years, in addition to having the rich interfaces we have, will have personal agents as well?
AK: Yeah. One of the ways I think about looking ahead into future is to try and find analogies that might actually make some sense, and also to look for driving forces. So, one of the driving forces for the PARC-type user interface came out just from there being inexpensive integrated circuits around. You start getting a proliferation of computers that are inexpensive enough for people to buy, and all of a sudden the kinds of people who might want to use computers changes completely, and so all of a sudden you need a much easier-to-use user interface.
There is a driving force now to do something, because it isn’t just graduate students anymore. To me the driving force for agents is pervasive networking, because the techniques used on the Macintosh don’t work well when you’re connected up to a trillion objects scattered all over the world. You need something looking for potential objects that will further your own goals. And you need that something to be looking 24 hours a day. So we think that what we’ll have is 10, 15, 20 or more little agents — many of them not particularly intelligent but able to flexibly take on a goal that we have.
Like, an example of one is an agent that goes out and finds you the newspaper you’d most like to read at breakfast every morning. So, all night long it works. It can touch dozens of different news sources, the Associated Press, New York Times and so forth, looking for things that are relevant to you. It can go to other sources for getting photographs and so forth. It can do the news-gathering with a particular interest in the kinds of things that you have been involved in. A headline could say, “New fighting in Afghanistan”, or it might say, “Your 3 o’clock meeting was cancelled today”, because news now could involve your own electronic mail. A sidebar might say, “Your children slept well last night”. And this is an interesting example of an agent because it’s one that was built about ten years ago. It did not require a large amount of intelligence in order to work. Its major strength was its ability to work 24 hours a day while you weren’t there, and with limited ability of matching against what you said you wanted and what it thought you wanted. It could do a great deal of useful work for you.
Int: There have been a number of revolutions in the history of computers so far, but we’re thinking now… most people think of the computer as a standalone, desktop object, right? Where do you see — tying this in with your Dynabook concept, what you’ve been saying about networking and agents — where do you see the next thing taking us?
AK: The way I think about that is these three very different ways of relating the human to the computer. One is this institutional way of the time-sharing mainframe; one is the desktop way where you control all the stuff; and then the third way is the intimate way, which is the Dynabook way, which is continuously connected into the worldwide informational network.
Int: What is a Dynabook exactly?
AK: Well, a Dynabook is sort of a figment of imagination. It was a Holy Grail that got us going. It was a cardboard model that allowed us to avoid having meetings about what we were trying to do. It was a lot of different things, but it was basically a service concept, not a box concept. So, there were actually three physical renderings of the Dynabook we thought about: one was the notebook; one was something that went in your pocket that had a head-mounted display and glasses (as I had worked with Ivan Sutherland’s head-mounted display in the 1960s); and then one was Nicholas Negroponte’s idea of the sensitive wristwatch that in the 20 years future or so when there is a network wherever there is an electric plug in the wall, then your user interface will follow you from room to room as you go. Everything has become pervasive: you don’t need to carry a big computer (or even a tiny computer) around with you. So, the whole idea behind the Dynabook was the kinds of service and your relationship to it, which should be intimate, casual, mundane. You should be able to aspire to the heights on it, just as you can when you learn English, you can aspire to the heights of Shakespeare, but you’re not forced to do what Shakespeare did every time you use the language. So, this idea of having a nice, connected ramp without lots of bumps in it — as Seymour Papert likes to say, “Low threshold, no ceiling”.
Int: Now, given you have that concept of the future, your current work at the moment at the open school here in Los Angeles with children who may inherit such a marvellous computer, what are the objectives of this work?
AK: Well, it’s several. One is that Apple traditionally has been a company very interested in the educational process and helping children in schools, so we do a lot of things in this school that have to do with thinking about how schooling might be in the future. Then, specifically, one of the things that we do is a project that’s been going on for about four years now, to try and help find ways that will allow children to be able to write in the computer as fluently as they can now read using the Macintosh user interface. And what we do, since artificial intelligence is coming along, we’re trying to find ways to both understand artificial intelligence and understand how to program it by putting together a set of tools that allow children to do the kind of artificial intelligence programming normally only done by adults.
Int: Now, this is done through a rather ambitious simulation.
AK: Yeah, well… You think of adults try and simulate humans, and so forth. Humans are pretty tough, nobody has done a simulation yet. I’ve always felt it would be a good idea to work our way up through the food chain and start off with fairly simple animals, see how they interact with the world. That’s something that children are interested in. And so quite a few years ago we got the idea that it would be really great if we could give children an environment where they could create ecologies to study animals and plants, and then build those models into the computer, and see if those models reflected what they thought they understood from the real world. So, there’s a constant comparison between the real world and the computer model. The school has torn up part of its playground to make a Life Lab, which has both animals and plants in it. The classrooms have animal cages and aquariums, and so forth. And so there’s a lot of real animals to study. And then we also have Macintoshes with a new system we’ve designed called Playground, that tries to bring some of the techniques of artificial intelligence programming to 8, 9, and 10-year-olds right now.
Int: Are you trying to achieve literacy — computer literacy — in children? Are you thinking about that?
AK: Well…
Int: Is that what you mean by it, or…?
AK: Yeah, I’ve never… You know, in one sense I think so, in the sense that I’ve always wanted to close the loop at least with something that was like reading and something that was like writing. And right now, the something that’s like reading is using the Macintosh user interface language to deal with 9–10,000 applications that are out there. That seems to work reasonably successfully right now. And the equivalent of writing should be something that allows children to aspire to the same kinds of things that are built on the Mac. Now, they may not sit down and do Aldus PageMaker or something like that, because that’s something like a large play or something. But they should be able to see a continuity between what they’re doing and these tools that adults make for them.
We want to do something like what Dewey was talking about in the last century, which is, he pointed out that in most of the ages of mankind the games that children played were serious simulations of what the adults did. So the African child practicing with a spear, or the Eskimo child learning how to kill birds because he’s eventually going to have to go and kill seals for food, is doing something that is content-rich relative to the adult world. But the 20th Century child dressed up in a nurse’s suit and playing nursie with her doll has been cut off from the content part of that adult activity; only the form is left. So the kids are very disenfranchised from most things that are happening to adults. And one of the things that was noticed right away with computers is that when you put a child on a computer, they know they’re doing the real thing, that they can see instinctively the continuity between it and the other things that are going on — actually much better than adults do.
Int: The computer, many people claim, is going to have a big role in saving American education, and so forth. And this is a worrying thing, because you said yesterday that the computer was a meta-medium, so it can be what we choose it to be…
AK: Yeah. The story I always tell is, imagine the parents were afraid that their children wouldn’t make it in life unless they were musicians, and the state legislature said, “Well, okay, we’ll put a piano in every classroom, but we don’t have enough money to hire musicians, so we’ll give the existing teachers two week refresher courses”, and music doesn’t get into the classroom. And I think we have a very similar problem when we want to think of the technology as being the magic ointment. Musicians will tell you, the music isn’t in the piano. If it were, we would have to let it vote. So, at best what we have is an amplifier, and often these things that could be amplifiers will turn people away. Pianos often turn people away from music rather than turn them towards it.
So, I think the most important thing is to have for people who want healthy schools is to have parental involvement, because down deep it’s the value system that the children pick up about what’s important in life, that they mainly get from their parents, that is going to condition what they do with their time. It’s hard to learn things in school. There are lots of things going on. School, to me, is basically a resource for finding out that various things existed that you didn’t think existed. But as far as learning them, most of the learning I think is done outside of school. And what the child decides to do outside of school with his time is going to depend on the value system. Once you have that going really well, then it’s fairly easy to use all kinds of technology, because then you will simply amplify out from this interest in getting stuff happening in here.
Int: Interesting that this medium should come along at a time when the previous important medium like writing… Everyone is so concerned about literacy in the schools. Do you see it as a solution, or possibly a worse problem?
AK: Well, I think that was one of McLuhan’s tongue-in-cheek jokes — that we’ve had all these great inventions like the book, and they’ve hardly affected education at all because, if you go into most schools in the northern hemisphere, you find 30 humans writing down what another human is saying, and that’s what was going on in Oxford in the 12th Century. So, you know, where is the book in all of this? So I think that the kinds of social whirlpools that exist when you get different kinds of humans together, like teachers and children, are going to have a lot to do whether technology gets used at all. I think the most important aspects have to do with areas of control and other kinds of things which are, theoretically, outside the domain of the education — but a lot of school is about controlling the kids.
Int: Now, if we’re looking into this future, some of the things that might be important, can we count on the hardware continuing to improve for another decade?
AK: Yeah, yeah, we can definitely count on the hardware continuing to improve another decade, and probably more. But, I mean, this is just extrapolation: the current kinds of hardware that we know how to build has a very stable outlook for the next ten years.
Int: What about key ingredients from software from artificial intelligence? Do you think projects like the Cyc Project are going to be vitally important?
AK: Well, Cyc Project is one of my favorite projects partly because it’s done by one of the smartest guys in the U.S. in computer science, and partly because it’s one of the hardest projects that anybody is trying to do right now. There are not a lot of researchers working on what I would call really frontier difficult projects, but this is one of them. And its success will be and is about turning up new ideas for representing things. Whether the system is actually able to turn into what its design goals say it is, which is a model of human common sense, I don’t think it’s nearly as important as the wake that it’s throwing up. When you got a smart guy working on something really hard, and a bunch of people being ingenious about it, you’re always going to get good things.
Int: Why would it be important to incorporate common sense, a representation of common sense in future computers?
AK: That’s a good question, because common sense is frequently wrong. Scientific discoveries of the last 300 years have been against common sense. But, whatever common sense is, it’s a kind of an underlying fabric for getting between things that we know in much higher detail. So, one way of thinking about it is, there are things, little things that we’re expert in, [which] are like islands, and then there’s this ocean that we can paddle around in from island to island. There’s a way to get from one island to another. And the problem with most expert systems up to now is that they’re an island completely surrounded by a cliff. If you go anywhere off what it’s good at, it just drops you completely. There’s no way of paddling to the next thing. And as I said yesterday, somebody (Picasso, I think) said, “Art is not the truth. Art is a lie that tells the truth”. And common sense is not the truth. But it’s a lie that provides a continuous fabric to work around in that we can then delve deeper into.
Another thing about common sense that’s kind of interesting is that it might be possible to use the computer to enlarge what we think of as common sense by giving us sensory contact with things we’ve never been able to have sensory contact with before, like things that are a million times smaller than us. Because common sense I think has a lot to do with the sensory domain and reasoning from things that are on our scale. Science, in a very literal sense, is non-sense because it’s outside of the sensory domain. Almost everything that happens in science is very far, you know… [slaps his leg] Common sense says this is solid. But science says it isn’t. Common sense says the sun is going to come up tomorrow morning, and science says no, the Earth is turning. And yet we still say, “What time is sunrise tomorrow?” So I think the importance of Cyc using common sense has a lot to do with that, regardless of whether we are scientists or not, we have this one way of knowing the world, rightly or wrongly, that is very comprehensive and gives a sort of universal way of weakly getting from one topic to another.
Int: What do you think the main legacy of artificial intelligence is?
AK: You know, some of my favorite crazy people are AI people. AI in the 1960s was once defined as, “All that stuff we don’t know how to do yet”. And to some extent it’s been a moving target. Things that were AI problems in the 1950s and 1960s are now taught as part of computer engineering courses. But as far as the AI as something mimicking, in a strong way, human intelligence, we’re very far away from it. And so it’s a good goal. It gives you something to reach for. And I think for people in the field who have some biological motivation to their interest, it’s a good goal, because it has partly to do with understanding more how we do it, wonder if there are alternate ways to do it, can you only do it the way we do it? At what level do we do it? Do we have to do it at the absolute neuronal level? Do we have to simulate every neuron in a human brain to get artificial intelligence, or is there a higher level that we can do it?
And those are good questions to ask because, if you look at the way e.g. biochemistry is done by Nature, it is an appallingly inefficient, very low-energy transfer… The absence of some watcher from the outside saying, “Oh yeah, it would be much simpler to do it this way”. So that, for instance, the way we do chemistry in a lab and the way Nature does biochemistry is completely different. We do it much more efficiently. Nature does it much more ingeniously because of the way it’s contrived. And you can learn a lot from looking at the comparison between the two. And so there is a lot of reason to expect that you don’t have to go to the neuronal level to be able to do the kinds of things that we do. But nobody knows what level you actually have to go to.
Int: You mentioned you worked with Ivan Sutherland on the heads-up display. One group of people argue that the stage we’ve got to with the human computer interface — 2D, 2.5D — is just a stepping stone. That beyond that there is this virtual reality. What’s your view on that?
AK: I think many people will enjoy virtual reality, since many people don’t enjoy the current reality. Television is a kind of virtual reality. And I think things that go further in that direction will be very popular with a lot of people. I think the best thing about virtual reality is that you can deal with these things outside of the normal senses. You can take a trip down into a cell and see how incredibly agitated the thermal properties are down there. All the things that you only can read about now in terms of symbols, you can actually go there and get a much more kinesthetic and visual hit on doing those things. I think the use of it in fantasy will certainly happen. But if you look at what you actually have to do to get good dramatic situations in the theatre, then it’s going to be a while before something good can happen in a movie that’s partly being generated by your presence. On the other hand, if you look at a typical Arnold Schwarzenegger shoot-‘em-up, then those will be easy to do. I forget, what was the name of that movie? I can’t remember the name of that movie that he just…
Int: Total Recall.
AK: Total Recall. You know, somebody said, “If you like road accidents, you’ll love this picture!” [laughs] And that kind of stuff where you have five people you have to kill every 30 seconds or so, is very easy to set up in virtual reality. I’m sure that a large percentage of the population will enjoy it.
[…]
AK: …do a computer simulation that kids could understand from that one.
Int: That’s the interesting thing, clearly the fact that you can make thought experiments real. So, many of the classic thought experiments of Einstein and Bohr, you could do, [make real].
AK: Yeah, and of course you have to be careful because simulations are lies, in a sense. Then there is nothing that says the simulation has to be like real life. I mean, there have been plenty of thought experiments that are wrong. And most of the great scientists have been good guessers. So you can also set up simulations of situations that don’t have anything to do with the physical world… So you can delude yourself as well as help yourself along.
Int: And you think that might be one of the dangers?
AK: No, I don’t think that’s a danger. I think that anytime people try and make models, try and look at their beliefs from different points of view rather than just one point of view, I think it’s good.
Int: If we look back — I know you’ve thought a lot about the problems of predicting what is going to happen — If we look back at the history of the computer, it seems that almost everybody has been quite seriously wrong at every stage without very few exceptions. What lessons can we learn, if any?
AK: Well, there are lots of different ways of doing prediction but the worst one in the 20th Century has been extrapolation. So just because something is X, and 10% in some other direction of X gets you here, and so forth, doesn’t mean a thing. It’s like, if a computer could do so and so, it would have to be the size of the Empire State Building. Well, that was when people’s imaginations were limited by vacuum tubes. So the extrapolative way, I think, is out. But the reason the predictions that we made in the late 1960s were so good, and the reason Bush’s predictions in the 1940s were so good, had to do with a completely different way of predicting — which had to do with thinking about things that amplify human endeavour. And the amount of horsepower available that is interested in making human endeavour be amplified is very, very large. So, as you can hook into something like, say, if you can say, “Oh, the computer is a medium”, then all of a sudden you start seeing what the powers of amplification are, and you also start getting ideas about what to do next. You look at, say, Freud’s model of human mentality, which is a good one, but it’s all about drives. That doesn’t help you much in doing user interface. You look at Bruner’s mental model, which is about different ways we have of knowing the world, and all of a sudden you get ideas.
Int: It’s interesting, isn’t it, that one of the arguments that said there would never be many computers was the one that we would never be able to find enough things for them to do.
AK: Yes, and another one of those things is the… Again, I was looking at the kinds of things that computers are doing now. And as people used to say, “That’s right, you numbskull”, you know, but it was all the new things that they can do. It’s not, “we can do payroll on the mainframe” when the personal computer came along: it’s all those things we can’t do on the mainframe, like spreadsheets and desktop publishing, and so forth.
Int: You see the future of the computer as becoming totally unremarkable.
AK: Well, I would hope so. Nowadays, well, ten years ago, if you went into somebody’s office and you didn’t see a phone, that would have been remarkable. It was the absence of the thing that would have been remarkable. Nowadays if you go into somebody’s office and you don’t see a phone, you assume they’re wearing one, but you do not assume that there is no phone in a person’s office, because it’s something that is noticeable when it’s absent. And the computer right now is still more noticeable by its presence than its absence. When you go somewhere and somebody doesn’t have a computer on them — that becomes a remarkable thing. Then I think the computer will have made it. Its destiny is to disappear into our lives like all of our really important technology, the things that we don’t think of as technology like wristwatches, and paper and pencil, and clothing and all of those things. I think the computer’s destiny is to be one of those.
Int: And it will disappear into our lives, embodying all previous media, or many previous media as it goes?
AK: I think so. I think we have to be careful because when you simulate one thing by another, you usually give up something. And anybody who has ever seen the Book of Kells for real realises what you don’t get from photographs, realises what you don’t get from printed books, and also realises what you do get from printed books: that the compelling charisma, the transcription of the oral event that was a book like the Book of Kells, is completely different from the alienating regularity of machine type. And both those things have their place in the world. You’d hate to get rid of one completely and say, “Well, we’re replacing it with the new”, because I don’t think it works that way.
I mean, I’m building a rather large Baroque pipe organ even though you can ‘simulate’ them on synthesisers and stuff today, and the answer is: you can’t simulate them. You can’t get all of the stuff yet, and even if you could, even if you could prove beyond the shadow of a doubt that the waveforms from it [were identical], you still don’t get something that looks as neat. And so I think if you include all of our senses into an experience, that when you simulate something as you always do in science, what you’re saying is, “I am going to give up this in order to get that, and that’s my trade-off right now”. But a person who says, “I’m going to use this, and I’m not going to give up any of that stuff”, is just fooling themselves, because there isn’t a complete interconvertability between one medium and another.
Int: How would you rate the computer? You’ve studied a lot of human history: is this something we’ve been privileged to live through? Is this really a very remarkable development?
Kay: I think one way of rating the computer is to say it’s definitely a thing like the printed book. It is definitely in an unremarkable stage — like 30 years after Gutenberg — and almost certainly, if its promise is realised and it’s not just turned into television (because that’s one of the things it can simulate as well), but if it can deal with all of the range of things it can deal with, and people use it to express all of those things, then it very likely will have the same kind of effect on our civilisation as the printed book did. Whether it has that effect on any given person though, is a completely different question. Because, as people have noted, the Renaissance has come and gone and we have what we are pleased to call civilisation now, and a large percentage of the population — not just in Third World countries but in our own country — have never touched the 20th Century as far as its ideas are concerned. And in spite of all the libraries with all the books, and all the things, and what books have done to us, a very large percentage of people have never been carried along with them. And that is very likely to happen with the computer.
Int: So we haven’t really succeeded with the last major medium?
AK: Yeah, it’s probably the case that we never succeed with those things, that civilisation gets transformed and a certain critical mass of people get transformed by it, and they are the ones who transform the civilisation. And then, for one reason or another, a large number of people don’t get transformed by it. Another way of thinking about is, if you take a look at what the average person who has not been transformed by it, but has gone to college, today thinks about the world, it is a little bit better, a little bit richer, I think, than what people thought about the world in 1000 AD.
Int: But there is a problem with this literacy thing, you said, because I know you think of television as like a medieval stained-glass window in some ways… That actually you don’t need to learn anything to do it, is that the point…?
AK: Well, you have to learn something, because we know that when D.W. Griffith first invented close-ups, and moving cameras, and montages and stuff, that the audience was a little bit startled. But it didn’t take long: one pass through it and they got the idea of what was going on. And so the amount of learning in the visual domain is pretty low compared to what you have to do in doing reading and writing. And that is a big barrier.
The biggest problem though, I think, is that many people believe that there is an equilibrium between the two media, that what you can say in a book, you can say on television. And all of the evidence is against that. What you can do with television are some very important things: you can get people interested, you can give them an emotional hit, you can get them to know somebody in a way they didn’t think they could do it before. You can maybe get them to look, to be interested enough to look deeper. But its very strength is its weaknesses. Its strength is its involvement, its weakness is its involvement. Because in order to think scientifically about the world, you have to be romantic, it’s true, but you also have to be able to step back and say, “Gee, I wonder what else it is. It looks this way, but I wonder what else it is. I wonder what else is going on”. And I don’t think television gets people to do that kind of connected thought away from the dominance of the senses.
Int: There’s also a thing you write about called the amplitude problem — that some media require more concentration than others, don’t they? Written media do, and some forms of television, some forms of games and such don’t. The question is, will the computer disappear into the television, or the other way around?
AK: I’m not sure whether it requires more concentration, but it may very well be that like… You know, some people read well and it’s nothing for them to read a book a day. And [for] other people this is a big deal. It’s a struggle, and what likely is going on is that most people never learn to read very well. And so the amount of concentration they have to put into the mechanics is what defeats them with the amount of material that has to be read. And I think it’s much more like that, because it’s remarkable how much concentration you have to put into something like tennis until you learn to play it, or how much concentration you have to put into music, on the mechanics, until you actually get fluent at playing. But then, once you’ve gotten to that place then the hours go by without even realising, because you’re deep into what the content of the medium is.
Int: Last question: what has surprised you most about the history of computing?
AK: What surprised me most? Well I think the thing that surprised me the most is how long it has taken to get ideas out of laboratories into the commercial world; how many different kinds of inertia there are, both for good reason and for bad reason. But just the sheer amount of time where a decade after an idea is a very short period to see it emerging in the commercial world — that is certainly surprising to me, because most of the scientists who work on these things work on them because they were obvious. They were so obvious that they just want to make them. And for something that was revolutionarily obvious to have to go through an evolutionary stage that may take 10 or 20 years is quite surprising.
Int: Do you think there is any way of cutting down that period?
AK: I don’t see it right now, because I think it’s a question of having people grow up being interested in multiple points of view rather than being disturbed when they’re shown something outside of their single way of looking at the world. And our civilisation is very prone towards single-minded ways of looking at the world. We come from a monotheistic religious background, which says there’s a right and a wrong, and you’re either for God or against God, and so forth. And these attitudes trickle over into our reactions to everything. If what we’re doing now is right, then something that’s different from it can’t possibly be right. The Japanese seem to be a little more flexible in some of those scores. They have several religions existing side by side in Japan, and many of the people adhere to several of them at once. They don’t see any big conflict. And I think any civilisation that can treat ideas as more interesting in an array than as treated singly, is going to make it into the future.