TRANSCRIPT: UX Week 2010 – Chris Noessel & Nathan Shedroff
Make It So: Learning From SciFi Interfaces
Nathan Shedroff: Hi, I’m Nathan Shedroff, and I’m the program chair of the MBA in Design Strategy program at CCA, which is about three blocks that way, right behind me.
Chris Noessel: And I’m Chris Noessel, one of the principal designers at Cooper Interaction here in San Francisco.
Nathan Shedroff: And today we’re gonna give you a taste of some research that we’ve been doing for about three years. And by “research,” I mean we’ve been watching lots of science fiction for three years. We’re actually uniquely qualified to take on this subject ’cause we’ve been actually studying it for our entire lives.
[Audience laughter]
And this is all for a book that hopefully will be done in the spring or something like that.
Also, before we start, this is a really good time to tweet “Oh, my god, I’m so glad I stayed to the end. These were the best presentations of the conference.” And if you don’t like me now before saying that, then do it during Mark’s session.
[Audience laughter]
Okay, so let’s get going.
Chris Noessel: Excellent. So as Nathan said, we’re publishing a book. We do have some notes available, and I know URLs aren’t easy things to write down. We should tweet it.
Nathan Shedroff: Someone tweet it.
Chris Noessel: I’ll tweet it later.
[Inaudible audience comment]
Chris Noessel: Okay, thank you.
Nathan Shedroff: Thank you.
[Inaudible audience comments]
Nathan Shedroff: Tweet shout-outs to the audience. And so this is just a couple of the notes, but please don’t read them now, ’cause we’re much more entertaining live.
So as Peter introduced, our investigation is into the relationship between design and science fiction, but those are giant, monolithic words. And so – do I need to provide filler, or are we close? Rock on. So we should answer which design and which sci-fi, because they’re pretty big. For design specifically, we’re looking at a number of different things over the course of the research: interaction design, industrial design, and information design. But our focus is really on interface design in science fiction, although of course it bleeds out into all those other things since they’re not entirely easily separable.
Heading to the other side, I’m gonna talk about science fiction again. It’s a giant genre, so what specifically are we talking about? Since we’re looking primarily at interfaces, that means that we need visual science fiction, so there go all your favorite science fiction novels. Sorry about that. Since we’re also looking for interfaces, we need that visual representation to be moving, so there go all your favorite graphic novels. We also need it to be consistent from depiction to depiction, and so there go all your favorite hand-drawn animations, because it’s actually kinda easy to make a mistake about where this dial was or where that lever was.
And what that leaves us with is really screen science fiction, television and movies. And while we suspect that a lot of the research that we’re doing can sort of bleed out into other genres of sci-fi, we’re not focusing on it. So point one is, your favorite science fiction will not be part of this presentation.
It’s a danger when you present to geeks, like they’re gonna say, “Wait a minute, what about Episode 7…?” Okay.
[Audience laughter]
So what we’re gonna be presenting to you today is sort of a larger theoretical model underneath this notion of influence. The center ring that you can see here is about individual inspiration. Of course, design isn’t a big thing, and science fiction isn’t a big thing. It’s made up of people, and those people draw inspiration from the things they see in the world. People in Hollywood can take a look at something in the real world and go, “Oh, wait, I can totally use that.” Same thing, us designers can take a look at science fiction and say the exact same thing. But when we sort of back up and we take a look at the larger patterns, design influences science fiction by establishing the paradigms that science fiction extends. That’s pretty dense, so I’ll leave it hanging there for a second.
Then when we head down to the bottom of the model, we’re gonna be talking about three other sort of layers. The first is, science fiction influences design by setting audience expectations. Where’s my jetpack? It’s a fine question. A second way is by reminding us of the social context in which these things occur. Of course, science fiction is kinda all about technology evolving, but the humans involved in that are typically not any different than the humans who are watching it, and we need to be reminded of that sometimes. And then the final way is by actually proposing a paradigm. The people who sort of make science fiction are actually prototypers. But they’re just not aiming at something to be realized fully; they’re aiming to tell a story.
So now what we’re gonna do is sort of zoom into these things. And this first part we’re gonna talk about is that first notion I talked about, about establishing the actual paradigm. To do this, we’re gonna zoom all the way back to the beginning of science fiction.
Chris Noessel: Yeah, this is the first science fiction film, Le Voyage Dans La Lune. And we’re not gonna show you a clip here. In fact, we don’t have time to show you all the clips in even this presentation. But it’s worth noting here that there really are no interfaces, certainly as we’re accustomed to describing them. And that’s because this is actually the original direct manipulation. If you wanna open a door, you push on it; it opens, right? There’s not even knobs, usually, at this early age of science fiction. So it’s interesting to note that that’s sort of where things started, and keep that in mind as we show you what happens over time.
Nathan Shedroff: So now we’re gonna jump forward in time a little bit. Actually, there was this weird gap between Le Voyage Dans La Lune and the next film, which is Metropolis, Fritz Lang’s masterpiece from 1927. I’m gonna show you a little clip, and in this clip we’re going to see Joh Fredersen entering his office and checking this large, beautiful wall-mounted telecommunications device to call a fellow in the Lower City. I’m gonna call your attention – I mean, there’s loads to love about this clip. But I’m gonna call your attention specifically to the way that he selects the channel for the video.
[Video begins playing]
So he comes in. He checks messages by ticker tape. He comes and turns a weird dial to engage the visuals. And then he has to tune the channel. You can even see them sort of overlapping slightly, so he’s just getting the tuning correct. Then he picks up a handheld telephone, and we get to sort of meet this wonderfully hammy over-actor in the Lower City. And we’re gonna pause right there.
[Video pauses]
‘Cause he’s fantastic. So, clearly, what’s going on here is that the audience were coming in with a set of preconceived technological notions. They understood ticker tape, they understood radio, and they understood film. And so what we saw in the video there was, “Oh, it’s going to be like film, but you can tune it like a radio.” Weird concept to us now.
But we’re gonna jump ahead to the next sort of serious piece of science fiction, which occurs 25 years later. Buck Rogers in 1939 was a serial. And we’re gonna take a look at another interface of a similar nature, which is a large wall-mounted communications device. But pay attention again to the way that they change the channel.
[Video clip plays from Buck Rogers]
So I’m gonna pause you there, although we’re skipping a lovely part where they have to go into a different room to engage in radio contact.
Chris Noessel: The radio room, right?
Nathan Shedroff: Yeah.
Chris Noessel: ‘Cause that’s what people knew.
Nathan Shedroff: [Laughter] Radio room. So these were largely similar interfaces, but the controls were completely separate. Remember, you had to sort of tune the Metropolis interface, and you just turned a knob in order to change the channel in Buck Rogers.
What happened? Television happened in between. Suddenly a new paradigm was sort of let loose on us consuming public, and we didn’t have to tie our notions of the interface to radio. We could now just say, “Oh, it’s gonna be like a TV, but on my wall.” And these sorts of things happen all throughout sort of the chronological history that we’ve developed, where you can actually sort of see the influence of the real paradigm on science fiction.
Chris Noessel: And as technologically as we think we are today, this still happens. Here’s Jurassic Park in 1993 I guess it is, where if we were to play the whole clip, this is the famous scene where she sits down at the computer, and the velociraptors are trying to get into the lab. And she sits down and says, “This is Unix. I know this.” Right? But there’s all these shots to the keyboard, and especially the mouse, because they have to establish for the audience that “Oh, it’s a personal computer,” because personal computers are fairly new, not completely mainstream, and they have to have that touchstone so people know what’s going on.
Nathan Shedroff: Oh, zoom back. So, again, the sort of shorthand version of this talk, so that – trust us. Now we’ve established that design establishes the actual paradigm that science fiction extends, and we’re gonna go into that sort of lower ring now to talk about individual inspiration briefly.
With individual inspiration, we can see that individuals see something in the real world and they go, “Oh, I could totally use that in my science fiction.” This first example that we have is with the Visible Human Project, which was first started in 1989. If you’re not familiar with it, it’s fascinating and a little morbid, where people who’ve donated their bodies to science get frozen after they die and then sliced very thinly, and then the slices are put together as scans. And then there’s an online interface where you can get through on that little green rectangle and sort of zoom in and out and see all the sort of beautiful internal shapes within the body.
I think it’s pretty cool, and somebody at Fox Movies thought it was pretty cool too, because in X2, the second X-Men movie that was released in 2003, there is a scene where a guard is approaching Magneto’s plastic cage, and he gets scanned. And there’s some animation on the screen that that guard sees, and in the lower right-hand corner, you can actually see the Visible Human Project, just colored blue, so you know it’s science fiction.
[Audience laughter]
So we caught you, Fox, and we’re gonna call you out on it. And this doesn’t happen just sort of with technological devices. Damien Hirst’s sort of notorious for chopping up animals and then putting them in formaldehyde boxes. And this showed up in The Cell in 2000, but of course, science fiction, not content to replicate the real world, actually had those tissues sort of pumping, and it was kinda gory and beautiful. But we’re not gonna show those clips.
So that’s the upper arc and actually an interesting one I think mostly for sort of makers of science fiction, although we’re fascinated by it as well. But now we’re gonna go to this lower arc, which is more pertinent I think to us as designers.
The first ring we’re gonna go through is similar to the one we just saw: individual inspiration. And in this case, we’re gonna go back to the first X-Men, which was released in 2000. The reason we’re showing this movie is because a fellow by the name of Douglas Caldwell was successfully petitioned by his teenage son during the summer to go see this cool movie about superheroes. Douglas Caldwell is no fan of superheroes, but he wanted to bond with his son, so he went. But to his surprise, while sitting in the audience, he saw the solution to a 2,000-year-old problem.
[Video clip plays from X-Men]
So I’m gonna pause there. It looks like a ton of those pin boards you can sort of smash up against your face, but it’s computer-controlled and gives sorta this lovely narrative about what’s about to happen in the film.
The reason it was so intriguing for Douglas Caldwell is because he worked for the U.S. Army Topographic Department. His job was to send 3D maps out to field generals so that they could plan maneuvers in a 3D space. Turns out generals are really good about thinking in 3D, and kind of need to. But it was very error-prone. Things could break on the way. What if you sent the wrong map? You’d have to go back to sorta Step 1. And he saw here “Hey, all we could do is outfit those generals with a board like this, and then we’d just send them the data, and they could see any terrain that they happen to be around.”
So he used that idea, built an RFP, and within three years had a working demo called the Xenotran, built by a company named XenoVision. Here is an example of that very box. This is with the top that’s down. If you get a picture of that top open, you can see the array of pins, all right there, completely computer-controlled just like you saw in the movie, but they did it one better. What that top contains is a thin latex layer that then gets vacuum-sealed on top of the pins, and then it actually sort of gets this smooth surface that the pins provide, and they can actually project satellite imagery down.
So in our interview with Douglas, he actually said that he referenced the movie several times in the RFP, and during the course of the development of this device, he would reference it as well. It’s like not an accidental influence; it is a deliberate one. He’s like, “Yes, this is the thing I want to build.” And once they had it – and this is a video sort of panning across it – they were able to not just do terrain, but they have modeled tsunamis and the surface of Mars and all sorts of really beautiful stuff, and it’s a great example of this sort of individual inspiration.
Chris Noessel: Okay, so let’s talk a little bit about expectation. It’s probably no surprise to people in this room that when you put things out there, whether it’s a prototype or whether it’s a sort of speculative fiction, it sets up a lot of expectations. In fact, every time we go into user testing, we know that there are expectations that users bring because – and get triggered by what they see on the screen. In fact, often, the finer the resolution and the greater the finish, we know from user testing, the less people wanna comment on it ’cause they don’t wanna tell you it’s bad if it’s finished, right? And sometimes the rougher it looks, the better and in fact completely different kinds of feedback you get. Now, we’re gonna cross over robots because we’ve seen a lot of robots, and actually, they’re not as interesting in this context. We’re gonna go right to Star Wars.
Nathan Shedroff: Star Trek.
Chris Noessel: I’m sorry, Star Trek.
[Video clip plays from Star Trek]
Kirk gives it to IT.
[Audience laughter]
[Video continues, then ends]
See, they’re on AT&T as well.
[Audience laughter and applause]
Sorry. [Laughter] So what’s interesting about this – if you look at the date, this is 1966. This is about as popular and mainstream in science fiction as you could get at the time. And exactly 30 years later to the year, we see come out of the market these StarTAC flip phone –
Nathan Shedroff: Thinly veiled.
Chris Noessel: – from Motorola. And we’ve heard anecdotal evidence – we haven’t found anyone that will cop to it yet – that the scientists were actually referencing and they remembered their childhood of seeing this sort of flip phone, and that it would influence their expectations about what these devices should be. What’s interesting is it’s not just influencing the expectations of the audience about what’s acceptable and what’s possible; it’s also influencing, albeit over usually a larger timeframe, engineers and designers and projects leads and even funders, I’m sure, about what’s possible and what should be, because we’ve now established use cases even though they’re fictional.
So we’re gonna leave the realm of expectation a little bit and talk about the social context, because this is where science fiction – really good science fiction is really about a commentary on the present. It’s really about people, not so much the technology, and that’s just there as a sort of twist or a trigger to do things that you couldn’t necessarily do in other kinds of drama or comedy. And specifically, we’re gonna talk a little bit about anthropomorphism, because we see this over and over in science fiction, and of course, this is just a mechanism to talk about people and our relationship to each other and our relationship to technology. But because it’s science fiction, we can sort of play with the story and play with people’s emotions and make the characters do some things that otherwise just wouldn’t make sense at all.
I’m sure everyone here has – I didn’t hear any hisses, though – remember the prior attempts at anthropomorphizing things in the interface. We have Clippy, and we have whatever the Mac thing that I don’t even think had a name, and Bob. We see these things over and over. These happen to be from Microsoft, but lots of companies tried this. How many people have ever seen Ms. Dewey? Wow, more people than I would’ve expected. How many people think – how many people liked Ms. Dewey? Not a whole lot.
What was interesting – and this corresponds with Bob as well from Microsoft – is that they made some interesting choices. We don’t have a clip of this. Sadly, it’s no longer available, and we can’t get clips of this. But they chose to render Ms. Dewey first of all in this sort of slinky maid uniform. I’m not quite sure what it is.
[Audience laughter]
And she’s smiling here, but this is actually sort of a rare show of positive emotions. She used to sort of – most of her emotions, at least when I did searches, she’d walk around and sort of snit and be generally annoying, actually. And that was an interesting choice. And all of these examples – Bob and Ms. Dewey and Clippy, etc. – really point to the difficulty that if you’re gonna play this anthropomorphism game, you really have to play it correctly or else it doesn’t work; in fact, it can work disastrously.
This is a more positive example. This is Knowledge Navigator from ’87. This goes back a ways.
[Video plays of Knowledge Navigator]
Chris Noessel: So not only was Knowledge Navigator sort of heralded as a really good example of a corporate future-think release to the public done well, but one of the reasons why it was done well is that Phil doesn’t have too much emotion. He has agency, but he doesn’t have too much agency, and they sort of played this balance about how much anthropomorphism, how much likeness, how much behavior you want your computer – or, in this case, a computer – to exhibit, and how much you don’t, where it gets either annoying or in the way or silly or unbelievable.
One of the things to note about anthropomorphism – the examples we’ve seen so far are very visual examples of “Yes, that looks human.” And then we tend to think that anthropomorphism needs those visual representations. Turns out it’s not the case. Any kind of reference and any kind of sensation to humanness – or, in some cases, animal-ness – will still trigger these effects. And so here are examples where the anthropomorphism is carried entirely by voice.
[Video clip plays of Star Trek]
So aside from a commentary about voice recognition and voice synthesis and whether voice interfaces are appropriate or not, what happens when you use voice – certainly in science fiction, but also it applies as a lesson to other kinds of technical media – is that it telegraphs capabilities that may or may not be there. When we hear the computer respond in a natural way and respond in intelligent ways, or what we think are intelligent ways, we tend to forget that it’s programmatic. We tend to assume that there’s capabilities that are more humanlike, whether or not in fact that’s the case. And this is where anthropomorphism gets difficult, because we can imply capabilities and responses that may not be able to be realized, and therefore may be disappointing for either audiences or users.
This is a really great clip that I love as well.
[Video clip plays of R2D2 from Star Wars]
Okay, so R2D2 is one of the most beloved characters in Star Wars, and he doesn’t talk, right? Which is kinda weird because all the other robots in Star Wars can talk, but leaving that aside, one of the reasons why he is so beloved, besides the fact that he’s the best actor, is that –
[Audience laughter]
– he has these sort of humanlike traits, right? We know he’s afraid ’cause we hear these sort of whimpers. Well, he has no voice. He only has sound effects. And yet it’s telegraphing emotion, and it’s telegraphing intelligence; it’s telegraphing capabilities. So we don’t need a humanlike representation that’s visual. We can carry that anthropomorphism entirely in the sound channel as we like.
We have “Knight Rider.” We won’t go into that. We all remember him: KITT.
[Audience laughter]
So it turns out that even voice isn’t necessary to carry off this sort of anthropomorphized effects and all of the triggers that come with it. Sometimes just behavior is enough, and that’s what interaction design is all about.
So this is actually a fairly subtle example, but when you go into Amazon or any – or the Apple store, for that matter, ’cause it has a one-click interface, and it recognizes you, right? We have put on the rest of the page “Welcome back,” etc., and here’s all your recommendations. When you can just hit one button on a buy page and it takes care of everything else, essentially there’s an anthropomorphism that happens back there that says, “I know who you are. I trust you. I have agency, and I’ll take care of all the details for you.” It’s not unlike walking into your favorite bar and nodding at the bartender, and by the time you get up to the bar, he has your favorite drink ready, and by the way, he’s started a tab and you didn’t even have to pay for it right now. So all of this wraps up into behavior that is more humanlike than we’re used to in most kinds of systems. And that’s one of the interesting lessons that comes out of this work.
Going back to the visual issue of how the representation is carried visually, one of the things that we’re finding, of course – and let’s…
[Video begins playing]
Does anyone remember Until the End of the World?
[Video continues playing]
Okay, so this is Bounty Bear.
[Video continues playing, then ends]
All right, so “I’m searching, I’m searching.” This is a search interface. He’s actually searching the entire network. He’s basically tracing a call. But it’s anthropomorphized.
Well, so what does the anthropomorphism bring? It makes it kinda funny, to start with, but it also implies an intelligence and a capability that otherwise wouldn’t be there in a normal Google interface, or certainly back in 1991, what we would imagine probably a way search might look like. So we’re telegraphing – or the film producers are telegraphing capabilities that look futuristic.
Now, in terms of likeness, Bounty Bear is still pretty rough and probably good for computer animation of his day, but doesn’t look particularly real and humanlike, or, in this case, animal-like. He doesn’t get too close to the uncanny valley. But taking it all the way down the valley and back up again, we have The Matrix.
[Video clip plays from The Matrix]
Okay, so Agent Smith is a program. He’s an algorithm. He’s a set of code, right? But here he’s represented as a person, and he’s represented with just as much resolution and reality and likelihood as the other characters that we also know are human. Of course, we know on some level they’re all in this shared reality or shared fantasy, however you wanna describe it. And by putting him in a absolutely equivocal human likeness, that telegraphs more danger, more capability, more cunning, more – or at least equal abilities to a human, far surpassing what we really normally consider as capabilities of technology, even of this time, right?
So this helps audiences understand that this is real danger and a higher danger than if it was just a computer algorithm or a set of code. But it also telegraphs the power of this working in terms of what we could do in our own interfaces. I’m not saying that you should go out and put “agents” in incredible likeness in your interfaces, but realize that these anthropomorphic lessons are lessons that you can apply to your own work. This takes another – I’ll show one more clip here.
[Video clip plays from The Matrix]
Right, so the algorithm is difficult to believe that it can predict the future, and it has these sort of metaphysical traits, if it was just a set of code, if it was a screen-based, sort of keyboard-based computer. But when you represent it this way, both audiences, and therefore users, have a much easier time understanding and assuming that these capabilities are real and in fact powerful.
Nathan Shedroff: So going down to sorta the last arc of influence, if social context is really about the human side of the equation, the proposed paradigm is where we as designers can actually take a look at the interfaces in science fiction and sort of glean lessons from them.
Some of these are lessons that we all kind of know, but it’s nice to see written large, such as the lesson “constraints ease the learning curve for new users.” Like, we know that, but when you can sort of go up and see it on the screen, you’re like, “Ah, good reminder.” We have a positive example here where people from in The Fifth Element sort of figure out how to operate a millennium-old weapon without any instruction set. But what I’m gonna do is zip ahead to the negative example, which is a short film by Pixar called Lifted, in 2007, where a little alien gets to sorta learn the ropes of human abduction.
[Video clip plays from Lifted]
So poor little alien has zero constraints on this giant interface, and that’s sorta where all the comedy comes from. We all know this. Hey, look, it’s written up large, and now we can reference it and laugh at it. We don’t want our Ford steering wheel to look anything like Lifted’s interface. Which it doesn’t.
[Audience laughter]
A second sort of lesson that we can sort of glean from them are things that are known probably in the labs, but that we don’t really have a great deal of experience with. An example of this is that input should know the effective or the emotional states of their users in order to sort of really help them do what it is they’re really trying to do.
We have this – again, an example from The Fifth Element, which is a negative example where Zorg is choking on a cherry pit, and he begins to mash on this keyboard on his desk, and the interface doesn’t go, “Hmm, he probably needs security.” It actually tries to do what the little buttons that he’s pressing are telling him to do.
Instead, we can take a – actually, I’m gonna bypass this one as well for time. But in 2001, we see a positive example, where Dr. Floyd is talking to his young daughter, who’s down on Earth, from a space station, and he’s able to – I’m sorry, during the course of the call, she ends up sort of mashing keys. But the computer knows, “Oh, this is a little girl. Her attention is on her dad. We probably don’t need to – she’s probably not trying to influence the call,” and it actually ignores it. Nope, I’m gonna skip it.
I’m gonna show just two quick clips here about this last lesson, which are things that you can really only learn by prototyping in science fiction. The lesson here is that social hierarchy can be reflected in holography. And for purists, I put “holography” in quotes. And for this we go to Star Wars. You’ll remember the scene from Star Wars VI: The Empire Strikes Back. When Darth Vader sort of steps out of his little pod and addresses the emperor, there’s a very particular sense of scale to the hologram.
[Video clip plays from Star Wars VI: The Empire Strikes Back]
Nathan Shedroff: And there’s the master. And of course, he’s not talking to a giant; the emperor’s not huge. It’s just scaled large. But when we contrast that with the way that the Jedis are represented in holography – this is from Star Wars III: Revenge of the Sith, in 2005.
[Video clip plays from Star Wars III: Revenge of the Sith]
You can see that they’ve taken pains to be equal in size. And it is this sort of notion that just passes you by while you’re watching the film, but on inspection you’re like, “Well, of course. The Empire is hierarchical, and Jedis are egalitarian.” And their media reflects that.
Chris Noessel: Right, so we’re gonna leave it at this, with this example. This is actually in a lesson that you can take directly into your interface design, right? So if you have a social interface, like iChat or FaceTime or Skype, etc., where people are represented, social hierarchy being reinforced or broken is a possible – is something that you can play with, right? So we’re finding many of these lessons in science fiction played out that have direct relationship to interaction design/interface design. If you go to the notes, the PDF that’s online that we showed you at the beginning of the talk, there’s all these lessons from these examples culled out, and hopefully we’ll be able to show the rest of them with you in the spring.
One of the major lessons, though – the last thing I wanna cover is that if it works for an audience in science fiction, it often works for a user. In fact, it almost always works for a user. And so that’s a really good way for you to watch and deconstruct the science fiction that you see. But there are exceptions to this, especially gesture control and voice control, breaks for users, even though it looks really cool and works really well for audiences. So with that, we’ll leave it there, and –
Nathan Shedroff: Bring Mark up.
Chris Noessel: – thank you very much.
Nathan Shedroff: Yeah, that’s it.
[End of Audio]
Transcripts provided by Verbalink
Tag your photos: uxweek2010 flickr.com
Location
Mission Bay Conference Center at UCSF
1675 Owens Street
San Francisco, CA 94105
Google map
Hotel Palomar San Francisco
12 4th Street
San Francisco, CA 94103
Google map
Hotel reservations