13 August 2008
Thank you for coming by my new blog, I intend to use this space to share some of my thoughts and I hope that you’ll contribute your own via the comments.
My job at BT is to find innovative new uses for broadband that grow and maintain adoption. I build software and hardware prototypes that we then user-trial. My driving passion is to make technology as simple and fun to use as possible; too often our everyday experiences are neither of these things. I would like to start by talking about this.
Traditionally we have attached great importance to the usability and reliability of critical systems. For instance, the control panel of an aeroplane must present information in a form that allows smart decisions to be made quickly and acted on accurately and reliably.
Today technology is woven into evermore aspects of our lives; consider the ubiquity of telephony, electronic communications and network connected devices. As a result our most precious and critical social and business relationships are frequently mediated through hardware and software interfaces. Here the ones and zeros of a digital message are emotionally charged.
Misunderstanding of a system error or glitch as a loved-one’s deliberate act, can cause much heart-ache and in extreme cases relationship breakdown. As such I argue we should attach the same significance to usability, reliability and good design in these systems as we do to a nuclear power station.
Studies have suggested that email, used by millions daily, is frequently misinterpreted. For instance, in 2005 Kruger et al published a study , in which participants were asked to gauge a partner’s tone of voice in an email.
Results suggested they performed at little better than a 50/50 guess, while believing they had been quiet accurate. My own work in video-conferencing has highlighted how a lack of eye-contact, can be misread as inattention, rather than the misalignment of cameras.
While we can find less ambiguous ways to communicate, users must also become adept at selecting the right medium for their message. Good design can help us make these decisions.
While there are many aspects to usability, unnecessary complexity is one of the biggest inhibitors. Simplicity of design rarely occurs by accident. It requires a deep understanding of the user, their motivations and their context, in addition to the capabilities of the technology.
John Maeda’s book The Laws of Simplicity lays down ten methods for approaching a system. The first is to “reduce”, finding and keeping only the essential features. In software, this can be at odds with the wisdom to increase the product’s feature-list, not reduce it.
Maeda’s tenth law summaries the others and states, “Simplicity is about subtracting the obvious, and adding the meaningful”. For communication systems, our challenge is to apply the right technology to the right problem, in such a way that the user’s experience and relationships are meaningfully enhanced.
12 October 2008
In my last blog I talked about how our relationships with others are being mediated by technology, making the usability and simplicity of these system ever more important. Today I’d like to think a little more broadly about new approaches that make computing accessible to all, specifically the potential for Tangible Computing.
These are ideas that have been developed in academic circles for the past twenty years and now beginning to be found in commercial products.
We are all familiar with Gordon Moore’s observation that the number of transistors on an integrated circuit can be doubled almost every two years for the same cost, resulting in a doubling of computational power. The corollary of this being that the size of an equivalently powerful device will half every two years. Coupled with wireless networking, display and power technologies, there is the promise of ever smaller and cheaper computing devices. As computation becomes ever smaller it can be embedded into everything things; into walls, floors, pens and desks.
Ubiquitous Computing (ubicomp) was a term coined by Mark Weiser of Xerox PARC in 1988 to describe computers defused into the environment. More recently the work of Hiroshi Ishii’s Tangile Media Group at the MIT Media Lab has incorporated Weiser’s agenda, developing the Tangible User Interface (TUI) paradigm as an alternative to today’s desktop metaphor.
The key characteristic of these interfaces being that digital information is represented physically by properties of an object and manipulating the object manipulates the information. Examples of TUIs include: SandScape where a landscape is modelled in sand and resulting properties such as drainage are projected in real-time onto the surface, genieBottles in which sounds are released from a bottle by removing the cork and inTouch which communicates the touch of a far away loved-one.
Through our collaboration with MIT and the Media Lab, my own work has been influenced by these ideas. The BT Balance prototype uses accelerometers to make a tablet PC and graphics react as if it were a physical object, for instance turning the page of a virtual book or moving across a map.
Aspects of theses ideas are now to be seen in commercial products such as the Apple iPhone, Nintendo Wii, Violet’s Nabaztag and Microsoft Surface. I believe a future of highly intuitive systems are now within our grasp.
23 October 2008
In my last blog I talked about Tangible Computing; the use of physical properties of objects to represent and manipulate data. In contrast today I would like to think about our use of graphical displays. The ever falling cost of high-quality flat display technology is enabling moving graphics to be shown on an ever wider set of devices and surfaces. A modern living-room may now contain not only a highly capable large display television set, but also the screens of mobile phones, media players, computers and digital picture frames.
In public spaces screens are to be found in shopping aisles, window displays and public information systems. The London Underground is now populated with an array of moving high-resolution advertising boards. I argue our relationship to displays is changing with the inclusion of what I term the Mural Display.
With the traditional use of displays the viewer is at a relatively predictable distance; at arms length with a laptop or a few metres with a television. As such they tend to have the majority of the viewer’s attention, who generally remains quite stationary. Matthew Pillsbury’s long exposure photographs illustrate this beautifully.
Mural Displays by contrast are those which are typically attached to architecture of a space, such an advertisement or digital picture frame. Their relationship with viewers is more transient and fleeting, as people move around them. They may be used for glanceable or ambient information, like a wall-clock or seek to attract our attention like an advertisement. The uncertainly of viewing distance makes legibility of text and images hard to predict and a multi-resolution approach can be employed. This is well-known from the design of posters in advertising and artworks for gallery walls. Where large imagery and text attracts a viewer forward to a distance where the detail becomes apparent.
However, more is possible in the digital medium. Orit Zuckerman’s Moving Portrait demonstrates how these Mural Displays may engage dynamically with viewers. Using a camera and computer-vision to find the attentive faces in front of the display, the portrait is brought to life just by looking. This technique also allows the distance of a viewer to be estimated, allowing the content to be adapted to be most legible.
Current trends suggest that displays will continue to proliferate into our environments. Films such as Steven Spielberg’s Minority Report depict a world of displays on every cereal package. We have already seen the cover of Esquire magazine with an embedded e-ink display. It’s easy to imagine these environments becoming completely overwhelming. The challenge is to avoid that. I believe the kind of context and attention awareness seen in the Moving Portrait is going to be key in managing the visual hubbub.
2 December 2008
In the coming Petabyte Age we will be able to generate, store and process a mind-boggling quantity of our own data.
In 2003 Jim Gray of Microsoft Research identified the “personal petabyte” of data that could soon be accumulated by an individual. This would include video, emails and personal sensor data. The question is, how to make use of it to allow us to find the things we want and make smart decisions with it?
In today’s blog I would like to think a little about the growing importance of data-visualisation techniques in understanding these vast and complex personal data-sets.
John Snow’s map of the 1854 Broad Street cholera outbreak is a classic example of the power of visualisation. When mapped, the clusters formed around the local water pump, allowing the wisdom that cholera was an airborne disease to be challenged and its spread to be correctly treated.
In the recent US election we saw a number of really fascinating info-graphics, including Mark Newman’s maps from the University of Michigan. In these each state rescaled according to its population or Electoral College size, showing at a glance the distribution of the vote.
The cholera and election maps make salient features of a large data set apparent so hypotheses can be tested and plans can be easily made by the viewer. Edmund Tufte’s series of beautiful books describe effective communication with information graphics. In addition he highlights common mistakes and bad practices that lead to a viewer being misled.
Themail by Fernanda Viegas allows user to visualise their emails over a period of months or years and reflect on how their relationships have changed over time. Now at IBM, Viegas is a member of the Many Eyes team that allows people to upload, visualise and discuss large data-sets socially.
For video data and other rich multimedia formats, projects such as the The Human Speechome Project at MIT are developing techniques to visualise vast quantities of video data [QuickTime, 3.6 megs]. Allow a specific moment to be found in amongst all the data. Data visualisation goes far beyond making “pretty” graphs, they are a powerful set of techniques for handling our ever-growing personal data.
12 February 2009
In recent years GPS has been widely adopted in car navigation systems, guiding drivers to their destinations. With mobile devices using GPS, cell tower and WiFi location, a range of interesting new social applications have recently been announced beyond the navigational. In the past week Google has introduced the Latitude service, allowing friends to share their location and status. Others include Loopt and Brightkite. In today’s blog I’d like to think about services like these and social applications of location technologies.
As these new services emerge we can look back at the experiences of earlier work in this space. One playful example is that of Geocaching; a kind of treasure-hunt where people seek hidden containers or caches, at published locations, which has developed a large on-line community. Another is Blast Theory’s Rider Spoke, which allowed people to hide secrets around a city, that others may stumble upon. BT’s research project Elevate, also allows users to contribute located user-generated content for public consultation, on an urban renewal plan.
As we have seen these social applications become ever more useful as a crowd gathers around it. As it will be here, where friends introduce each other to a service. ZoneTag from Yahoo Research demonstrates the power of a network of users, to suggest descriptive tags for geotagged photographs. As there is a good deal of agreement on points for photographs, particularly around tourist spots, the chances are that someone will have taken a shot from a similar position before and the words they used to describe it will be good suggestions to tag your photo. The question is of course how a new service grows in popularity to this point.
This same technology can not only tell us about the local environment, but also allow our devices to adapt to the context in which they operate. For Nathan Eagle’s PhD thesis on Reality Mining, he equipped one hundred MIT students with mobile telephones that were aware of their location, the identity of devices around them, as well as the calls and messages made; for a period of eight months. From this Nathan could predict where a user was likely to be next and with whom, given their current location and those of their friends.
It is clear that we will have to wait to see how these technologies are adopted and made sense of; addressing the inevitable privacy and social issues that arise. As the mobile telephone already allows us to make far more ad hoc arrangements, “give me a call when you’re near”, I think these location services continue that trend. Arriving unprepared in a strange town becomes far less daunting, when your mobile device provides access to local hotels and restaurants, with user-contributed reviews. Will these technologies make us all more spontaneous and unplanned? Is this a good thing?
24 March 2009
I have previously talked about how the economics of display technologies will soon put Displays Everywhere. There I highlighted some methods for avoiding a visual hubbub by understanding the viewer’s attention. Today I would like to think about devices where screens are undesirable or impossible and alternative ways of displaying information.
In recent years we have seen the emergence of ambient or glancable displays. These are typically screenless devices that change their appearance, perhaps by changing colour, to reflect a change in some interesting property; for instance the weather, stock prices or a friend’s online status. Like a wall clock, the information is available at a glance and importantly can easily be ignored when busy. Examples of these include: the Nabaztag rabbit, Ambient Devices’ Orb and our own Nurture prototype. Here information is represented using colour changing LEDs and physical actuators such as gauges and motors.
In my previous blog, Grasping a Tangible Future, I talk about further ways to present digital information physically.
In addition to visual and physical properties, sound, texture and even smell can be used to represent information. Each have different characteristics that make them more or less desirable for a specific application. Our senses can each distinguish over different ranges and degrees; drawing our attention in different ways. As Marshall McLuhan reminds us, “The Medium is the Message”. An ambient fire alarm is clearly inappropriate.
Sound, be it speech, music or environmental, is very rich way of conveying information and can be used very effectively in interface design. The swinxs console is an outdoor games system for children, using only a voice and sound effects. Apple’s new iPod shuffle announced this week, uses a voice to read back the track names and artists, without any visual displays.
Surround sound and binaural techniques allow the placement of sound in the space surrounding the user. This allows us to consider displaying information that can be moved and manipulated by a user, without a display.
I believe sound interfaces are an interesting direction, especially for mobile devices where the user’s attention is split between operating the device and their changing environment.
The use of audio and tactile interfaces, as alternatives to visual displays, is well established in access technologies for the blind and partially sighted. These include Screen Reading software and Braille displays. Recent research, such as the European Union’s ENABLED project, use a combination of haptic interfaces and audio, in this case to present a haptic city map.
While there have been advances in the synthesis of scent, it is yet to be widely adopted.
Configuration is a challenge for many screenless devices and many co-opt other local displays, for instance via a webpage interface.
As the shape of computing devices changes as they become increasingly ubiquitous and embedded into our environments, we will need to rethink the ways in which information is presented to us, if we are to avoid an overwhelming hubbub.
15 April 2009
While teleportation remains firmly in the domain of science fiction, I think we are seeing the start of a significant growth in transporting three dimensional objects across our data networks. In today's blog I'd like to think about developments in personal fabrication.
In the past few years we have seen the steady fall in the cost of 3D printers; machines able to print three dimensional objects by bonding particles of material (typically resin) together layer-by-layer until a full object is formed. For small objects the printers can be desktop devices. Last year it was pointed out that 3D Printers were as cheap as laser printers had been in 1985. In recent years these have been a useful rapid prototyping technology, but we can now envisage much wider use of people designing and making their own things.
While we await the 3D printer cheaper enough for the home consumer, a number of online services such as Shapeways and Ponoko allow people to submit designs online, to have their physical object them mailed back to them and to others sold online. Another, Thingiverse, allows designs to be shared with an open-source community.
The Reprap project is an interesting open-source project, allowing the enthusiast to created a 3D printer from cheap readily available components, that is in turn able to self-replicate 60% of its own parts.
Novices can now translate their ideas quickly into the required digital formats using applications such as Google's SketchUp. An alternative approach is to scan an existing object. The combination of these technologies allows us to begin to think about copying physical objects across the Internet.
Initially these objects are somewhat superficial, rather than having functional and structural properties. However, the 3D printing techniques continue to advance allowing composites of materials, working mechanical and electronic parts. The garage is now able to fabricate the replacement engine part themselves and the consumer able to create individual market-of-one solutions to their needs.
This is the world that Neil Gershenfeld describes in his book Fab: The Coming Revolution on Your Desktop - from Personal Computers to Personal Fabrication. With the Fab Lab initiative Gershenfeld shows how fabrication technologies can empower communities in deprived or rural communities, to solve their own local problems.
If the personal fabricator becomes as ubiquitous as the desktop printer, the traditional models of mass production, physical distribution and storage are challenged, with one would hope positive environmental results.
2 June 2009
One of the aspects of my work is to track how technologies mature and if appropriate apply them to the problems we encounter. In today’s blog I would like to look at the computer-vision technique of face detection.
I first became aware of algorithms to detect faces in images in around 1996, with the CMU On-line Face Detector which allowed people to upload pictures that were then analysed, and if any faces were shown they would be highlighted with a green rectangle. Processing took up to ten seconds per image or frame of video. For me, at the time, this was an interesting novelty.
A theme of my research has been facial communications, how humans communicate a vast amount both in the appearance of their faces and expressions they make with them. As such I have looked at how we can equip machines with the ability to read this facial information and how it can be communicated to others via telecoms.
In the Prometheus project, a collaborative project with the BBC and others finishing in 2002, we looked at the production of three-dimensional television content. We built software to allow facial motion-capture from of an actor’s performance, without the need for special markers to be attached to the face, to animate a virtual puppet. This required that both the face was detected in the camera’s view and features of the face are tracked to express emotion.
I have since applied face detection algorithms to smart image cropping, where thumbnails maximise the size of the faces and to video-conferencing that respects the importance of the face.
With the development of the algorithms and faster processing, full-rate video can now be analysed in real-time. Face detection algorithms are now routinely integrated into many digital cameras, allowing the faces to be put automatically in focus and even for the camera to trigger when a smile is detected.
With a combination of face detection and recognition, there are many applications in security and search over picture collections or video. The latest version of Apple’s iPhoto addresses the problem of searching vast photo collections by automatically finding and identifying the faces, allowing searches to be made according to who is in the picture. The CMU technology has now been licensed to Pittsburgh Pattern Recognition, who have several interesting demos including one to index people in video, creating a searchable timeline of the Star Trek episodes.
As I mentioned in my previous blog about the ubiquity of displays, I think gauging attention is another very interesting application of this technology that should help us avoid an overwhelming visual hubbub. If a camera is mounted by the screen, the appearance of a forward-facing face is a good indication that the display is being watched. Orit Zuckerman demonstrated how this may be used to create an intimate portrait of a loved-one with her Moving Portrait. Other applications include creating a real-time measure of the attention that an electronic billboard is receiving, giving a fine grain metric back to the advertiser.
Face detection is now a well established technique that I think we will continue to find useful applications for.