What exactly is computer-mediated reality? In the immortal words of Sly Stone, "it's a family affair".

Computer-mediated reality is the "superset" – the set that contains all sets. This superset includes the extended reality (XR) family and their not-too-distant relatives, the modulated reality family (ModR).

You may be familiar with XR as the catch-all term for virtual reality (VR), mixed reality (MR), augmented reality (AR) and assisted reality (aR). XR, however, doesn't include MoDR. MoDR refers to mediated realities that either alter or remove real-world elements from the user's view. 

A quick and easy way to remember the distinction is that XR, as a whole, adds to the user's environment, whereas ModR alters and in some cases subtracts from the user's environment. Most devices use a mixture of adding, manipulating and subtracting content, but all are computer-mediated. 

The origins of computer-mediated reality

The term is credited to computer scientist Steve Mann who, during the 70s and 80s, developed a helmet intended to help welders. The "digital eye glass" used cameras and "high-dynamic-range imaging" that modified the user's sight in dark areas and reduced the lighting in bright areas.

Mann's seminal work Fundamentals of Wearable Computing and Augmented Reality is a foundational textbook, outlining numerous ways in which wearable computer-mediated reality devices will shape future societies.

In it, he describes high-tech clothing with computing power and outlines its possible uses. By many people's standards, his ideas may appear futuristic and almost outlandish – but, to XR aficionados, they're visionary and inspiring.

"Clothing with computational capability can function as an information filter, allowing the user to block out material he/she might not wish to experience, whether it be offensive advertising or simply a desire to replace existing media with different media."

Mann then describes how such devices might protect our privacy, "allow[ing] us to block or modify information leaving our encapsulated space. In the same way that ordinary clothing prevents others from directly seeing our bodies, computational clothing may, for example, serve as an intermediary for interacting with untrusted systems."

Categories of computer-mediated reality

Since you may be less familiar with ModR, we'll bring you up to speed on these terms first and then recap the XR family for newcomers. 

Modulated reality (ModR)

ModR is the term used to describe mediated realities that specifically change or "modulate" real-world objects, whether through a smartphone or head-mounted display (HMD). ModR does not include additional virtual elements. There are three distinct categories of ModR. 

1. Modified reality (MfR)

MfR refers to the technology of manipulating objects in the outside world through the use of filters or image processing. It's a subset of AR but shouldn't be confused with AR, per se. 

For example, the bunny ears filter on your phone is an augmented reality, because it adds virtual ears to the user. On the other hand, a big nose filter is a modified reality, because it stretches and distorts the user's actual nose. 

2. Diminished reality (DR)

DR seems at first to be counterintuitive to mediated reality's MO. Why would you want a lesser version of reality? Here's why.

DR includes any technology that removes objects from the user's perception. The technology can be used to literally erase objects from your view. This has therapeutic use cases that enable full control of virtual environments.

In combination with AR, DR may be used to remove objects for the purpose of replacing them with virtual ones.

(And there are audio applications for DR, too. Active noise cancellation is a common type of DR.) 

3. Severely Diminished reality (SDR)

SDR is the extreme end of DR and may be considered a type of sensory deprivation. 

Theoretically, an interior designer with AR glasses could remove everything from a room and overlay new design ideas onto the tabula rasa. SDR is a nascent area of research and also presents interesting use cases for therapy. 

Extended reality (XR)

XR technology focuses mainly on adding virtual layers of reality on top of the user's real world and – in the case of VR – enclosing the user in a fictional reality completely cut off from their own. Let's run over the four main categories of XR and how they relate to ModR.

1. Virtual reality (VR)

In VR, a completely separate reality is created that effectively replaces the real world. There are no ModR elements within a VR space because in VR there are no real-world elements. Users can interact with virtual elements with the use of haptic technology, and in rare cases, smell and taste the virtual reality with olfactory (smell) and gustatory (taste) VR. 

2. Mixed reality (MR)

In an MR system, the virtual and real are blended together. This is the most powerful and versatile of the mediated reality systems as MR brings together all the elements of ModR and XR in one device. Users can move between VR and AR seamlessly and experience a mixture of DR, AR and MfR.

In theory, augmented reality gaming in the future will be able to use a mix of AR and DR to replace players with avatars while maintaining the natural environment in which they play. 

3. Augmented reality (AR)

Augmented reality systems overlay virtual elements onto the user's environment. Unlike VR, the user can still see their immediate surroundings, but with the addition of a virtual layer. AR overlaps with ModR just in the same way MR does, but without the option of moving into a completely virtual space.

4. Assisted reality (aR)

Assisted reality devices do not attempt to interact with or replace our lived reality. This is a critical distinction between aR and AR (which are often confused). aR headsets include a display, which can be used to present information to the user, and a front-facing camera that allows "see-what-I-see" capabilities within video calls.

aR has no overlap with ModR functions, partly because the technology isn't included, but also because their goals are fundamentally different. aR makes no attempt to alter the user's reality, instead aiming to support workplace safety and productivity by providing hands-free access to important information and features.

Are you ready to explore extended realities? Browse assisted and augmented reality headsets from leading manufacturers like Microsoft and RealWear.