Headed out of Beta/Picking Your Superpower

Those were the words I noticed when interviewing Augmented World Expo organizer Ori Inbar several days before AWE2015, the trade show of Augmented and Virtual Reality. “We’re not in beta anymore…” Inbar said, “We now have companies implementing enterprise-scale Augmented Reality solutions, and with coming products like the Meta One and Microsoft HoloLens, the consumer market is being lined up as well.” With the addition of the UploadVR summit to AWE2015 the event was a blitz of ideas, technologies and new hardware.

AWE/Upload is a trade and industry event that also includes coverage of the arts and related cultural effects, although it is smaller when compared to the industrial aspect of the show. In this way it is similar to SIGGRAPH and this is much of my rationale for covering this, and also SIGGRAPH later this year? Doing so is as simple as McLuhan’s axiom of “The Medium is the Message” or, better yet, examining how developers and industry shape the technologies and cultural frameworks from which the artforms using these techniques emerge. The issue is that in examining emerging technologies we can not only get an idea of near-future design fictions but also the emerging culture embedded within it.

To put things in perspective, Augmented Reality art is not new, as groups like Manifest.AR have already nearly come and gone and my own group in Second Life, Second Front, is in its ninth year. Even though media artists are frequently early technology adopters, what appears to be happening at the larger scale is a critical mass that signals the acceptance of these new technologies by a larger audience. But with all emerging technologies there is drama driven by those industries’ growing pains. For AR & VR the last two years have certainly been tumultuous.

Last year’s acquisition of Oculus Rift by Facebook sent ripples through the technology community. Fortunately, unlike my upcoming example, the buyout did not eliminate the Rift from the landscape; instead it gained venture capital allowing for licensing of the technology for products like the Sony Gear VR. Also the current design fictions being distributed by Microsoft for its Hololens give tantalizing glimpses of a future “Internet of No Things” full of virtual televisions and even ghostly laptops. This was suggested in a workshop by company Meta and the short film “Sight”, in which things like televisions, clocks, and objective art might soon be the function of the visor.

MetaThe Internet of No Things. Image Courtesy Meta, Inc.

However disruptive events also happen in the evolution of technologies and their cultures. The news was that scant weeks before the conference a leading Augmented Reality Platform, Metaio, was purchased by Apple. Unlike the transparency and expansion experienced by Oculus the Mataio site merely said that no new products were being sold and cloud support would cease by December 15th. In my conversation with conference organizer Ori Inbar we agreed that this was not unexpected as Apple has been acquiring AR technologies, which has been related in rumors of “the crazy thing Apple’s been working on…”; But what was surprising was the almost immediate blackout, part of the subject of my concurrent article “Beware of the Stacks”. For entrepreneurs and cultural producers alike there is a message: Be careful of the tools you use, or your artwork (or company) could suddenly falter in days beyond your control. Imagine a painting suddenly disintegrating because a company bought out the technology of linseed oil. Although this is a poor metaphor, technological artists are dependent on technology and one can see digital media arts’ conservative reliance on Jurassic technologies like Animated GIFs for its long-term viability, but to go further I risk digression.

Everyone in Headsets!

Another remarkable phenomenon this year was the near-assumption of the handheld as a experience device, and their use seemed almost invisible this year. What was evident was a proliferation of largely untethered headsets, ranging from the Phone-holding Google Cardboard to the Snapdragon-powered (and hot) ODG Android headset, boasting 30-degree field of view and the elimination of visible pixels. In the middle is the tethered, powerful Meta One headset with robust hand gesture recognition. Add in the conspicuously absent Microsoft Hololens and the popular design fictions of object and face recognition are emerging.

Like You’re Going to Have One Soon….

That is unless you are a brave early adopter, developer, or enterprise client. The fact that there was an entire Enterprise track and Daqri’s release of an AR-equipped construction/logistics helmet made it clear that the consumer market, much more prevalent last year, has clearly been placed in the long-term. For now, consumer/artistic AR is largely confined to the handheld device, as experienced through Will Pappenheimer’s “Proxy” at the Whitney Museum of American Art or Crayola’s “4D coloring books” in which certain colors serve as AR markers. This isn’t necessarily a bad thing, as an audience is likely to have a device that can run your app through which they can experience the art. As an aside, this is the reason why I chose to use handhelds for my tapestry work – imagine trying to experience a 21’ tapestry with a desktop using a 6’ cord! At this point, clarity and function, both partially dependent on computer power, have created a continuum from strapping your iPhone to your forehead like a jury-rigged Oculus for under $50, to potentially using a messenger bag with the Meta at $512, to the expensive ($2750), hot, but elegant ODG glasses you might try on if you visit the International Space Station.


Image Courtesy Osterhaut Digital Group

Where the Rubber Hits the Road

While discussing the general shape of technology gives a context for its content and application, a media tool is often only as good as its app. Without meaning to show favoritism, Mark Skwarek’s NYU Lab team has been going outstanding work from a visualization of upcoming architectural developments to a surprising proof of concept for a landmine detection system, which I thought was amazing. Equally innovative was the VA-ST structured light headset for the visually impaired, which has several modes for different modes of contrast. These alternate methods not only was surprising in terms of application and possible creative uses but also changed my perception of AR as possessing photorealistic, stereoscopic overlays.

Other novel applications included National Geographic’s AR jigsaw puzzle sets, of which I saw the one outlining the history of Dynastic Egypt. I felt that if I were a kid, building the puzzle and then exploring it with AR would seem magical. There are other entertainment and experimentation platforms coming online like Skwarek, et al’s “PlayAR” AR environmental gaming system. But one platform I want to hold accountable for still being in late beta is the” LyteShot” AR laser tag system, which got an Auggie Award this year. My pleasure in the system is that the “gun” per se is Arduino-based, meaning that it could be a maker’s heaven. It uses the excellent mid-priced Epson headset, but at this time it is used primarily for status updates although there is a difference between AR and a heads-up display. So, from this perspective, it means that there are some great platforms getting into the market that are highly entertaining and innovative, but there are a few bugs to work out.

Ideas vs. Product

For the past thousand words or so I have been talking about the industry and applications of AR, but for me, my “soul”, if you will, set on fire during the “idea” panels and keynotes. For example, on the first day, Steve Mann, Ryan Janzen and the group at Meta had a workshop to teach attendees how to make “Veillometers” (or pixel-stick like devices to map out the infrared fields of view of surveillance cameras. Mann, famous for creating the Wearable Computing Lab at MIT and being Senior Researcher at Meta, still seemed five years ahead of the pack, which was refreshing. Another inspirational talk was given by one of the progenitors of the field, and inaugural Auggie Award for Lifetime Achievement, Tom Furness. His reflection on the history of extended reality, and his time in the US Air Force developing heads-up AR was fascinating. But what was most inspirational is that now that he is working on humane uses for augmentation systems such as warping the viewfield to assist people with Macular Degeneration. This, in my opinion, is the real potential of these technologies. In fact this array of keynotes was incredible, with Mann, Furness, the iconic HITLab’s Mark Billinghurst, and science fiction writer David Brin, (who comes off near-Libertarian) gave vast food for thought.

Mann and BrinSteve Mann and David Brin. Image Courtesy Augmentedworldexpo.org

Auggies: The Best of the Best

Every year, the Augmented World Expo gives out the “Auggie” awards for achievements in technology, art, and innovation in AR. I think it should be noted that the Auggie is probably the world’s most unique trophy, consisting of a bust that is half naked skull and half fleshed head with a Borg-like lens with baleful eye wired into that head. The Auggie is another aspect of AWE that signals that the world of Reality media is still a bit Wild West.

There are several categories from Enterprise Application to Game/Toy (LyteShot having won this year), and many of them are largely of interest strictly to developers. For example, the fact that Qualcomm’s Vuforia development environment won three years in a row gives hint to its stability in the market, and Lowe’s HoloRoom is a wonderfully strange mix between Star Trek and Home Improvement. The headset winner was CastAR, a projective/reflective technology where polarized projectors were in the headset instead of cameras, which worked amazingly well. The other winners were gratifyingly humane applications such as Child MRI Evaluation and Next for Nigeria (Best Campaign). The prizes impressed on me that the community, or part of it, “got it” in terms of the potential of AR to help the human condition, which is perhaps a “superpower” that the conference framed itself under.

So, Where’s the Art?

Being that I am writing this for an art community it would be of interest to know where the art was in all of this. The Auggies have an Art category, as well as a gala between the end of the trade show events and the Auggie Awards. The pleasant part about AWE’s nominations for the best in AR art is that those works have integrity. Manifest.AR regular Sander Veerhof was nominated for his “Autocue”, where people with two mobile devices in a car can become the characters of famous driving dialogues (“Blues Brothers”, “Pulp Fiction”, “Harold and Kumar”). Octagon’s “History of London” is reminiscent of the National Geographic puzzles, except with far greater depth. Anita Yustisia’s beautiful “Circle of Life” paintings that were reactive to markers were on display in the auditorium but, besides a Twitter cloud and a Kinect-driven installation, the art was swamped by the size of the auditorium.

The winner of the art Auggie, Heavy & Re+Public’s’ “Consumption Cycle”, (which this writer saw at South by Southwest Interactive) was a baroquely detailed building sized mural of machinery and virtual television sets. I feel a bit of ambivalence about this work, as Heavy’s work tends to rely on spectacle. Of the lot I felt it did deserve the Auggie, purely for its execution and the effective use of spectacle. But with the emerging abilities of menuing, gesture recognition, and so on, I felt that last year’s winner, Darf Designs’ “Hermaton”, employed the potentials for AR as installation in a way that was more specific to the medium.

Consumption CycleConsumption Cycle, Image Courtesy Re+Public

HermatonHermaton, Image Courtesy Darf Design

Wasn’t there VR as well?

Yes, but it was in a much smaller area than the AR displays. There were standout technologies, like the Chinese Kickstarter-funded FOVE eye-tracking VR visor, a sensor to deliver directional sound, and Ricoh’s cute 360 degree immersive video camera. The Best in Show Auggie actually went to a VR installation, Mindride’s “Airflow”, where you are literally in a flying sling with an Oculus Rift headset. Although a little cumbersome, it was as close to the flying game in the AR design fiction short, “Sight”. So, in a way, the ideas of near-future design and beta revision culture are still driving technology as surely as the PADD on Star Trek presaged the iPad.

AirflowAirflow, Image Courtesy Ori Anbar

This year’s AWE/UploadVR event showed that reality technology is emerging strongly at the enterprise level and it’s merely a matter of time before it hits consumer culture, but it’s my contention that we’re 2-4 years out unless there’s a game changer like the Oculus for AR or if the Meta or ODG get a killer app, which is entirely possible. So, as the festival’s tagline suggests, are we ready for Superpowers for the People? It seems like we’re almost there but, like Tony Stark in the beginning, we’re still learning to operate the Iron Man suit, sort of banging around the lab.

Digital Art Week (DAW) in Asia: Augmented Reality and Social Reality meet

The artist and curator Art Clay was born in New York and lives in Basel. He is a specialist in the performance of self created works with the use of intermedia and has appeared at international festivals, on radio and television television in Europe, Asia and North America. His recent output focuses on large media based performative works and spectacles using mobile phone devices. He has received prizes for performance, theatre, new media art, music composition and curation. As an educator, he has taught media and interactive arts at various art schools and universities in Asia, Europe and North America including the University of the Arts in Zurich. He is the initiator and Artistic Director of the ‘Digital Art Weeks International’ and the Virtuale Switzerland.

Eva Kekou: Could you tell us about your work and what inspires you?

Arthur Clay: The question about inspiration has been posed to me before and most often in the moment when people first see the program that the Digital Art Weeks (DAW) is offering. As an artist I am very much inspired by the every day. I think it is important to be aware of the things around you and by so doing, my artwork seems to have more of a present day dialogue and the events I curate more to do what is actually going on in the society around me. So basically, everything and nothing and all the things in between inspire me. There are no rules, but with a lot of effort to try and come closer to things might be the one I apply the most.

On the one hand, I am a practicing artist and most of this work is concerned with sound. On the other hand, I believe that curation is an art in itself and requires a high level of creativity. It is easier for me to make an artwork in comparison to harmonizing a group of artworks. The two meet in the fact that I grounded the DAW projects in order to provide a platform for my own work and that of others, who think in like minded ways.

Cartoon stickers on a Taxi Window driving down Orchard Road in Singapore

EK: You were the curator for the Augmented Reality exhibit “Window Zoos & Views” in collaboration with the participation of the School of Digital Media and Infocomm Techology from Singapore Polytechnic. On the project site, it mentions that the main element of the project was inspired by an image of a car driving down Singapore’s legendary Orchard Road. Could you tell us a bit more about this and the project?

AC: As DAW Director, I am confronted with the same situation each time we bring the festival: Representation and integration. By representation is meant that we have institutions both public and private that act as stakeholders and in turn expect a high level of visibility; talking about integration is much more complex, because there are many levels of integration that must be addressed. First off, the festival structure demands that we integrate local artists into the program along with the international artists participating. Integration of local artists can also mean or entail a high level of knowledge and skill transfer. We do all this during what we like to call the “Exploratory Phase”, which is basically entails dropping the DAW team into a unfamiliar city and making every effort to make it familiar. This includes becoming aware of the general culture the festival will address, how digital that culture is, the art and non art spaces that will become the stage of the festival, and of course trying to find out who is doing what in terms of arts and technology.

To get back to your question about the image of the car driving down the street, the most important element of integration is becoming aware of what is going on in the world of things in which the festival is to be presented. The car we are talking about was a car whose windshield was plastered with cartoon stickers. So, imagine you’re self-sitting at the wheel of that car driving down Singapore’s Orchard Road. It is a really surreal experience; the stickers take on a kind virtual elelment as it floats down Orchard road. The image sparked my imaginations and the AR Parade project was born, which was a very important part of the “Window Zoos and Views” exhibit. Basically, the AR Parade mimicked the effect of the stickers on the car window, but instead of a car windshield, we used an iPhone app to view the images. It was very cool, popular on the street and got a lot of clicks.

AR Float Parade in Singapore with Curious Minds, concept; Singapore Polytechnic students, Objects (DAW Singapore 2013)

Tamiko Thiel. ARt Critic Face Matrix” at Ion Orchard, Orchard Road, Singapore, (DAW Singapore 2013)

EK: There were various artists in the festival showing Augmented Reality art as installations. This included artists such as Tamiko Thiel (DEU), John Cleater (USA), Will Pappenheimer (USA), Lily & Honglei (CHN), Marc Skwarek (USA), Lalie S. Pascual (CHE), John Craig Freeman (USA), and Curious Minds (CHE). It must of have taken a lot of time and energy to organize this, especially the technological aspects of the project. How difficult was it to set all of this up?

AC: In one word: impossible. It is new area of technology, a new approach to curating art, and above all you are dealing with a non-art public – basically anyone who is on the street. That is a big challenge. Add to it the new type of management skills that such an international project work requires and I think you are close to the impossible that I am referring to. For such exhibitions, “Management 2.0” skills are a necessity. The artists, the tech people, and the curators and admins meet and work solely in virtual space. So there are no walls, no tables, and there is no going for coffee together after the meeting. It is a different world from all sides.

Flotsam & Jetsam, John Craig Freeman, (DAW Singapore 2013)

Another interesting aspect of this work is that you have to develop a feel for the city and develop a dialogue with it. For the Hong Kong show we did for SIGGRAPH Asia, Monika Rut and I spent about a week working on site, visiting the different areas of the city to check things out and to get a feeling for which works should go where and why. We travel with a lot of special equipment so that we can make tests on site and produce a mock up of the exhibition and test how the experience of viewing the exhibit will be. It is very inspiring but exhausting work and the dialog between Monika and I helped greatly in making all the decisions. Basically, we get to know the cities we are working in quite well and the kick back is, you get to know where the best coffee houses and local restaurants are.

HoerRoom by Art Clay (Participatory artwork at NAMOC, Peking 2009)

EK: Why hold the DAW in Asia, and what kind of differences do you experience culturally when working in Asia compared to working in Europe?

AC: The DAW is at home in Asia and much of our curating has to do with having visitors get pro-active. This means that it is not just about looking at an artwork, it also entails actually touching the artworks in many cases. In Asia, the museum visitors are not that schooled in museum etiquette. They like to touch things and this is great for the kind of work we like to do. Things break, but it is kind of a “I Like” thing for us.

The other aspect to consider is that you have a language barrier, because no one is going to understand anything if it is not translated. Here, it is also important to know that the approach Europeans take in terms of explaining artworks, does not really come over so well in Asia. Things are often inspired by poetics of nature in Asia and are much less conceptual than works coming from Europa.

Last but not least, the role of size also plays a large part in Asian arts. When a dynasty was at its peek, it produced very large artworks, monuments etc. So size is historically a sign of wealth and prosperity in Asia. When a dynasty fell, they produced much smaller works. So the bigger the better, so to say. For a European artist this is completely meaningless, however, bigger artworks have more visibility. So when you are curating group exhibits in Asia that artists from diverse countries, it is a challenge to keep balance in terms of impact of the works.

HandFalls installation an performance by Ichi Ikeda. (DAW Singapore 2012 at ArtScience Museum Marina Bay Sands)

EK: Getting back to There DAW AR Float Parade, which was the first of its kind and celebrated as the coming age for Augmented Reality art. Could you tell us more about this aspect of the festival’s project and how it turned out?

AC: Knowledge transfer plays a major role in getting a project like the AR Parade to work. The DAW has an “OutReach Program”. This is a program that invites creative leaders from around the world to hold workshops before or during the festival. The contents of the AR Parade in Singapore were the results of a workshop with the AR artist John Craig Freeman and the Curious Minds group. John Craig dealt with the technical details and the bootstrapping a group of twenty-five creative industry students from the Singapore Polytechnic. The members of the Curious Minds art group hung around, integrated the group into the DAW, taught the students about public interaction, and came up with how to go about actually making the AR Parade happen and come to life.

The proof of a good project for me is when it takes on its own life on after its initial presentation. After the Singapore DAW, the AR parade went to Hong Kong and in 2015 it will be shown marching down the Bahnhofstrasse in Zurich as part of the Virtuale Switzerland festival – the world’s first festival “virtual biennale” that focuses solely on virtual artwork. For this we want to go a bit deeper into the creative industry world and see if we can act as modern alchemists and pick up on the float parade from New York and turn Miss Kitty into an artwork by shifting context from the real to the virtual.

Live- Performance “Beads are the Breath of the Landbridge “ with 1st Nations artists, Peter Morin. (DAW Singapore 2013)

EK: What is your view regarding the social contexts of the DAW?

AC: Basically, we are concerned about things and we make an effort to improve things that are within our reach to improve. For the festival in Taipei that will take place in 2015, we are really trying to make people aware of the role of creativity in general. Artists are highly creative people and the world really needs good ideas as well as a more social approach. We try to provide an answer to the question: “How is what we do of benefit to the society in which we are operating?” We think this should be more a question that businesses should be asking themselves. The future needs fewer companies who are “profit first, prosperity second” and more social entrepreneurships that embrace social needs as part of their business model.

EK: Could you tell us what themes and aims we can expect from of the DAW project in the future?

AC: We are off to Seoul, South Korea in 2014 and the theme there is “Creativity and Convergence”, which is a hot topic in Asia, because the government feels that innovation is intimately connected to being creative and thinking out of the box. I think they’re thinking of what makes the West tick and in an odd and ironic way to imitate – which is not exactly creative, but then again I have lot of respect for Asia and admire the work ethic of Asian people. So I think they will do well by addressing these themes. In 2015, we are off to Taipei and the theme (working title) there is “ImagiNation” and here we are trying to run a kind of “Skills-Festival” and platform creative businesses and the approaches that they have taken. It is a new approach and if you think about what Beuys said, “Everyone is an artists”, we might stretch it a bit and also say “everything is art.” At least this is the start and interestingly enough we are working with groups from the labor department in Taiwan, with knowledge transfer departments of universities, and with spin off and start up companies from Switzerland. It is very exciting and really what the DAW is about: creating a platform for research and experiments in social-cultural context.

Digital Art Weeks International
The DAW INTERNATIONAL’s is concerned in general with the bridge between the arts and sciences in cultural context with the application of digital technology in specific. Consisting of symposia, workshops and cultural events, the DAW program offers insight into current research and innovations in art and technology as well as illustrating resulting synergies, making artists aware of impulses in technology and scientists aware of the possibilities of application of technology in the arts.

The Virtuale Switzerland
The Virtuale Switzerland is a biennale for virtual arts. It focuses on the use of public space and mobile communication technologies, inventing “playful” new strategies to coax the public into the festival as “real” visitors with a unique experience of the virtual. The Virtuale Switzerland encompasses Artworks using Augmented Reality, Urban or Location Based Gaming, and Digital Heritage applications. It is interdisciplinary in nature, bridging areas such as art and technology, digital heritage and tourism, as well as digital culture and art mediation.