On: iOS5 Gestures

@globalmoxie wrote a few months ago about his concerns regarding the new iOS5 multitouch gestures. You can read his article here: link

TL;DR of his article

Apple is using the wrong approach by not providing proper differentiation between App and OS gestures. Ideally Apple should have used Edge Gestures, similar to the ones on Windows phones or WebOS. And also, due to the fact you can perform those gestures anywhere on the screen, your app cannot take advantage of them.

That’s an interesting and very valid point of view.

I think I can offer a different point of view though.

TL;DR of this article

I think there are also some valid reasons for them to do so.

• enforce simplification of 3rd party apps by reserving complex gestures for non-essential device-wide actions
• uniformization of interactions accross platforms: iOS and Lion share interaction patterns

Canvas real Estate

Regarding Apple’s use of the whole app canvas for gestures.

It may not be a bad thing. I’ll start by listing a few ideas and show you my point further down below.

Cognitive friction

Gestures like the single finger tap, drag, swipe have very low cognitive friction because very close to the natural interaction we have with objects. We use 2 or more fingers for grasp, to convey a stronger intention, sometimes precision.

Apple has reserved the more complex gestures that require the use of 4 fingers to implement non-essential global gestures.

I don’t think it’s correct to think about this is a ‘logical’ way. Perhaps we shouldn’t be thinking of the technological details behind the screen, making the separation between App and OS. The reasoning for is that the user doesn’t make this distinction. I think it’s safe to assume that from his point of view, he’s interacting a device as a whole. It has a default state and many extensions he installs.

Apple’s global gestures provide an easy way to manage context. Above the App level you have:

• multitask overview - 4-finger drag up / nudge the App
• multitask stack navigation - 4-finger drag / push the App
• home - 4-finger pinch in / shrink the App

If I’m forgetting any, then it probably has more implementation issues than these.

Still in the subject of cognitive friction, I’d like to hear some feedback from people, but I think the 4-finger gestures are so expressive, provide such a great contact level with the device, that anything else than applying an action to the the full canvas wouldn’t be natural. That is, outside the context of games and other immersive apps.

Complexity

There’s possibly a slight relation to the previous cognitive friction section in here, but yeah, I guess it boils down to it: reducing complexity.

With the complex 4-finger gestures reserved for non-essential interactions, you’re left with the most common ones, which is a good thing: simplify your app. Just because you can implement 4-finger drag/swipe/double tap doesn’t mean you should abuse it.

Yes, Apple is being very paternalist, again, but I think this might be the resoning behind their decisions.

Also, don’t forget there’s a greater plan here being put to action by Apple, which is to introduce more direct manipulation gestures in both their mobile and desktop computers.

Lion also leverages 4-finger gestures, also for device-wide actions, like:

• accessing Mission Control - 4-finger drag up
• accessing App Exposé - 4-finger drag down
• switching between Spaces - 4-finger drag
• accessing Launchpad - 4-finger pinch out
• accessing Desktop - 4-finger pinch in

Makes sense they want to reserve these for their own interaction layer. It’s hard to miss the similarities between iOS and Lion of these.

Edge UI

As Josh put it, this pattern allows the separation between OS and App gestures, I still stand by my reasoning that this separation shouldn’t exist.

From what I’ve observed so far, this pattern doesn’t map very well to the user expectations.

This is based on my observation of people interacting with touch devices. I’ve noticed people tend to direct the control gestures to the center of the screen, sometimes the bottom if they’re lazy, but never initiate a gesture from the edge of the screen.

I don’t think there’s a proper affordance for this pattern.

Conclusion

Refer to TL;DR.

Input is always welcome.

Do read my other writing on ‘Improving gestures in NUIs’, where I try to explain the differences between 1:1 and trigger gestures. And how much I prefer 1:1 direct manipulation over triggers AKA button gestures, read the article for more details.

Are We Naturally Trying To Regain Balance?

I’m thinking the distributed guerrillas battling SOPA and the like, fighting Hollywood and other corporate mass media content producers and distributors that are creating an imbalance of wealthiness.

It’s isn’t the imbalance itself but it’s effects and the framework required to support it.

The extra high profit margins being a side effect and objective, an intended side effect then.

And the lack of innovation, an unintended side effect, caused by how well the profit frameworks work and the moving pieces wherein.

Moving pieces such as old distribution channels still yielding results, synthetic, recipe-based content still being accepted by the end user.

Audience conditioning via the powerful distribution channels that offer convenience but calculated content selection.

Content tailored based on the rule of greatest acceptance, that in turn becomes the content of greater demand.

In the past, the information wasn’t so broadly available, communication wasn’t as effective and immediate as it it today.

People with common ideals couldn’t form a bigger distributed mass as easily as today. A mass that empowers them to make themselves heard and take action against what they deem to be against the common good.

In practice, these distributed interest groups are effectively fighting and forcing the big blobs like the music industry adapt and acknowledge they are no longer in their comfort zone. Content can be monetized much easily and the critical revenue mass can be attained very easily.

As Louis C.K.’s experiment demonstrated, the middleman is no longer relevant, there is no longer the need of help in distributing and monetizing your content. Therefore there is no longer room for the middleman to tap onto that resource.

And that’s a good thing, we’re effectively scaling down, becoming self sufficient and distributed. Without loosing out individuality and opening doors to a wider variety of content due to the possibility of reaching the required niche mass for sustainability of the own niche.

Small people serving other small people, reciprocally. Instead of the usual: Small people plugged into a monolithic sustainable* system.

*sustainable but requiring maintenance, unnatural behaviour, and serving the interests of few.

And did all this began with us? Did we naturally steer away from the old model? Did the market balance itself?

Improving Gestures in NUIs

TL;DR: choose between 1:1 manipulation and trigger gestures, always think them through.

---

I’ve been noticing two different implementations of gesture controls on mobile and desktop devices.

The first one, is what it appears to be a gesture trigger, where an action is fired after the gesture reaches a certain threshold.

OSX gestures such as ‘Show Desktop’, ‘Launchpad’, ‘App Exposé’, all implement this pattern.

The second implementation is a 1:1 manipulation of the user interface.

Using the OSX gestures as reference once again, you can see an example of this approach in the ‘Switch Space’ gesture, where it even allows you to peek into the adjacent desktop and snap back without totally switching between them.

Which one is right?

I think there’s room for both, but personally prefer a 1:1 manipulation of the UI.

The gesture trigger is sort of a fallacy in the metaphor, it’s a button in disguise:

• Feedback - triggers don’t provide feedback until they’re fired, that’s bad for discoverability, bad for the metaphor of a natural user interface;
• Perception - the lack of, or improper feedback affects user perception of the interface, it stops behaving like an object so the user no longer perceives it as such, your app is one step closer to a synthetic contraption.

Still speaking about the NUI metaphor, if you look at real world objects, their state doesn’t vary between on or off, there are usually many intermediate steps.

This isn’t true for buttons, switches and other man-made objects obviously.

But the real world is a mess!

Yes it can be a mess and I’m not advocating for things like 3D desktops where you can place your files anywhere or pin them to the virtual wall or whatever.

Our medium is digital, and even though we’re designing with the real world in mind: masquerading our bits and bytes as flesh and bone, we should take advantage of that medium.

A digital bookshelf should offer the digital benefit of infinite storage, search; while at the same time present a familiar display of data.

Another way to eliminate that mess is to do intention handling and automatically set the state of the UI to a pre-defined one.

Take for instance the ‘Switch space’ gesture in OSX, or its brothers: the home screen switching in Android or iOS.

All of them allow for 1:1 manipulation of the interface, but once the user stops giving input via the touchscreen, the interface decides between returning to the previous state or jumping to the next screen.

Continuity & Consistency

A gesture should offer continuity.

Wether if it’s a trigger or 1:1 manipulation, a gesture should offer continuous effect over the interface.

Don’t force the user to stop and resume the control gesture in order to revert the state of the UI.

An example of a trigger gesture where continuity is respected is the ‘Launchpad’ gesture on OSX, you don’t need to lift your hand to hide it, just perform the gesture in the inverse direction.

The same doesn’t hapen with the multitask gesture on the iPad, where you have to stop touching the screen in order to hide the app tray.

Granted, these are artificial examples, as in: continuity isn’t really required, but it serves to illustrate my point.

I would also say triggers and direct manipulation shouldn’t be mixed in the same app, as it breaks the overall metaphor.

Performance

A very, very important factor in your app, if you offer 1:1 manipulation but if your interface lags way behind the user commands, your metaphor is broken.

To sum it up

Gesture implementations should be as well though as animations in your app. Both tell a story to your user and that story can vary between ‘Hello, Frankenstein’ and ‘Hello, World’.

Notes

Another thing that may impact the overall application metaphor, and still talking about consistency, is the way gesture navigation relates to the information hierarchy.

See Path for example, you can drag the main screen to the left in order to reveal the friends list, but after you select a friend and the app scrolls the ui to the left, revealing the new content coming from the right, you would assume it’s possible to swipe right in order to push the UI and go back to where you started. That is not the case, you have to resort to buttons in order to get back.

Flipboard isn’t immune either. After selecting a topic, direct manipulation is discarded in favor of triggers, which is okay I guess but not consistent with the initial perception, plus, the trigger has a very low movement threshold. I’ve notived they’re doing something smart here, a speed threshold is also in place and slow gestures are ignored.

One idea for Flipboard would be to make the left swipe bring back the latest selected topic when on the main page, this is sort of related to the continuity issue I raised above. Could perhaps be fitted into error handling as the user may accidentally perform a gesture that takes him back to the home screen with no way to get back (remember he’d been drilling down content on a specific topic and we shouldn’t assume he keeps track of the topic at all times).

Also, if you look at certain OSX trackpad gestures videos in the System Preferences, like the Mission Control one, you’ll notice they’ve been manipulated to look like 1:1 manipulation of the UI instead of how they really behave.

And yet another note, do play around with iPad multitouch gestures, the ‘Send application to background’ - 4-finger pinch-in, features 1:1 manipulation and even allows you to revert the action even when the animation has been 100% completed, provided you don’t release the screen. Unfortunately, they first thing people try to do, which is a 4-finger pinch-out doesn’t bring the app back to foreground.

KOMNI

[ communication - community - omnipresence ]

TL;DR

KOMNI is a Location-Based Mobile Social Network designed to promote real-life interactions between strangers or friends, as well as interest, location-based discovery of places, events, people.

Its main interaction vector is the mobile and revenue is generated through deals with mobile operators that benefit from increased revenue from mobile phone sales and service usage, and in turn provide visibility for the service.

The duo was composed of Diogo and I, the context of the project was academic. We tried to get funding but weren’t successful, the project was iced due to lack of resources.

The Story

The project was first conceived in the summer of 2004, its purpose was to be an exercise that would allow us to put our skills to practice.

It matured along the months, sitting a bit idle between our classes, and finally in late 2005 we started working on it.

We worked on the business case, features, technology, development strategy, and came up with a pretty solid project.

These weren’t requirements in our assignment, and weren’t even graded, we chose to do this because we wanted to go the extra mile and this was a very exciting opportunity to prove ourselves, create something we liked and solved some of the problems we saw in the social networks at the time - Hi5, Last.fm, MySpace, etc.

It was also great to be able to count on:

• a psychologist - Ricardo, who helped create a market study to assess the viability of the project based on our target audience behaviour, the sample was of 200 people;
• a star web developer - André Medeiros, who introduced us to Rails and modern web development;
• Dora Nobre, Luís Lobato, Rui Lança, Cândida Martins, Inês Medeiros, and more who provided valuable advice and feedback.

Production started in late 2005 through early 2006, 5 months of part-time development that included familiarization with the technologies being used:

• Linux (dev and local);
• SVN (client & server);
• Rails - Agile Web Dev With RoR;
• RadRails IDE.

We did the setup of our development environment, picked an IDE, grabbed a copy of Agile Web Development with Ruby on Rails and got down to business.

The first prototype was a web-only version of the network, it was mainly a demonstration of how we would structure the network and the experience we envisioned for it with a set of basic features. The mobile discovery based on the user’s interests and location wasn’t implemented as it required R&D, time and deals with Mobile Networks (GPS wasn’t common on mobile devices at the time, we needed to be able to access triangulation data, and don’t get me started on the precision of that data).

A Service Shaped by Need

User Experience

We hated HI5, MySpace for their awful UX, cumbersome navigation and interaction.

KOMNI’s UI was properly structured and tested, with contextual actions clearly separated from website navigation, the design didn’t allow the user to mess it up with superfluous customisations like MySpace did that ultimately led to damaging their brand and be seen as ‘that crappy website full of sparkling stuff and audio players’.

Mobile, Omnipresent

KOMNI was mainly a mobile experience because we loved mobile and the web, and because we wanted to reach out to people who didn’t use social networks because that meant being at home, not outside socializing with friends, and that for them screamed: Looser.

KOMNI would be always with you, proactively notifying you about an event that might be relevant to you because you were frequently near the venue and your interests made it likely you’d enjoy it. Plus, besides suggesting the event it would also suggest people, places and more.

Yes it has a bit of Foursquare, Dating Service, Songkick, Last.fm, but that’s the point, it’s a useful, relevant service.

Utilitarian

Because we were sick of networks where the generated content and interactivity was purely for leisure, no added value, no benefit for the world.

We wanted a network that besides leisure, would provide value for everyone, similar to what Gowalla tried to do, by offering you thematic lists, KOMNI would allow you to discover places/events/people/music, based on your personal preferences and profile.

Data would be curated by people, ratings, comments, views, attendance, etc.

You, as a tourist in a new city, would be able to pick up a KOMNI-enabled phone and explore the city knowing the suggestions made to you were relevant and not and not a generic ‘go here because everybody does’, you’d be able to communicate with the local tattoo community if that was your thing, which normally would require you to have an intimate relation with the locals - you’d still be able to forge those but no special previous connections would be required.

Project Details

Features

KOMNI, despite innovative, was a child of its time, it featured:

• User Profiles
• Messaging
• Groups
• Forums
• Medialog - basic blog meant to be updated via sms/mms/videocall
• Mobile Web App
• Discovery - interest and location-based, proactive suggestions
• Child Tracking - yes: safety, control.

Target Audience

The target audience was relatively broad, individuals aged from 16 to 32, living in a dense populational area, medium income.

Tourists (KOMNI Traveler) were also part of our target audience, ideally with a KOMNI device you could rent/buy, and after a few questions the user would be able to take advantage of the already existing data built upon hundreds or thousands of interactions from the local users eg:

• users who like or have tattoos usually visit this shop
• users who like reading in calm places usually visit this place
• users who like Rambo are going to attend the monthly Rambo meetup in place X
• you have node.js in your skills, there’s a cool meetup coming up!
• you seem to have a Paleo diet, you’re near this neat Paleo restaurant, shall I add this to your todo?
• you’re near another Berliner visiting this town, wanna say to him?
• etc

Monetization

The service would be monetized in various ways:

Partnership with a mobile operator

A Mobile operator would provide visibility for the service on their mobile portal, or create a dedicated one, and data on their user’s location - to overcome technical limitations of the devices at the time, controvertial.

At the time, there was a MVNO whose target audience was very close to ours, called Yorn by Vodafone, and their market positioning was ideal for our service, they even had very irreverent ads like this one here and were frequently disruptive with their pricing and features. Years later a different Mobile Operator would create their own social network called Tag.

The selected Mobile Operator would: * have a service that promotes the usage of mobile communications - sms/mms/videocalls, * sell devices at a low price tied into the service * attract more users by * offering an innovative service * use a service-centric approach to revenue instead of the (still) current practice of charging for device and service

Promoted content

Companies and Users would be able to promote their events, places, businesses in the service, akin to twitter’s promoted content, but in case of KOMNI they would be hyper-targeted and thus increasing their efficiency.

Strategy

Our development strategy was the classic Release Early, Release Often.

We’d implement features gradually, listen to user feedback and metric before evolving the service further.

Project outcome

As mentioned before, the project was put on ice.

I didn’t manage to get funding, due to various motives:

• lack of experience and exposure to vc funding;
• lack of skills required to better plan the roadmap, investment requirements;
• very bad attitude when questioned about what’s a social network and why should they care about it.

I didn’t have the team, network, resources to further develop the service, eventually it faded into the background.

Were I not in Portugal and had these difficulties, KOMNI might’ve had a different fate.

I kept the service details unpublished for years and never wrote about it until today. The ideas in it lost its freshness over the following months. Twitter, Facebook, Foursquare, Tumblr defined a new social landscape.

From time to time I bump into an article such as:

• 2011/12/23 - Facebook Now Scours Your Account To Suggest Real-Life Events

and think of KOMNI.

There’s also a huge list of articles, bookmarks I collected along the years that are related to KOMNI -> pinboard/u:lmjabreu/t:minerva

Minerva - godess of wisdom, was our codename, and there are bookmarks on mobile web apps, DRM and content ownership, mobile browser stats and usage, initiatives from Google, Yahoo, H3, and many other research articles related to KOMNI.

Curiosity

The K in KOMNI comes from a texting trend among the Portuguese youth of replacing the Cs with Ks, it reflects the KOMNI’s target audience.

Improving Mobile Web App Responsiveness - Step 1

Some may say that web apps aren’t capable of providing the level of user experience native apps do, functionality and distribution channels aside, I don’t believe so.

I believe web apps can be as good as native apps, especially on iOS.

From the top of my head, Twitter and Asana mobile web apps are two interesting examples of what can be achieved, but there’s still room for improvement.

There are many factors that may prevent your web app from being as good as a native one, I’ll write a post on that and share my findings. But today:

Improving Mobile Web App Responsiveness

By leveraging Touch instead of the default Click events. 

The problem with Click events on mobile browsers is that they come with a 300ms lag, which is OK for websites, but not for controls or app navigation.

Fixing it is as simple as replacing ” onclick ” with ” ontouchstart “, and I’ve done so on a random library I came accross today: SwipeJS - swipejs.com

Examples (view these on your mobile)

onClick: swipejs.com

onTouchStart: lmjabreu.github.com/Swipe

Notice the responsiveness difference when switching slides it selecting tabs on the touchstart example.

Source if available over at github.com/lmjabreu/Swipe, slapped a chunk of code on both branches to support ontouchstart when available, demo purposes only.

Got Taught a Lesson

Got taught a lesson on how to stay away from trouble by an 11 year old, in a dream.

Long story short, we were talking about trust, he asked me where was the nearest subway station, I thought I knew so I tried to answer, but when I tried to I couldn’t tell him for sure.

He turns around and tells me my answer is wrong, I tell him: oh yeah? Imagine in your friend and need to change the background colour of an HTML page, how do I do that? He replies: I don’t know! You see, I’m not entirely sure how to do that, so I’m not going to lie to you.

He then kept talking about something else, but I was left thinking how he acted better than me in this situation.

To give you even more context, I got to this situation because while walking around in my hometown, I noticed someone getting out of the pope mobile, not the pope, someone else, and there was a huge apparatus around him and the area.

I was stopped by a security and checked for dangerous items. Having turned over my toy guns I had with me, just purchased as a gift among other things, I got stolen by a little girl and couldn’t retrieve what she took as I didn’t want to hurt her or anything, she went away and a group of kids showed up, after witnessing the scene, and started trying to take my laptop away from my backpack, but very calmly as I wasn’t even seen as a threat.

The one who was trying to take my laptop was the kid who would later teach me the lesson, I can’t remember why but we started talking about trust, and the rest I have already told you.

So, this was a kid who had tried to rob me, we didn’t just bumped into each other, and he ended reminding me that sometimes it’s important to admit you don’t know the answer to everything.

All this happened in a dream.

Weird stuff.

Also: totally composed and published using Elements for iPhone, impressed by the traditional touch keyboard efficiency, and it’s a nice workflow.