Riccardo Mori

An opinion I’ve held for a long time is that Apple so far has done a mediocre job in turning the iPad into a ‘pro’ device. The hardware is fine, the current specifications for the iPad Pro models are more than fine. But the software — and to a certain extent the user interface and usability — is the weak spot. I won’t repeat myself about this; I think I said enough in Faster than its own OS back in November.

At the end of January 2010, this is how Steve Jobs introduced the iPad:

…And so all of us use laptops and smartphones now. Everybody uses a laptop and/or a smartphone. And a question has arisen lately: is there room for a third category of device in the middle? Something that’s between a laptop and a smartphone? And of course we pondered this question for years as well. The bar is pretty high. In order to really create a new category of devices, those devices are going to have to be far better at doing some key tasks. They’re gonna have to be far better at doing some really important things: better than the laptop, better than the smartphone. 

What kind of tasks? Well, things like browsing the Web. That’s a pretty tall order; something that’s better at browsing the Web than a laptop? Okay. Doing email. Enjoying and sharing photographs. Watching videos. Enjoying your music collection. Playing games. Reading eBooks. If there’s going to be a third category of device, it’s going to have to be better at these kinds of tasks than a laptop or a smartphone, otherwise it has no reason for being. 

Now, some people have thought that that’s a netbook. The problem is netbooks aren’t better at anything. They’re slow, they have low-quality displays, and they run clunky old PC software. So they’re not better than a laptop at anything, they’re just cheaper; they’re just cheap laptops. And we don’t think they’re a third category device. But we think we’ve got something that is.

I could use this quote to emphasise how all the tasks Jobs enumerates are consumption-related. That the drive to create such a device came from the need of having some hardware that was more convenient and capable at delivering certain content for people to enjoy. The creative angle came later. Again, in retrospect it’s crucial to notice just how the iPad was not conceived as a creation tool. It’s interesting to realise how Steve Jobs didn’t mention production tasks or creative tasks when he was talking about the thought process leading to the creation of this ‘third category device’. I’m sure Jobs was aware that, with the right applications, the iPad could do more than just being a vehicle for content consumption. Still, that didn’t seem to have been a priority.

Instead, what I want to emphasise in this quote is this part: In order to really create a new category of devices, those devices are going to have to be far better at doing some key tasks. They’re gonna have to be far better at doing some really important things: better than the laptop, better than the smartphone.

Far better at doing some key tasks. Better than the laptop (but let’s just say better than a Mac or any other traditional computer), and better than the smartphone. Think about that.

For the first few iterations of its existence, the iPad and iOS delivered on their mission. In 2010 I had a brand-new MacBook Pro and I was still making the most of my iPhone 3G, but I couldn’t wait to get an iPad. I wanted to use it especially for reading, so I waited very patiently for an iPad with a retina display. And in 2012, with iOS 5, the iPad was still a great device to do everything it was designed for. A fast device with an intuitive operating system with an extremely low learning curve. Some apps for more creative tasks had appeared, and with the addition of a Wacom stylus I had fun at drawing and painting some stuff.

Then some people got very excited about the iPad, and another question arose: why can’t we use the iPad for all kinds of tasks?

That’s when things started to go awry, in my opinion. 

Because while it’s technically still true that an iPad is better at doing some key tasks — better than a traditional computer and better than a smartphone — it’s not better at doing everything.

The integration between the hardware and the software Apple is renowned for means that the software running on an Apple device is (ideally) optimised to grant the user the best experience of what the device has been designed to accomplish. “The iPad is just a big iPhone” was a common criticism back then, and it was a misguided remark, because a few of the iPad’s key strengths came exactly from it being like a big iPhone. The familiar gestures people had quickly learnt to master the iPhone’s user interface still worked very well to operate an iPad. At the time, there weren’t any significant functional changes in how iOS 5 or iOS 6 worked on an iPhone and on an iPad. Apps optimised for the iPad needed a bit of user interface retouching and rethinking, but as far as the user interaction was concerned, there was nothing particularly disruptive. Things worked well. Users didn’t need additional training or additional attention to master an iPad.

But in order to accomplish additional tasks — especially complex tasks that require a certain degree of interoperability among apps and services — just resorting to third-party ingenuity was not enough. The iPad’s operating system needed changes and improvements. Which of course, inescapably, meant an added layer of complexity. As I observed back in 2016:

In iOS’s software and user interface, the innovative bit happened at the beginning: simplicity through a series of well‐designed, easily predictable touch gestures. Henceforth, it has been an accumulation of features, new gestures, new layers to interact with. The system has maintained a certain degree of intuitiveness, but many new features and gestures are truly intuitive mostly to long‐time iOS users. Discoverability is still an issue for people who are not tech‐savvy.

[See also Tap swipe hold scroll flick drag drop]

I don’t mean to dismiss the efforts Apple has done to make iOS work better on supposedly ‘pro’ iPads, but it’s undeniable that iOS has matured very slowly on this front. On iPhones, I believe it’s still a great operating system, because it still delivers on what you’re supposed to accomplish with a smartphone. The hardware/software integration is tighter there. On the iPad, my impression is that things have been messier, less focused, less optimised to make the most of it. Now, if you caught me in a more exasperated mood, I’d probably put the blame on Apple, saying that they could have done a better job, etc.

But the thing is, a touch interface can only do so much. There are still a lot of tasks for which a traditional computer is better and more versatile, and there are tasks for which a smartphone is better, because (among other things) certain touch gestures are simply more effective on its smaller screen. Some will undoubtedly insist that an iPad today can do anything a traditional computer can, and I may even agree on a theoretical level, but the fact is: just because an iPad is better than a computer or a smartphone at certain tasks, it’s not necessarily better at doing everything these other two kinds of devices were designed to do.

While successful, the iPad hasn’t been as revolutionary as many hoped (including some Apple executives, I presume), and in recent years Apple has made repeated efforts to turn it into a revolutionary device, perhaps paying too much attention to some hardcore iPad fans in the tech sphere. Apple has even neglected the Mac in the process, but so far the outcome has been underwhelming on both fronts. We have a generally weaker Mac, with serious hardware design flaws in its laptop line, and an operating system that hasn’t really evolved since probably Mac OS X 10.9 Mavericks. Then we have an iPad platform that hasn’t really improved all that much — the main differentiator between a regular iPad and an iPad Pro is essentially their technical specifications; it’s a hardware thing. Not a revolutionary new user interface or paradigm. Not even a tighter hardware/software integration (if anything, we’re in for yet another layer of complexity and new gestures and actions to memorise).

21st Century tablet

My habits and preferences betray my somewhat long history with computers and technology. I didn’t grow up with smartphones and tablets. My first home computer was a Commodore VIC-20. I was 27 when I first used a mobile phone. Despite what some people may think, I’m not averse to change and my brain is still flexible enough to pick up new habits or change old ones. What happens when you get older, though, is you tend to consider more often whether changing a habit or rethinking a workflow is actually worth it. And what I’ve always said about the iPad in this regard is this: if I’m faster, more efficient, more productive with a Mac (or, in certain fringe cases, with an iPhone), why should I learn a more convoluted path to be able to do the same thing — but more slowly and less efficiently — on an iPad? 

This state where you can simply have an iPad that does everything you need, without compromises, and does it better than any other class of device, is still pretty much ideal. Unless we witness a major hardware or software redesign, the trajectory the iPad is following is that this device is going to progressively resemble a Mac with a touch interface. We’ll ultimately have a device whose operating system will reflect a general reinvention of the wheel, feature- and functionality-wise, and whose distinctive features will be its touch interface and its extreme portability, and… that’s it? Where exactly is the progress in this scenario? You may tell me I’m simply not considering all the amazing new technologies that can still be added to the iPad in the coming years. Okay. But for now I look at what we’ve got. And what we’ve got is a tablet that at its very core is still the same iPad of almost ten years ago. Sure, it has got cooler to use and more powerful than the original 2010 model. But as someone who looks at technology as a forest and not at this or that tree, I see the iPad as an enormous waste of potential. 

While having a tablet as the iPad was originally intended to be — a convenient consumption device — has been a great addition, I feel that a general mistake on the whole industry’s part (allow me a bold statement every now and then) has been to focus on the iPad paradigm with too much tunnel vision, and not consider other ways to approach the idea of a tablet, both from a functional standpoint and from a user interface standpoint. Other manufacturers just followed Apple’s example and now we have a lot of mediocre alternatives that look and feel just like big phones and try to ape traditional computers for certain tasks. We have a third category of device that, instead of evolving into being something distinctive and even independent from the other two, has become a mix of smartphone and traditional computer envy. (I’m generalising and I’m aware there are exceptions in some of the iPad’s features.)

When it comes to ambitions for a tablet device, I keep thinking that the Newton was on a way more intriguing path than the iPad has been for the past nine years. I know that the technology had limitations. But don’t just put a Newton MessagePad and an iPad side by side and compare the two. Of course it’s going to be an unfair comparison — there’s a technology gap of about 15 years between them. I’m talking about vision. Just take one of the Newton’s basic features: handwriting recognition. Yes, on the first generation of Newton devices it wasn’t great, you may remember the jokes, and so on and so forth. Few people seem to be aware that it got much, much better with NewtonOS 2.x and on later, more powerful devices. I’ve been a Newton user since 2001, and to this day I can turn on my MessagePad 2100, create a new note or document, start writing on the device with its stylus as if it were a paper notebook, and the Newton will correctly understand and translate 99% of my scribbles into legible typewritten characters. It’s something I still can’t do on an iPad. 

And that’s because one day the industry decided that pen computing had no future. So, while using a stylus to draw, paint, or as an input device in certain specialised settings and applications is considered normal and natural, apparently writing on a flat surface with a stylus — something humans have been doing for at least 7000 years — is not. 

Well, if I had to describe my kind of tablet, a tablet I may consider using for productive tasks, I think it would be some sort of Newton on steroids at its core, with an input interface that would use touch where appropriate, and stylus where appropriate. That includes gestures: imagine splitting the screen between two open applications simply by drawing a line with the stylus instead of memorising some sequence you have to do with your fingers and in a certain way otherwise it’s not registered.

It would have an amazing handwriting recognition engine: so fast and accurate that it would make a virtual keyboard redundant. Mistranslated words could be corrected with a tap, and the tablet’s autocorrect would learn from those mistakes. Machine learning finally put to good use.

It could be easily connected to a Mac/PC, and you could use it as a giant trackpad, as a graphics tablet, as an additional display, even as backup device for sensible data and projects, which could be encrypted on the fly if needed by using biometric identification such as TouchID or FaceID. The exchange of files and documents would be of course seamless.

The user interface would feature a healthy selection of ‘drawing gestures’ and certain drawn elements could be smartly interpreted and subsequently rendered by the OS. Imagine you’re putting together a report and you’re making a draft with a series of items that will have to be organised in a table. You start handwriting the items and the associated data in different columns, just like you would do on a paper notebook. Once finished, you would draw lines along and across the items and the OS would ask you if you want to create a table; you would confirm and you’d end up with a perfectly laid out table you can drag inside the document (if it’s a separate object, otherwise it would already be part of it and you could drag it around to fit in the document’s layout). Once a series of items has been transformed into a table, the system could also handle it with its built-in spreadsheet feature, or you could export it to your favourite application for further editing and refinements.

As you may have guessed, I’m a fan of the old document-centric approach. The application-centric model has its advantages, of course, but I believe that an ideal tablet with an enhanced pen-based input interface could use some document-centric paradigms and it would feel very natural. The tablet’s OS could have a series of core functionalities (or services) that are invoked by what you intend to do. You create a new document and it could be a letter, an email, a financial report, the chapter of a novel, the page design for a magazine, a post on your preferred social network, a spreadsheet, a new webpage for your site, a new post for your blog, etc., and the tablet — via a series of ‘smart agents’ — would either understand what you’re doing or you’d simply tell it via a Create as… / Save as… command once you’re done. The OS could have some basic built-in services (e.g. an HTML/CSS editor for when you’re creating a new webpage), but you could also integrate third-party apps and services to have a richer experience and achieve more specialised results.

Visually, you would have a sort of desktop, but think of it as more of a workspace, not as a container of apps and files. A workspace where you can create things from scratch directly, or invoke/import things to ‘consume’. But even when you’re consuming content, imagine having this intermediate, invisible layer, that lets you manipulate whatever you’re reading or watching or looking at, in case you find something you need. You’re reading an amazing article on a website and want to save that insightful quote for yourself or to reuse in one of your articles? You highlight it with the stylus, either by underlining words or simply by enclosing the quote in a rectangle, and now you have a clipping you can reuse (the system could also save the original URL as metadata, so that the source is always retained). This could work with different kind of content: text, audio, video, still images, etc. You could use your finger or the stylus as an eyedropper tool when you see a particular colour you want to save or use for a project. These are just a few basic examples, but you get the idea. 

I’ve been thinking of an interface and operating system like these for years, and I confess I was excited when the Microsoft Courier research project surfaced back in 2008 [you can still find concept videos on YouTube, like this one or this one (truncated, sadly); also check this video about Microsoft’s Codex prototype, which predates Courier]; and I still think that some of its gestures and user interface ideas are more innovative — at least more intuitive — than what Apple has done with iOS on the iPad. Courier ended up being little more than an investigation, a concept, but it treated the tablet as a tablet, not as a wannabe traditional computer with a multi-touch interface on top of it. 

With this approach, my ideal tablet would certainly have a potentially complex interface, but by including a more robust stylus input and gestures that heavily borrow from the fundamentals of drawing when it comes to manipulating content and indicating intention, I think a lot of the user interface and interactions would be easier to grasp and master. There could be even a ‘tutorial mode’ the user could toggle, and when it’s enabled, certain parts of the tablet’s interface would be subtly highlighted; by tapping on them, the user could be presented with labels or tooltips explaining how to interact with that particular element. 

More importantly, the tablet would share part of the burden when a user wants to accomplish a task — imagine something like predictive text, but applied to many other different actions. Instead of being confused as to how to perform a certain action, the user could start doing something with the stylus, and the OS could offer some suggestions about which actions can be carried out from there. Or, if all else fails, it could ask the user if they want additional help. This, of course, should be a last-resort scenario, because ideally the interface would be so intuitive and discoverable that users wouldn’t need help or tutorials — but at least help and tutorials would be planned and included, and people wouldn’t be left on their own to figure out how to do something. 

In case my examples haven’t been clear enough, my kind of tablet would be strongly focused on applications and services interoperability. Precise, rigorous interface guidelines would ensure a great integration with third-party solutions. Developers could write standalone apps, but also services and system extensions to expand the tablet’s functionality and scope, ultimately contributing to its overall flexibility. In a model like this, workflows would have less friction because you would be adding functionalities and ‘actions’ made available either by the manufacturer or by third parties. 

If this is getting too abstract, imagine an even more reliable and ‘hardwired’ version of what iOS currently offers with Siri Shortcuts or with the older Workflow app. You download/purchase additional modules to accomplish specific tasks. Once added to the system, these modules or ‘actions’ (or whatever you want to call them) would in turn be available and accessible to third-party applications. For example, imagine you could add a “Markdown to HTML” module to the OS. From then on, that action would be available to the built-in text editor, but also to any third-party text editor you may get in the future. If a third-party developer wanted to write a text editor using their own Markdown-to-HTML converter, they could do so, and the user could choose which to use by changing a preference setting. But if a third-party developer wanted to write a certain kind of text editor that is more focused on beautiful typography or other specific aspects, they could do that without feeling the need to also offer text converters. Again, these are just crude examples off the top of my head. Perhaps a few user interface mockups would tell a clearer story, but I hope you’re getting my drift nonetheless.

I think that a tablet with this kind of OS that prioritises modularity, tasks, and app integration, and with a user interface that treats the tablet as a tablet and lets you interact with it by ‘speaking a tablet language’, would make for a versatile device with a good degree of extensibility, and a good degree of independence. You could attach any kind of accessory to it to make your life easier, such as an external keyboard, but the idea is that all you’d need to have is the main device and its stylus. And if you wanted to use such tablet as a mere ancillary device, you could do so by seamlessly connecting it to your computer, and the tablet would become an accessory or extension as needed. 

There is nothing particularly sci-fi in my ideal tablet — just perhaps a rearrangement of a few conceptual pieces — but I understand if some of my ideas sound weird or unfamiliar or unfeasible, especially if you’re satisfied with the way the iPad and iOS-on-the-iPad work today. I think the time has come for Apple to either embrace the interface limitations of the iPad and try to make the best hardware/software integration within those limitations, or to start designing something new from scratch with the express purpose of being a creation-/production-oriented device and operating system.

Let me know what you think, if you like.