Apple Could Power the Web

Update (2015.06.12): Apple is open-sourcing Swift and making it compatible with Linux.

Update (2014.01.26): I view Google's Go and Facebook's D as good evidence the claims made here were correct. Apple has less interest in a web presence and so has not entered Objective-C into the competition.

This is an informal technical note.

Intro

The gist of this note is that Objective-C is a few steps away from taking over the responsibilities of several languages in the web stack. Apple only needs to give Objective-C a little push before it becomes the primary language of the web backend. To be clear, that does not mean that this outcome will happen. This is an opportunity available to Apple. Promoting Objective-C would require a company initiative. Apple could promote Objective-C if they wanted, and Objective-C would then, this note predicts, become the primary player. Objective-C is discussed specifically in the context of the web backend, but most of the discussion applies equally well to other sectors, including financial services backends. There is already demand for Objective-C on the server.

In the past few years, quietly, almost invisibly, Apple has transformed its Objective-C language into the best language available. I have been working with Objective-C since the release of the iPhone App Store in 2008. In that time Objective-C has evolved from a clunky, boilerplate-heavy language, into a tight, efficient joy. It is an amazing tool. Anything that I would not write in C I would want to write in Objective-C, were support available. The changes that made Objective-C this way are fairly recent. Basically, they were introduced in 2011, and weren't supported by enough iOS devices until some time after that. Developers have been able to observe the new features only recently. So I am fairly certain that what I have to say is news.

Assuming Apple takes steps to ready Objective-C you can make several bets about what will happen inside web companies. For instance, it is common among high-traffic websites to begin porting the backend services to Java. Those services should instead be written in Objective-C. From time to time people resistant to Java will write services in Python or Ruby: those also should be written in Objective-C. Critical, long-running applications like Hadoop and Lucene/Solr are written in Java. Going forward we should write those kinds of applications in Objective-C. Long term it would be better to rewrite Hadoop and Lucene in Objective-C now. Massive sunk costs here are likely to prevent that, sadly.

The updates Objective-C needs to take over this market will require some investment on Apple's part, and of course community adoption. Whether Objective-C actually becomes this successful is a matter of chance, and probably a matter of marketing competition. Whether Objective-C, given these investments, is the best choice for an application is no question. It is like asking, all other things equal, whether you should write a new application in COBOL or in Python: there is no debate among reasonable people.

For low traffic sites that can be scripted entirely in Python or Ruby, there may not be much point to using Objective-C. However, sites that expect to grow past a certain point, or who wish to concentrate mobile developers, may wish to start in Objective-C, particularly if templating support improves (which should be simple). For small sites the edge that comes from using a scripting language will diminish over time. If I had to run a monolingual web stack, I would choose Objective-C for the sole language.

The Old (Java, and Certain Scripting Languages)

I have been using Java since 2001. I have used Java at financial companies and at web companies. I participated in the ACM-ICPC using Java. For more on my technical background, see here.

To understand the opportunity facing Objective-C it will help to summarize where Java fails. The original promise of Java was that an application written once would compile and deploy on any architecture. Ignoring that this is false, web shops don't use Java for this reason. Platform inconsistency is an issue for almost no one, and it was never an issue to port correct C/C++ code, universal compatibility being the original promise of C as well. This promise however spurred the creation of the JVM, which was Java's first mistake. The JVM is a nonsense abstraction over the assembly and UNIX system layer. Now code runs through an additional layer, which can be slow, and system interactions must be translated through an otherwise pointless, just-for-this-purpose Java API. In the ideal case, this API replicates the entirety of the UNIX system layer in Java-ese, obscuring any helpful C idioms or UNIX-system knowledge in the process, and creating a pointless set of new knowledge to understand. In the less than ideal case, the API fails to implement system-level functionality and creates a barrier between the application and the machine.

The reason why Java was adopted is it manages memory without introducing the unreliability of a scripting language. This is where Java makes its second mistake: it uses garbage collection instead of reference counting. Because of the widespread adoption of Java over the past decade plus, and because it has been taught in schools, people have come under the impression that garbage collection is the only acceptable way to manage memory, and that a good garbage collector is an indispensable tool. In fact, though it can be hard to see from our vantage point, garbage collection is often a bad choice. Garbage collection makes execution (and memory usage) unpredictable. You cannot postpone garbage collection forever. The more critical the execution, the more you want to postpone garbage collection, but the longer you postpone garbage collection, the more of a problem it will eventually be. This is a disaster for applications that need to scale. I have seen developers spend more time debugging the GC than it would take to go back and write the application in plain C code with manually managed pointers. The debugging of the GC is a process that never ends. In this sense it is a black box. You can tune and tune, but ultimately you cannot make the GC do what you want, which is to go away. If you have worked with reference counting, then you understand that reference counting is simple, fast, and predictable. Why spend so many man-years fooling with a GC?

A third mistake Java made is that it fell into the wrong hands. (Here Java™, excluding things like Dalvik, but the discussion does affect Android Java.) Oracle now owns Java and is a hostile entity. Java is done. Its future as a product is finished. Whatever your relationship is with Java now, expect it to deteriorate. No good comes of Oracle. The brilliant employees from Sun were hemorrhaged.

Most of the remarks on Java apply equally to scripting languages in the relevant area. More damning for scripting languages, if a given scripting language tends to be replaced by Java as a website grows, then there must be something additional lacking in that scripting language. As an aside, tying any new language to the Oracle JVM is destined to be a mistake, for reasons previously mentioned.

The New (Objective-C)

The convenience of Objective-C centers around three main properties:

  1. Object-oriented
  2. C
  3. Automatic memory management

(Savvy programmers will note that, taken literally, the above also holds for C++. One perspective is Objective-C is the language that makes good on the promise of C++.)

The ever-present discussion about functional languages being the next big thing continues, so it's worth reiterating why object-oriented programming is the correct decision for large-scale team programming. Object-oriented programming simplifies software projects. Perhaps not because of anything useful in the theoretical model, but because of the imposed, shared set of assumptions about where code should live and how it should behave. It is primarily a benefit that appears in practice. There is little distinctly useful in the theory of object-oriented programming. Some day maybe there will be a better concept that replaces object-oriented programming.

Functional languages will not replace object-oriented ones. In practice, object-oriented programming lets large teams of competent programmers build usable software. The same is rarely said for functional languages. In cases where functional language applications do succeed, they are often treated as prototypes and rewritten.

Business applications are primarily large, interlocking state problems. The elegance of functional languages is wasted. Large, interlocking state problems cannot be solved with elegant solutions. Problems that can be solved with elegant solutions are replaced and solved until they become large, interlocking state problems. Finer-grained, imperative languages do better with these types of problems.

The second major benefit of Objective-C is that it allows you to drop into C at any time. C gives you maximum, standardized control over the operating system. Unix is written in C, and it is designed to communicate with C programs. There is a minimum of interference between C code and the application that runs on the machine. In part, this is why all of the primary components of the web stack are written in C: Apache (nginx), MySQL (Postgres), Memcache, Linux, most of the scripting languages, etc. Being able to use C types ensures you are able to write efficient programs, and that these programs rely on the same shared knowledge of other efficient implementations. Of course, all of this is optional. You can write programs entirely in the safe environment of object-oriented programming without ever dropping into C. Standard UNIX libraries like zlib can be added to your project directly. You can interface with them using standard C methods, or you can instead use Objective-C wrappers. Objective-C wrappers tend to be quite good. See FMDB for an example of an Objective-C wrapper for SQLite. The Objective-C wrappers around system methods also tend to be good, but nothing is stopping you, for instance, from dropping in and interfacing with Berkeley sockets directly.

The third benefit of Objective-C is that memory is managed automatically using reference counting. ARC (automatic reference counting) was launched in 2011 and works perfectly. ARC code is backwards compatible with non-ARC code. Apple settled on ARC only after suffering through a failed implementation of garbage collection. Apple implemented garbage collection and decided its performance wasn't enough. The aftereffects of this experiment can still sometimes be seen in Xcode, which is saddled with Objective-C plus garbage collection as a legacy feature.

Before the introduction of ARC, Objective-C code had to be manually memory managed. Every object allocation had to have a corresponding [object release] statement, and every additional assignment had to have a corresponding [object retain] statement. In this way programmers essentially still managed memory themselves. The downside of a garbage collector was avoided at the expense of manually counting references. Now Objective-C has improved to the point where it takes care of reference counting for you, and it is excellent. All of the upside of reference counting, none of the downside of memory management. No garbage collection. Objective-C could still do more with respect to reference cycles, but that is primarily a non-issue, and easily solved in future versions.

It is difficult to state the benefits of reference counting, since they are primarily visible in the light of garbage collection, which is the wasted motion. Imagine if you used to have to circle the entire town and then turn around to get to your destination, when now you can head straight there. Reference counting is "the right way" to do things, and garbage collection is a weird invention. Next generation reference implementations for scripting languages should all use automatic reference counting. Objective-C implements automatic reference counting using syntax analysis, but scripting languages should implement it at the execution layer.

Steps Apple Has to Take

Apple tried this a long time ago with WebObjects, which never took off. So the first step is they have to ignore that failure. Things have changed drastically since then. Like the iPhone and the Newton, it's time to try again. Abandon everything about WebObjects and start over with a new push. Give developers what they need to build their own open-source frameworks. People have this figured out now.

The second step Apple has to take is to resolve to support a piece of software that will run outside their hardware. Objective-C won't take off on the web if web shops have to buy Apple's servers or run Apple's server OS. Web shops won't go for Apple servers. All the development, all the monitoring, all the operational knowledge is based on a few varieties of Linux. Web companies like Linux and don't want to leave. OS X is fine for a development machine, but not for a server. The benefits of Objective-C aren't enough to cause web companies to migrate from Linux. Objective-C has to run on Linux. Apple has already backed out of server hardware, so this step should be easier these days. There is precedent here in iTunes for Windows. The opportunity is big enough to justify crossing over. (The alternative is porting everything to Darwin and making Apple-BSD the server OS of choice, which seems like more work.)

While web companies won't accept Apple servers, web developers already develop on Apple computers. Before I switched to mobile development I worked at Etsy. Everybody at Etsy used a Mac. We had one holdout then, a friend of mine, who I learned has recently switched. So of the developers who worked at Etsy when I was there, 100% either used a Mac to develop or use a Mac to develop now. This preference is standard in the industry. Whichever Bay Area web company you walk through the desks are under iMacs and MacBooks. Mac minis are hooked up to all the TVs. Xcode already works with all these devices.

Xcode itself will only need a minimal amount of porting. Some web-ready features will need to be added. Potentially, some of this work already exists because of WebObjects. Xcode doesn't need to be ported to non-Apple platforms. You can expect that developers will have Macs. The same is not true of the deployed Objective-C applications.

All of the Darwin-based Objective-C code has to be ported to Linux. This is likely the hardest piece, development-wise. I am not sure how big of a job this is. It might be trivial, it might be substantially hard. It is certainly within the realm of feasibility for a company the size of Apple. You can already deploy Objective-C if you want, but it has to be on an Apple computer such as a Mac mini. That has to change so that you can deploy to Linux.

What are the benefits to Apple?

  1. Licensing fees
  2. Sales of Macs for development
  3. Increase share of Objective-C at the expense of Java
  4. Get more devs capable with Objective-C, which is necessary for OSX & iOS development
  5. Developer good will
  6. Steer development on the web

Apple can still charge for Xcode, although they should probably give away a free version. They could possibly get away with charging web (or financial) companies large fees to use Objective-C, provided A. the licensing terms are invisible to developers B. the licensing is sensibly implemented in order to account for web-style provisioning, whether it's several expanding datacenters or hundreds of fluctuating instances on Amazon EC2. I've watched large deals founder because the out-of-date licensing institution would not agree to sensible terms with respect to ephemeral EC2 instances. C. small companies aren't charged for the software until they grow to the size where paying for it is of no concern. Nobody will risk using it if it looks like it's going to be a headache.

Other Major Factors in Support of Objective-C

Earlier I gave the three biggest reasons that make Objective-C the language of choice, but there are a host of contributing factors.

  1. Based on Clang/LLVM, driven by Apple
  2. Most important language for mobile devices
  3. Increasingly excellent community support (largely via mobile devs, GitHub)
  4. Grand Central Dispatch is a better way to handle threading, queues ("threading for OO")
  5. Xcode is an excellent IDE, with tolerably good git support
  6. Tightened (& tightening) syntax: @literals, automatically synthesized properties

Perhaps most interestingly, Apple drives the development of Clang/LLVM, which is quickly displacing GCC as the compiler of choice. That is big news. Clang has better licensing, better error reporting, and compiles faster than GCC. Development on the project moves faster. Clang has the momentum and is set to overtake GCC. Clang is cool.

Objective-C is also the most important language for mobile devices, which we expect to play an increasingly large role in our lives. If Objective-C is going to be running the foreground applications on iOS, why shouldn't it spread to the background operations occurring on the internet?

The Grand Central Dispatch toolkit is an excellent improvement on traditional threads. Many simple types of multithreaded applications are trivialized with Grand Central. Queue support is built in. Many common problems have system level solutions such as "dispatch_once". Grand Central is simply a better way to think about concurrent execution. And if you still prefer to use pthreads, you can do that, too.

Parting Thought: Syntax is a Red Herring

Objective-C's syntax has improved in many ways. Number objects can be created directly using the @1 syntax. Collections like dictionaries and arrays have useful shorthands. Synthesized properties and methods that used to have to be specified explicitly, a significant source of boilerplate, now appear for free. IDE auto-completion works wonderfully. The library import process is less tedious than in Java. The syntax in Objective-C is solid. It used to be so-so. Now it's solid.

Apple's suggested style is to use long names for methods, and this does work better on large projects. But if you don't like it, you don't have to use it. You can write very tight Objective-C code if you want to. It isn't (and very likely couldn't be) better than all scripting languages. It is now better than Java, and better even than a few scripting languages.

Ignoring that Objective-C's syntax is now solid, syntax has little effect on the applicability of a language to a domain. Performance and ability to complete tasks are the dominant concern with any language. Time spent typing is a negligible concern in any project. That's why Java displaces other languages who syntaxes are arguably superior. For all the carping PHP is (or was) the dominant web scripting language by a large margin. This happened because of performance and ease of use, and not because of any inherent superiority in syntax.

What this should be taken to mean is that, in a comparison with any popular scripting language, Objective-C will win, since while the scripting language may have superior syntax, it is also sure to be incapable of accomplishing many things which are trivially accomplished in Objective-C.