Some follow-ups, corrections, and expansions. Being correct takes effort.
First, someone very knowledgeable on the internals noted that I was sloppy with the terms ‘datastore‘, ‘GBase‘, ‘GoogleBase‘, ‘GQL‘, and ‘BigTable‘. Mea culpa. Datastore is the most generic term and the specific one for Google App Engine is referred to as the “App Engine datastore“. The App Engine datastore is accessed through GQL, a language reminiscent of SQL. The App Engine datastore is built on BigTable and exposes some of BigTable’s capabilities (see Wikipedia, a formal paper, or video documentation). GoogleBase is an independent Google product that is also built on BigTable. GBase is a guitar search engine and the naturally elided form of “GoogleBase” after saying it a hundred times. Whew! Terms.
Speaking of terms, the contract term dealing with indemnification in the Terms of Service:
13.1. You agree to hold harmless and indemnify Google, and its subsidiaries, affiliates, officers, agents, employees, advertisers, licensors, suppliers or partners, (collectively “Google and Partners”) from and against any third party claim arising from or in any way related to (a) your breach of the Terms, (b) your use of the Service, (c) your violation of applicable laws, rules or regulations in connection with the Service, or (d) your Content or your Application, including any liability or expense arising from all claims, losses, damages (actual and consequential), suits, judgments, litigation costs and attorneys’ fees, of every kind and nature. In such a case, Google will provide you with written notice of such claim, suit or action.
The annoying clause is “(b) your use of the Service”. Given how claims are written in patents, it is entirely likely that use of the API would be an actionable breach if the Google App Engine violated patents. Google could require indemnification by users of the API. For most people and smaller companies, the reputation of Google and pledges to “not be evil” should be sufficient.
Speaking of evil (note the clever transistion), the lazy index evaluation that makes the database look like “read committed” is discussed here.
Finally, high availability is hard, and “9′s” go faster than you remember.
90% (1-nine) is a downtime of 36.5 days per year.
99% (2-nines) is a downtime of 3.65 days per year.
99.9% (3-nines) is a downtime of 8.65 hours per year.
99.99% (4 nines) is a downtime of 52 minutes per year.
99.999% (5 nines) is a downtime of 5.2 minutes per year, or six seconds per week.
A claim of about 2-nines reliability is reasonable. Google App Engine was launched around four months ago in mid-April, so about a day of downtime is 2-nines. It was down on June 17 for some unreported number of hours, and was down again on June 19 and June 25. Add in little outages where various features broke, blocking PayPal, and other nits. There is a list that occasionally reports downtime, but no exact statistics are available. If there were no future outages, and you wanted to demonstrate four nines reliability, it would need to take years to overcome the existing outages.
Keep on working on it. I hope that Google App Engine will be more fun in the future.
[I gave this talk at the BayPiggies Meeting on August 14, 2008. The 'Newbie Nugget' talk ran about seven minutes and used only spoken words. This is my recollection of what I said.]
I drove here today. I got in my car, used the six speed manual transmission, steering wheel, and pedals. I had to know how to change lanes and the rules of the road. I needed to know traffic patterns, and directions from my house to Google. I got here.
It was the most dangerous thing I did today. Traffic accidents kill about 40,000 people a year in the U.S. alone. It’s the number one cause of death for everyone from age five to about my age where heart disease and cancer start catching up. We just take it for granted. That’s not the car I want.
I Want My Autodriving Car! I want to push a button on my cell phone, walk out the door a minute later, and hop into the open door of a car. It would take me where I want to go, take the best route given current traffic, and drop me in front of the door. It would never crash. It would just work. That’s what I want in a car.
Here’s what I want in an application engine. I want to write in one language. I want one set of tools. One naming convention. I want to be able to translate my thoughts into code cleanly. I want the application to be on the Internet and scale and to share that application with everyone through a url or something like the IPhone App Store. That’s what I want in an application server.
Google App Engine is not there yet. Google App Engine is a step along that path, and lets you see where application engines are headed. For the first time, you can see that installing and configuring MySQL on a server and then renting space in a rack for the server will someday sound the same as someone talking about pulling the engine in his car and changing out the rings in the driveway. I guess you could do that yourself, but why? You still need to work in the soup of languages: HTML, CSS, JavaScript, HTTP headers, Flash, Python, and more. You still need to coordinate your tools for each language: editor coloring, debugger, make system, test harness, code coverage, and documentation. It does us good to keep our eyes on the App Engine we want.
So, let’s talk about Google App Engine: what is does, how to write for it, where you would use it and not.
Google App Engine is fundamentally a deployment engine. After you write your application, it provides that application out on the Internet. It’s also very early technology. It sort of supports Django (‘jango’ for those who talk; ‘d-jango’ for those who read), but using the latest version runs up against its thousand file limit. Fully half of the seventy posts a day on the mailing list are about deployment quotas or looking for workarounds to deployment limits. You cannot run any long job that takes more than a few seconds. Still, it’s free.
Writing for Google App Engine is much like writing for any other application engine. You use the WSGI interface, which someone taught me is pronounced ‘whiskey’, that provides a mapping between the requested URLs and the classes to serve them. You write a class that has methods to respond to HTTP messages like get and put. Your codes spits out the HTML and whatever to respond. Sometimes you trigger urlfetch, from urllib, of your own application like remote procedure calls to avoid hitting computation time request limits. You store your data in the Google Base data store.
Google Base can be a problem. The Google Base datastore is not a relational database. It’s a great database for working data you collect from all over the web, like web pages or screen scraping, where you are working with massive amounts of data and some of it is always aging out. It should never be used for accounting. For example, you can’t just run a GBase query to update the total invoices for a month. You need to walk through all the records. If you update the date on an invoice and then retotal the invoices, you might get the wrong answer. GBase doesn’t guarantee the invoices index is immediately updated. It would probably work, but if you were to try running tests to see how often it fails you’d use up your quota and be locked out for a few days.
So where is Google App Engine the right choice? First, it’s free. It’s a good way of learning how to write for application servers. There are some good tutorials and videos and off you go. If your application needs a lot of bandwidth downloading from elsewhere on the internet, it’s probably a good choice. So if you want to try out an idea that’s not for making money, go for it.
Where is Google App Engine the wrong choice? If you are trying to make money, it’s right out. First, the license includes clauses that require you to indemnify Google if the API infringes patents. A Fortune 500 company won’t let you agree to that. Second, its reliability wants to get up to five nine’s (.99999) but is hovering between one and two nines. Third, it’s really early tech. You can watch lots of bugs get filed and occasionally fixed, but they are in the stage of getting it feature complete first.
In conclusion, I want an auto-driving car and I want my great application engine. Google App Engine is like getting a taxi. A taxi does the driving, but I still need to know directions and road conditions, and I still get into car accidents. With Google App Engine, I still need to know the whole stack of languages, my site still gets hacked, and it still goes down a lot. But you can see the future in Google App Engine, and it may turn into something great.
[Clarifications and follow-ups for this article got a bit long. I added some expansions on this blog post.]