 In my recent 2008 predictions for the future of Web services and open APIs for enterprise applications, I said that we'd finally see a large scale movement to newer, lightweight Web-based models for opening up our software systems and integrating them together. In other words, heavyweight SOA has finally fallen out of favor and lightweight SOA -- sometimes known as Web-Oriented Architcture (WOA), is in.However, this sea change has long since taken place on the Web and this year will see best practices in this area take another major step forward as we'll examine below. The recent convergence of the Web, SaaS, SOA, and other approaches has also made the boundaries between our architectures and systems increasingly intertwine and blur. As part of this evolution, we have also watched the gains that successful firms like Amazon and Facebook have made by opening up their products on the Web. And strategically, as an industry, we've begun to find it a lot smarter to think in terms of reusable, interconnected open platforms instead of single-play software applications. Along this journey, we have begun a major return to the roots of the globally linked structure of the Web. The rest of this post consists of two sections. One conceptual and one technical:
The daily reinforcement and continuous growth in the fundamental power of HTTP and URL link structure, which is directly driving the Web's overall network effect, has starting giving rise to a new generation of software architects and product designers. This generation has grown up deeply influenced by it and they tend to think about the creation of software in novel, new, highly Web-oriented ways. Though the classical software industry has a long and proud heritage of its own around methodologies, architectural approaches, and design patterns -- proven in the crucible of real-world implementations of years past -- in this decade the Web has managed to exert its own unique, irresistible, and pervasive influence on virtually all aspects of producing software. For example, agile processes have been pushed to the limit and beyond by the forces imposed by the realities of thePerpetual Beta. And the scale of even average sized applications on the Web are now the largest we've ever seen. The absolute necessity of cost-effective operations and the marketplace requirements of embracing the new business models for Web 2.0 applications -- including advertising, user generated content, and rich user experiences -- have also changed the fundamental technical and commercial ground rules for success. As a whole, these changes have been driving a need for new software platforms that are explicitly designed to help us efficiently produce scalable, compelling online applications while also addressing the reality of modern-day Web apps.
These factors as a whole have pushed us into a new era of productivity-oriented platforms that started years ago with languages like Tcl and Perl and quickly moved on to Python, PHP, Ruby. Ultimately we ended up where we are today, with advanced, highly-efficient frameworks for these languages such as Ruby on Rails and CakePHP. These tools now let us create Web applications literally 10 to 20 times more efficiently than the general purpose language platforms of the 20th century, and with both traditional software engineering as well as new Web 2.0 best practices already built in. These improvements have only spurred what can only be called a "radical" movement in the software business, which started with open source software (the peer production of software) starting in the 90s, and where we have arrived, with a dramatic departure from the way we used to look at software languages and platforms in terms of how vertical a software development platform could be before it lost general appeal. These new efficiency gains and vertical focus, however, are almost exclusively aimed at the twins goals of developer productivity and good design. These are both admirable and important goals since programmer time always been one of the leading costs in producing software. Software applications also spend most of their lifetimes in maintenance mode and clean application architectures from the outset can greatly facilitate updates and revisions. However, over the same time frame, the run-time efficiency of our programming environments, partially obscured by a little help from Moore's Law and Nielsen's Law, has been in a major decline. This has been largely intentional, when it comes to supporting improved developer productivity, or entirely unfortunate, such as the general failure of the software industry to figure out how to help software designers fully leverage the now ubiquitous generation of multi-core processors.
One is that the more popular, older programming languages tend to be relatively low level and general purpose and were designed for a different, older set of constraints. This has given us baggage that is often not very applicable to the modern Web-based world. Second, we've become very good at understanding the idioms and "syntactic sugar" that makes developers more productive for Web development and we've put that into the latest generation of programming languages and Web frameworks. Unfortunately, the combined newness of these new Web development platforms and their preference for coding time efficiency in favor of run-time efficiency has conspired to make the results they produce relatively slow and resource inefficient compared to what is potentially possible. Newness in this case is also a kind of performance tax since we just haven't had enough time learning how to make these new platforms perform well at run-time, similar to early versions of Java before the advent of the Just-In-Time (JIT) compiler. Fortunately, efforts like Ruby .NET have made some notable headway in this space recently, but are not commonplace yet. The intent of the rest of this article is to explore the new release of Ruby on Rails 2.0 and examine it in the context of the trends above. The ultimate 10 million dollar question in the Web development platform arena is: Are the developer productivity benefits, including the embodiment of many current Web application best practices, that are conferred by new generation Web development platforms like Rails worth their cost in terms of operational efficiency? Increasingly, whether you're a corporate IT executive or a programmer at an Internet startup, you're going to be facing this difficult decision when you choose your target platform. Questions like "is it worth 5x-10x the programmer time to get run-time efficient software?", or "should I just increase the investment in more processor cores and bandwidth in the data center?" will keep you up at night. Making the wrong choice has potentially serious long-term consequences in terms of what it will ultimately take to maintain and operate your application. A programming platform's implications for operations are particularly pronounced since Web apps require more operational resources on the server-side the larger they grow, unlike traditional, installed stand-alone applications. One way of thinking about the problem is that it's almost never a good idea to bet against significant improvements in computing and network bandwidth. So far we've not yet seen much to indicate that large, regular improvements won't continue for the foreseeable future. Another is assuming that a platform should be used in a slavishly monolithic fashion for an entire application. In fact, as an insightful interview with Alex Payne, a lead developer of one of most well-known Rails success stories, Twitter, shows it often makes sense to move the slowest parts of the app into something faster. This is such a common situation in software development that it's long been codified as the Alternate Hard/Soft Layers pattern. And while these two considerations alone will go a long way towards helping one decide which direction to take, one must also look to where the industry is going as a whole. The new productivity-oriented platforms are here to stay and adopting strategies to use their strengths effectively while being proactive in addressing their weaknesses, is the best route to success with 21st century Web applications. Where's the interface? REST doesn't have a contract description language and essentially uses duck typing.Read about best practices for WOA/Client development. So all of these issues form the lens through which we must look at the modern Web 2.0 applications arena. But let's take an actual look at what we're talking about here. How efficient can these new development platforms really be? And do they are actually encourage us down the right paths in term of modern best practices in the Web 2.0 era? Let's validate this by actually building, hands-on, an entire Web application using one of these new productivity-oriented programming platforms, specifically using the newly released Ruby on Rails 2.0. Those following along will need a little bit of technical skill, but you'll see that these new platforms are tremendously efficient from a developer perspective. In fact, we'll have an application up and running in literally a few minutes after you get your Rails 2.0 environment installed. Building a WOA-compliant Web Application in Rails 2.0We're about to get our hands on Rails 2.0 and build a complete data-driven Web application. But first we have to understand a little bit about REST and WOA since that's the "return to the roots of the Web" story I alluded to in the beginning. Nick Gall originally coined the term WOA, which he defines for us here. It's also called a resource-oriented architecture, but at the core of both conceptions is an approach called REST, which I've previously defined with specifics for those of you not familiar with it. But the key idea is that REST is just a way of using the fundamental protocol of the Web, the Hypertext Transport Protocol (HTTP), to exchange information with anyone else on the Web. REST treats the information on the Web as URL-addressable resources, which includes traditional Web pages but also pure data including XML, video, and audio. REST, which is really just a style of using HTTP, leverages the architecture of the World Wide Web in a natural, organic manner. In other words, REST is the best way we currently know of to open up our Web applications to the rest of the world, an approach I have called the Global SOA in the past. In contrast to object-oriented models for software, or the procedural models used by traditional Web services such as SOAP, REST only uses four methods, those built into HTTP itself: GET, POST, PUT, and DELETE, which themselves operate on data resources located at URI endpoints located on Web servers holding the data (typically a relational database under the covers). Consequently, REST applications tend to have a much larger (and transparent) set of surface area dependencies directly on sets of addressable Web data instead of on bundles of procedural methods through which XML schema instances are passed. Since platforms like Rails embody many of our latest ideas about how best to develop for the Web, it should come as little surprise that the principle creator of Rails, David Heinemeier Hansson, recently observed that the latest release of Rails consists mainly of "a slew of improvements to the RESTful lifestyle." One of the most remarkable things about Rails is how it pays more than lip service to this essential resource-oriented view of the Web. As you shall see, since open APIs are one of the hot topics in the Web applications business at the moment, it's nice to know that every Rails app automatically gets its very own RESTful API. So, let's see this for ourselves... Step 1: Getting on Rails 2.0To explore developing a Rails 2.0 app and creating/using its open Web API, you'll need to install four pieces of software on your test computer.
Now we're ready to start development and testing of our Rails 2.0 application. Keep WEBrick running in one command line window so you can see its debug output (it will show you all the HTTP requests that go back and forth), and have another command line in the Creating Our Open Web Application in Rails 2.0For the purposes of this demonstrator, we're going to build a very simple employee tracking application in Rails. We're going to use the newer syntax in Rails for easy creation of a full Web application with employee record creation, viewing, updating, and deletion. Rails will even create the database tables for us, all the user interface screens (albeit they will be unstyled), unit tests, and even a complete RESTful API, our final end goal. Astonishingly, we're going to create all of this using only two short commands at the command-line. You'll see why Rails is one of the most productive Web development platforms available and this step in particular shows some of the radical ease of use that Rails proponents (myself included) are consistently impressed with. Step 1: Create the basic employee tracking application Rails use a well-designed Model-View-Controller architecture and in our first command, were going to ask it to create all three items for us for our employee tracking database as well as matching unit tests and cross-platform database scripts. To keep it simple, we're only going to track two fields for the employee: their name and their extension. You are welcome to add additional fields but will have to deal with the additional fields in the steps below. In the
You will see a lot of output showing the files that the Rails framework creates to handle the employee data we've just specified. The command itself invoked the Now, believe it or not, the employee tracking application is now mostly finished, the only thing we need to do it to update the database so that it has the schema for the employee records. You can use Rails'
Now, you can run the application by going to the emps directory on the WEBrick instance. Note that
You should see the listing screen of the employee tracking database that looks like the one below:
Click on New emp to create a new employee, and enter the data as seen below (aside: Roy Fielding is co-inventor of HTTP and the person who created the original vision around REST), and click on Create.
The data entered is then transmitted from the browser to the server and stored in the back-end database. You can then view it, destroy it, or add more employees with the user interface that was generated for us by Rails. We've now completed a simple but fully functional Rails application from beginning to end. But what we've come here to see is the fully RESTful open Web API that was created for us along the way. For this we'll need to use cURL to issue the API calls via HTTP to simulate another online program integrating live with our Web application. Step 2a: Invoke the REST API to GET the employee resources Now we're going to exercise all the HTTP verbs on our open Web API to see how it works. The diagram below shows the overall lifecycle of a REST-based resource using our  Let's go ahead use the REST API to pull the data for the employee that we added above. We'll use the handy HTTP utility, cURL, to interact with the Rails application via HTTP. Note that the URL we'll use now has the '.xml' extension added to it. This tells Rails that we're trying to access the XML representation of the resource instead of using the HTML user interface (in other words, we're playing the role of a program instead of a human user.)
Enter the text above in a local command-line or shell with the cURL binary in the execution path. You should see output that looks similar to the following below. It's an XML representation of the employee data in list format, pulled fresh from the server via the REST API, with the <?xml version="1.0" encoding="UTF-8"?>
<emps type="array">
<emp>
<created-at type="datetime">2008-01-11T01:02:53+01:00</created-at>
<extension type="integer">1234</extension>
<id type="integer">1</id>
<name>Roy Fielding</name>
<updated-at type="datetime">2008-01-11T01:02:53+01:00</updated-at>
</emp>
</emps>It's idiomatic in Rails to use the Step 2b: Update an employee resource through the REST API via PUT Let's go ahead and update Roy Fielding's phone extension through the REST API. Since we can tell from the employee list above that Roy's Create a file called <?xml version="1.0" encoding="UTF-8"?> <emp> <extension type="integer">5678</extension> <id type="integer">1</id> <name>roy fielding</name> </emp> Invoke cURL with the following parameters to actually update the phone extension in the resource on the server (and consequently in the database.) The -H parameter sets the header so that Rails knows that an XML representation of the resource is being sent to it. -T makes the HTTP invocation a PUT operation, and the URL of the resource is http://localhost:3000/emps/1.xml where the number 1 corresponds to the
Step 2c: Add an employee resource through the REST API via POST Now we'll add a new employee to our application over the network using the REST API. This employee will be Tim Berners-Lee, so we'll create another XML file called <?xml version="1.0" encoding="UTF-8"?> <emp> <extension type="integer">1212</extension> <name>Tim Berners-Lee</name> </emp> To send this via a POST operation through the REST API using cURL, issue the following on the command line. The
A browse of the employees list via cURL or the employee tracking apps Web forms will see that Tim Berners-Lee has now been added to the application, including the database, via the REST interface. Step 2d: Delete an employee resource through the REST API via DELETE Now we'll go ahead and remove Roy Fielding from the database using our REST API. This process is straightforward and uses the HTTP verb DELETE. You can issue this via cURL using the following command:
You can now verify through the employee tracking Web forms that Roy Fielding's employee record has been permanently removed from the database. ConclusionWe've seen how Rails 2.0 makes it enormously simple to create a database-driven Web application, expose it via a REST API, and manipulate it via a REST-capable client in a clean, no-nonsense manner. Developing similar capability in C++, Java, or .NET environments is currently much more difficult. What you see above however, is only the beginning; Rails 2.0 has added a lot of other support for more sophisticated uses of REST and HTTP. I'll cover these in one of my upcoming posts as soon as I am able. The key point here is that the next generation of Web application platforms puts almost staggering amounts of power in the hands of the average Web developer while providing powerful capabilities like properly formed REST APIs automatically. This further puts the latest best practices for Web apps into places it wouldn't otherwise happen. Open APIs will help power the next generation of online success stories and for this and other reasons, Rails should be on the short list for those considering new Web development efforts. That is, only if they are prepared to do what's necessary to address Ruby's and Rails' shortcomings in run-time performance. Still trying to exactly understand why Rails is such a compelling option? Read an analysis of why platforms like Rails are a major improvement over previous generations of Web application platforms. If you have any trouble getting the code to work, please contact me at dion@hinchcliffeandco.com. |
architecture >
Many of us who have to create the next generation of Web applications have been taking a hard look at the new platforms that have been created for the modern era of very large-scale networked software applications. And I'll be very clear here: While a great many of the old ideas and techniques in software development are as applicable today as they were ten years ago, there are also an whole new set of constraints and enablers for which we have to be very good at optimizing. As the Web begins its 2nd major wave of maturity -- and depending on who you listen to -- there is a considerably less tolerance for older, inefficient methods for developing Web applications; vigorous online competition for marketshare and increasingly online-savvy businesses have a much better sense of what is possible and how much it should cost and when it should be delivered.
Out of all this there has grown a distinct and growing tension between the need to rapidly and inexpensively produce quality software and the requirement for it to scale cost-effectively to millions of users. The simple fact is, which you can readily see in the 
