the bloggard

The roles in web development and the skill sets required to adequately perform those roles have changed greatly with the mainstream adoption of web frameworks such as Ruby on Rails, Groovy on Rails (Grails), Django and the many toolkits for client side development such as jQuery, Dojo Toolkit, Yahoo UI. A critical factor contributing to the adoption of these technologies has been the movement by browser vendors to adhere to the CSS 2 / 2.1 specifications coupled with a vibrant developer community sharing solutions to browser compatibility issues.

Combined, these have changed the domain of web development nearly unrecognizably so over the course of ten years. Where once much of web development involved writing code to persist state to relational databases and servingup mostly static pages for each request we have moved to developing applications that need to scale to support orders of magnitude more users and interact responsively with rich browser based interfaces, behaving more like traditional desktop applications.

that was then

As recent as five years ago, within the consumer Internet space, the roles for a position on a web development team might typically have been categorized as:

• Server side / back end developer
• Web services developer
• User interface designer / developer
• Database administrator

The server side / back end team was the group most likely to contain the most experienced developers and likely responsible for the major decisions as to how the product, project or solution was designed, architected and ultimately implemented. Depending on the size of the project there may have been further sub-divisions to align the work with user interface, core business logic or database programming but for the most part the skill sets required were somewhat interchangeable among these subdivisions.

The web services team was the team responsible for building interfaces to both consume inputs from other web based systems and expose elements of the system being build to other consumers. This team typically consisted of developers with experience of SOAP based APIs and middleware technologies to expose and manage these APIs in a consistent fashion.

The user interface team handled the user experience and was primarily concerned with the look and feel of the application and the user interaction model between as the user navigated the application. It’s likely that these client side developers had some JavaScript skills for simple interactions but were mostly focused on the graphic design and actions leading to new page requests within the web application.

The team that administers the database needs little introduction suffice to say that their roles consisted mostly of ensuring optimal performance from the database with responsibility for supporting multiple applications hosted on the same database instance or cluster.

this is now

Comparing this to the highly sought after, highly desirable and highly paid roles in today’s consumer Internet web application development the difference is striking. A similar categorization of roles can be used to more accurately compare or better, contrast, the skill sets that are more recently in demand:

• Server developer
• API developer
• Client side developer
• Data access developer

The critical skills required for a server side developer are the abilities and experience to build an application to scale to a much larger anticipated user base. Software is much more commonly being offered as a service and the challenges of developing desktop applications such as support and maintenance and managing upgrades have now been replaced with managing and providing a scalable service where application users expect similar if not the same performance as a comparable desktop application.

The sever team is responsible for designing the application for scale. Stateless components support redundancy and allow additional capacity to be added without any downtime or end user impact. De-coupling components support improved separation of concerns and the ability to map computational power to components or clusters of components within the architecture to maximize concurrency and throughput. Designing the application to make the most efficient use of a distributed caching tier (e.g. memcached, redis or ehcache) and understanding the implications for cache clearing and challenges with cache tuning and writing code to take advantage of non-blocking libraries and support asynchronous messages are critical for success.

Gone are the days where much of the work on the server side development team involved designing an effective and reusable MVC framework to support the application interface. With the advent and widespread adoption of a ReST based architecture the API team has become subsumed within the server team. Viewing the server side application as a set of resources and the actions that can be taken on them lends it self very well to providing a stable, maintainable and easily reusable API for all consumers of the application. Such an architecture also better supports scalability by leveraging the capabilities of the HTTP protocol and the caching mechanisms provided by network proxies.

Once the application is viewed as a representation of the state of the application via it’s API, it is a trivial exercise to provide that state in various formats e.g. JSON, xml. The application interface itself becomes the primary consumer of this API and supporting additional clients such as a mobile or tablet based application becomes a non-task from a server side implementation perspective. Depending on the nature of the application, the API may also support the emergence of a community of third party developers and integrators adding to the application ecosystem and likely promoting the use and adoption of the application itself.

Client side development is the one category that has likely changed to the greatest extent. Much of the programming skills that were used on the server side team in the past have now migrated to the client side where the norm is now for Rich Internet Applications leveraging the ReST based APIs exposed by the server side development team. The skills required to build the interfaces for such applications are in great demand. Not only are top quality programming skills needed but also greater experience on the design side is almost a pre-requisite. What was once the domain of the graphic designer, client side developers must be deeply familiar with the CSS specification and compatibilities across browsers. Designing a web page and using CSS effectively has nearly become a software design challenge. The same traits that apply to good software design (e.g. reuse, simplicity, ease of maintenance) now equally apply to building a web page within a web based application.

Without a well defined set of practices and principles to follow for building the client interface the application development can quickly get bogged down in bug fixing and maintenance rather than new feature development. JavaScript is the programming language of choice for the majority of interface development but the selection of the appropriate library (jQuery, YUI, Dojo) or framework (Ext JS, backbone.js) can be a critical factor in the speed of development and maintainability of the finished product. Graphic designers who have the aptitude to become great software developers are idolized. On the flip side client side developers who become more competent with the skills and tools of a graphic designer can in turn be invaluable on any development team.

Arguably one the most challenging aspects of scaling and managing traditional web based applications has been the data access layer and the underlying database itself. Apart from the technical challenges, building a scalable and fault tolerant database layer is often times very expensive. In addressing these, web applications today have trended away from purely relational database models (e.g. Oracle, MySQL) towards document based storage (e.g. CouchDB, MongoDB) or NoSQL based storage models (e.g. Hadoop, Cassandra).

What was once at least well understood academically by the majority of computer science and computer engineering graduates, architecting a data access tier now has become a highly specialized competency and one where real world experience in managing such a system is pre-requisite for competency in the area.

and now what

Given the divergence to a much to a much more siloed set of skills required in each category or roles, it’s no wonder that the provision of services to support web application development have emerged as a thriving market:

• Scalable applications can now be built and deployed to platform as a service providers such as Heroku, Blue Box Group, Cloud Bees and capacity can easily increased to meet demand.
• Providing scalable access to store and retrieve data using services such as Simple DB, Big Table and Azure Table, has emerged as large revenue generators for the likes of Amazon, Google and Microsoft respectively.

Client side development is the one category that does not yet have a comparable service offering yet in the market place. In addition with the mobile revolution that is currently in full swing there are more and more cases of applications being developed for mobile first i.e. the mobile version is available in place of or before the web based version. This itself is revolutionizing the client side programming once again. Although leveraging the same server side API the skills required for mobile application development are radically different from the browser based skills. HTML5 is one standard which may slow this migration or at least level the playing field for client side application developers between browser based and device based interfaces.

Web applications developed in house within corporations to service internal and external business needs have yet to fully embrace what the cloud has to offer in these areas and as such still require the skill sets described in the earlier part of this article. They are however moving to adopt some of the newer frameworks and languages that are commonplace today and in some cases the platforms and services being offered in the cloud to take advantage of the productivity savings they espouse.

Now is the time for developers to retool and realign their skill sets within these new development roles to really maximize their career prospects and put themselves in a highly desirable position to be hired by prospective companies in both the consumer Internet space and by the traditional corporations using the web to support and run a large part of their business.