Alex Barnett blog : ROA

Why OData Matters (IMHO)

alexbarnett — Fri, 19 Mar 2010 15:24:00 GMT

Earlier this week I was in the MIX10 crowd as Douglas Purdy announced the Open Data Protocol (it was a great presentation - summarized here).

I want to share with you why I think OData could be a very big deal…But before we go there...let's start with the basics...

What is OData? Where Did OData Come From?

To understand the history of how OData came to be, you need to understand how project "Astoria" came to be...I won't go over that again as this is already pretty well documented. ~~Astoria~~ OData has come a long way since.

In a nutshell: Today, OData builds on a few conventions, popularized by AtomPub (see OData AtomPub Format), to using REST-based data services. These services allow resources, identified using Uniform Resource Identifiers (URIs) and defined in an abstract data model (see OData URI Conventions, to be read and edited by web clients using simple HTTP messages (see OData Operations).

An Open Data Protocol for the Web

OData offers a standardized way for programmable data to be made available across the web and in turn allowing "consumers" of that data to rely on a set of conventions to be followed that in turn allows many interesting things to happen if widely adopted...

...And to this end:

As announced, OData has been released by Microsoft under the Open Specification Promise (OSP) "to allow anyone to freely interoperate with OData implementations" . Since then, the W3C has invited the OData team to Bring OData to a W3C Incubator (I haven't seen a public response yet but I urge the team to do so.).
OData is not a Microsoft-only thing and it won’t succeed if it is. The originating philosophy was about transparency in the design process, with an Open end-point as the goal - not a .NET lock-in play (“agree on standards and compete on implementation”). This approach has already yielded an initial set of clients, servers, services, and tools. Today, a number of OData SDKs and libraries are available for .NET, Java, PHP, iPhone (Objective-C) and more – and there’ll be more coming.
There is a small but growing ecosystem of non-Microsoft "producers" and "consumers" - (where OData "producers" include Netflix’s catalog as OData and the VanGuide (a social map of Vancouver Open Data) and Public data from the city of Edmonton as OData) and OData “consumers” - standard web browsers, RIA "data explorers" – such as Sesame OData Browser, and the client libraries mentioned above – p.s. somebody build a javascript-only data browser please!)

Where can OData take us?

The clue is in the OData icon (next to the RSS feed icon. Can you see the similarities?

The big idea here is that in the same way we have the "RSS" feed icon, we'll get used to seeing the "OData" icon on commercial and non-commercial websites everywhere (especially for government-related data). So in the same way you know today that the RSS icon means "get an XML feed for this content", the "OData" icon means "get this web data" - you'll know (and your client will know) what to expect in terms of how to read in, and navigate through and query structured web data sets - and in many cases write against them - using a common syntax.

Q: Right, But So What?

A1: Open Government OData. From Open Government Data Principles

The Internet is the public space of the modern world, and through it governments now have the opportunity to better understand the needs of their citizens and citizens may participate more fully in their government. Information becomes more valuable as it is shared, less valuable as it is hoarded. Open data promotes increased civil discourse, improved public welfare, and a more efficient use of public resources.

That’s great, but it needs to be practicable…And number 5 of the 8 Principles of Open Government Data sensibly states that the data should be (via David Eaves):

5. Machine processable

Data is reasonably structured to allow automated processing.

It'll be down to each government agency (and local government) as to how they decide to implement this principle, but wouldn't it be great if they agreed to a standard (and a powerfully simple, web-oriented one at that)? This is what Jon Udell concluded here:

"The open data movement, in government and elsewhere, aims to help people engage with and participate in processes represented by the data. When you publish data in a fully articulated way, you build a framework for engagement, a trellis for participation. This is a huge opportunity, and it’s what most excites me about OData"

A2: To ODatarize your data is to RESTify your data.

As more data-oriented web APIs come online, each team responsible for the design of each web API is confronted by the same kinds of questions, and each team answers these in their own particular way. Increasingly, “RESTful” is a design goal of web APIs. Great…but what does that mean? How do you expose the data, the relationships between the entities inside the model, and what should the querying syntax look like? Unfortunately, there are as many answers to these questions as there are RESTful web APIs. And there needn’t be.

For to ODatarize your data is to RESTify your data. Do spend the time at the value layer - figure out the way your developers / consumers want to see the data and expose it that way. Do make it easy for devs / consumers to learn / navigate about the data and use it. Do not make them learn about the unique idiosyncrasies you’ve built into your API (or those that leak out of your originating store) :-)

From a developer’s standpoint, OData is ultimately about productivity.

For the OData “Production Developer”: Point at your data store – define your entity model and map it to the data model you already have (so your developers consume / program against the data that makes most sense to them – effectively ORM’ing) and expose as an OData service, inheriting: all the REST characteristics; entity relationship self-discovery; and querying goodness.
For the OData “Consuming Developer”: If you know the web API is OData…great! Pick up a client library, get to the API end-point (data.foo.org/blah.svc). Point and Shoot: Traverse the data model, query it (and bookmark as needed – it’s a URI)…play!

(see links at the bottom of this post to technical content that provides details on all this)

A3: Since the announcement, I’ve seen quite a bit of excitement around the web (especially Twitter) by developers who see the potential here…there is plenty of experimentation going on. At Intuit, my team is also experimenting with ODatarizing some of our data services, exploring how it might be applied across a number of our cloud based data services. And when our team’s architect Tweets that “Looks like #odata is going to be a good fit for our data services”, I know there’s something interesting going on here…

So I encourage you to find out more about OData and get involved.

More Resources

An easy way to set up an OData feed from your SQL Azure database - Stephen Forte
OData interoperability with .NET, Java, PHP, iPhone and more - MSFT Interop team
Microsoft, Bring OData to a W3C Incubator - W3C blog -
Silverlight 4 + RIA Services - Ready for Business: Exposing OData Services-4-ria-services-ready-for-business-exposing-odata-services.aspx - Brad Abrams (MSFT)
Open Data for the Open Web - Douglas Purdy (MSFT)
Producing and consuming OData feeds: An end-to-end example - Jon Udell (MSFT)
OData for collaborative sense-making - Jon Udell
The three laws of open government data – David Eaves
8 Open Government Data Principles
Open Data Protocol (OData) with Pablo Castro - Hanselminutes podcast interview with Pablo Castro (MSFT – Architect for OData)
OData, a RESTful contender for your API - Andy Thompson
Announcing Sesame Data Browser - Fabrice Marguerie
Retrieving Data with the OData SDK for PHP - Brian Swann
Got SQL Azure? Then you've got OData - OData blog
Implementing only certain aspects of OData-only-certain-aspects-of-odata.aspx - Pablo Castro
Every SharePoint 2010 server is a Data Services server - Pablo Castro
Microsoft Creates New OData.org Website for Open Data Protocol – CMS Wire
Microsoft delivers updates on OData, Houston, Dallas - ZDNet
Timeline of Project Astoria - Pablo Castro
How ADO.NET Data Services came to be (formerly known as Project Astoria) - Alex Barnett
previous Astoria / OData posts - Alex Barnett

Joining Intuit

alexbarnett — Sun, 21 Sep 2008 22:02:00 GMT

Firstly - thanks to everyone who has reached out to me in the last three weeks via email, phone calls and comments since I shared the news of my pursuit for the next adventure - I have really appreciated everyone's support and interest in my next steps.

The great news is I'll be joining Intuit as a Group Manager working in a fast growing start-up team responsible for leading the Intuit Partner Platform. Below is a snippet and some links to blog posts and articles that should give you a fairly good idea about where my focus will be a week from now.

Although I'll be working from the Orem office (Utah) to start off with, the plan is to ultimately move to the greater Boston area - another new adventure. I've visited Boston three times in the last year or so and have loved it more and more with each visit, so watch out Boston!

This is a fantastic opportunity for me personally - the team has ambitious goals and an amazing set of existing assets to leverage (see some of the numbers below) in becoming a significant player in the cloud computing space. I look forward to sharing stories of my new journey with you.

Intuit Partner Platform Opens to Developers (Sept 15 2008)

"The Intuit Partner Platform not only gives developers the opportunity to build Web-based applications, but successful SaaS businesses by taking the complexity out of managing infrastructure, hosting, user management, integration and billing. Now developers can focus on developing innovative on-demand software solutions that solve unique and important problems for the four million small and mid-market businesses across the U.S. that use QuickBooks and the 25 million employees that work in those companies.

The platform-as-a-service offering allows developers to combine the powerful open source Flex framework and Adobe Flex Builder and the proven database of Intuit QuickBase to build rich Internet applications that work with QuickBooks data. They can also leverage Adobe AIR to provide additional desktop-like functionality in their applications, such as pop-up notifications, local file system access, local data storage, and the ability to create a fully branded user experience.

"We have now accepted more than 1,000 developers into the program and it is exciting to hear their ideas and energy about what they want to build," said Alex Chriss, business leader for the Intuit Partner Platform. "Customers will benefit greatly from the imagination and expertise that developers use to solve problems facing their specific industries." "

Web 2.0 Expo NYC - Intuit Connected Services (Sept 19 2008)
Intuit Makes SaaS Play Guns For Salesforce (Sept 18 2008)
Intuit’s flexy PaaS (Aug 14 2008)
Intuit enters the PaaS wars (April 28 2008)
Intuit’s Radical New Flex + QuickBase Cloud Platform (April 16 2008)

How ADO.NET Data Services came to be (formerly known as Project Astoria)

alexbarnett — Thu, 21 Aug 2008 02:16:00 GMT

Pablo Castro has recounted some of his timelined memories about how "Project Astoria" evolved from a lunch time conversation to bits in .NET 3.5 SP1 and Visual Studio 2008 SP1 now known as ADO.NET Data Services Framework). Nice write up.

Three memories of my own to add to the story:

1. I was reading up on the whole REST thing in the summer of 2006 - its origins, philosophy and design patterns. I knew there was something interesting going on and some potential dots to join, but I wasn't sure which dots...So I collated and circulated a bunch of research / links to the team, then blogged the links (I liked How I explained REST to my wife. More recently see Explaining REST to Damien Katz). I got a few proverbial (and some literal) blank stares as I shared my enthusiasm for REST, asking how we could apply the ideas to the various projects we were working on. It was Pablo, and (as Pablo attests) Britt Johnston (now a PUM for SQL Business) who were able to develop the initial conceptual leaps into something more concrete like a Think Week Paper and a prototype demo.

2. When it came to brainstorming the code name, the team agreed on a "cloud" theme. A number of proposals were floated around along with their rationales, including "cumulus" and "cirrus". We were then advised that city and town code names were legal-safe. So there we were, struggling to agree on some city or town name we all liked (or at least not hate nor be confused by..."how about Nameless?"...), and then Mike Pizzo's proposal came in: "Astoria - hey, it's the cloudiest city in the USA!" (at least it was in 2006). Sold.

3. I think my favorite memory of all is the reaction Gary Flake provided (of Microsoft's Live Labs) to the prototype Pablo demo'd at one of the pitch meetings: "As God himself would have designed it!" Dr Flake exclaimed..."Cool", I thought to myself - "but does that mean no REST for the wicked?"

Astoria at MIX08 (REST in Vegas)

alexbarnett — Wed, 05 Mar 2008 02:11:00 GMT

As much as I'd love to make it to MIX08 this week, time will not allow me...But if I were, then I'd be going to the following three sessions related to the Project Formerly Known as Astoria:

Wed, March 5th - RESTful Data Services with the ADO.NET Data Services Framework
Fri, March 7th - Accessing Windows Live Services via AtomPub
Fri, March 7th - Building RESTful Real World Applications with the ADO.NET Data Services Framework

Time to Define "Platform as a Service" (or PaaS)

alexbarnett — Tue, 19 Feb 2008 09:27:00 GMT

Before joining Bungee Labs last year, I knew they were on to something big. I mean, really big.

A big idea, an ambitious vision: to provide developers with end-to-end development, testing, deployment and hosting of sophisticated web applications as a service delivered purely in the cloud.

Since we announced our private beta back in May 2007, we've had over 1,500 developers sign up. In January alone we had over 400 developers kicking the tires - not just signing up and disappearing, but 400 returning developers, learning, building and deploying out increasingly sophisticated apps on a fast evolving developer platform, requiring no install of anything on their machine - all through the browser.

And since May 2007, the trend to delivering software as a service (SaaS) has been moving at terrific pace. New web APIs are being made available every month and new announcements by start-ups as well established big players are reinforcing and fueling the acceleration to the inevitable world of cloud computing.

As we announce our move from private to public beta today, we've also tried to articulate the new category of product and service we believe Bungee Connect is at the forefront of defining, the category of Platform as a Service, or PaaS, and our big bet is that PaaS is the next big thing.

So what is a "Platform as a Service"?

In September 2006, Marc Andreessen posted his thought provoking "The three kinds of platforms you meet on the Internet" and it got a fair level attention from the web industry. And we took note. We thought what Marc was describing in his Level 3 definition where:

"A Level 3 platform's apps run inside the platform itself -- the platform provides the "runtime environment" within which the app's code runs.",

...was right, but only partly right. Given Bungee Labs' ambition and vision, we felt there was a lot more to Marc's definition of the highest level definition of an "internet platform", a definition more holistic and comprehensive than a runtime.

But we kept focused, kept working on what we were hearing our developers telling us we needed to fix and improve on Bungee Connect, to give what developers are telling us what they really want - a Platform as a Service - to provide everything required in the lifecycle for the development through hosting of full-on, sophisticated and highly interactive web apps, not just widgets.

As we were readying for our next phase -our public beta - we thought it would be a good time to put a stake in the ground and actually define what we mean when we use the term Platform-as-a-service, and thereby describe the comprehensiveness what Bungee Connect has to offer.

So early this morning, our CTO and Founder of Bungee Labs, Dave Mitchell posted a definition describing PaaS in concrete terms.

What follows is a summary of Dave's post, with a selection of my favorite "soundbites" and ideas, but I suggest you read the whole post for yourself - there's a fair amount to consider:

1) Develop, Test, Deploy, Host and Maintain on the Same Integrated Environment.

"It’s time to stop developing “here” and running “there”. Today, most applications are coded in one environment (usually custom-built for that project by a developer), then tested in another, and redeployed to yet another for production...In a completely-realized PaaS, the entire software lifecycle is supported on the same computing environment, dramatically reducing costs of development and maintenance, time-to-market and project risk. A PaaS should let developers spend their time creating great software, rather than building environments and wrestling with configurations just to make their applications run — let alone testing, tuning and debugging them...Also, an end-to-end PaaS should provide a high productivity Integrated Development Environment (IDE) running on the actual target delivery platform, so that debugging and test scenarios run in the same environment as production deployment.

2) User Experience Without Compromise

"A Platform-as-a-Service must deliver compelling user experiences, with all the richness and live interactivity that consumers have been conditioned to expect....Hiccups like software downloads or plug-in installations, browser dependencies and inconsistencies, or local executables break the web model, and are inherently less secure, less maintainable and less user-friendly. In order to be relevant and popular, PaaS must deliver the best user experience available on the web, comparable to or better than conventional approaches.

3) Built-in Scalability, Reliability, and Security

"Developers should be free to build applications with the comfort that the security of customer data, network traffic, source code (intellectual property) and even server hardware is maintained automatically by the platform through-out application development and delivery."

4) Built-in Integration with Web Services and Databases.

"Applications need to leverage existing software investments in databases, and internal or external third party web services, requiring that the platform offer a wide variety of connectivity options."

5) Support Collaboration

"A PaaS must support both formal and on-demand collaboration throughout the entire software lifecycle (development, testing, documentation and operations), while maintaining security of source code and associated intellectual property."

6) Deep Application Instrumentation

"With instrumentation, organizations can see exactly how users are using the application, the type of performance they are experiencing and any application crashes. This information can also be leveraged to create new business models where costs are tied to actual utilities, rather than flat-rate subscriptions or licenses."

Over the next couple of years we expect to be hearing a lot more about PaaS and how "Y announcement" by "X company" is now providing true a PaaS offering to businesses and developers.

But saying you are providing a Platform as a Service has to mean something, and we think the above definition sets a high but reasonable standard that must be met for any company to claim they are providing a "platform-as-a-service' and legitimately describe themselves as a PaaS player.

The amazing thing is, for me at least, is that Bungee Connect is delivering all of the above, today. From our point of view, delivering PaaS - the real deal - is not statement of Bungee's intent, it's a statement of fact. It's bold, but so is our vision. Yes, we've still a lot to do before we're commercially ready and we think that's coming soon, but so much is already there. Try it out.

Podcast interviews - smart people in the world of the web

alexbarnett — Sun, 20 Jan 2008 15:41:00 GMT

One of the fun parts of my job at Bungee Labs is to partner up with Ted and interview some smart people in the world of the web. We publish these as a podcast series (the Bungee Line - podcast feed here) over on the BCDN blog.

If you have ideas about someone you think we should interview, let me know! We're focusing on topics we think web developers might be interested in the worlds of software as a service and web app development, in particular profiling web apis. Related topics are good too.

I've listed out below our most recent podcasts below...plenty more in the works (previous podcasts are listed here). Hope you like :-)

Alan Lewis on eBay Desktop and eBay APIs

"As product manager for eBay Desktop, Alan Lewis relies on the same web APIs that eBay makes available to all developers. In this edition of the Bungee Line, Alan tells us about what the eBay Desktop is, how it came about, and various details about eBay’s developer program and web APIs. We ask Alan about eBay’s position Oauth and on open source."

Toby Segaran on “Programming Collective Intelligence”

"Since the publication of his O’Reilly book Programming Collective Intelligence: Building Smart Web 2.0 Applications, Toby Segaran has become well noted for his ability to explain easily-understandable algorithms for the kind of deeply complex problems involved in social applications. Toby joins Alex and Ted to discuss some of the high-level concepts that he tackles in his book."

Jon Aizen of Dapper.net

"Jon Aizen joins Alex and Ted to explain how Dapper.net provides a no-fee tool for making almost any structured web site data accessible via a REST API. In a past life, Jon was involved in creating The Internet Archive. Jon also helps the Bungee Line introduce romantic intrigue into the podcast.

Punditry Alert! At the end of this show, Ted and Alex speculate a bit about Android, Google’s open source mobile device platform, the Apache License, and whether Robert Love is involved. Please consider this as another demonstration of Ted’s idiocy, brought to you by the Bungee Line."

Jeff Barr on Amazon Web Services (Part 2)

"In part 2 of our interview with Amazon Web Services evangelist Jeff Barr, Alex and Ted ask Jeff about Flexible Payment Service, virtual user group meetings in Second Life, the Startup Project, and pry at Jeff’s views of possible futures of technologies that developers might anticipate."

Jeff Barr on Amazon Web Services (Part 1)

"Developer evangelist for Amazon Web Services, Jeff Barr tells Alex and Ted about how he became a native Amazonian, his recent visit to “The Business of API’s Conference,” and a bunch of stuff on Amazon Web Services, including: Mechanical Turk, EC2, and S3. Additionally, Jeff explains the newly announced S3 Service Level Agreement*."

8 Trends in Software as a Service Platforms

alexbarnett — Sun, 13 Jan 2008 17:40:00 GMT

To kick off the new year, I presented to around 40 or 50 members of Utah Technology Council (UTC) last week. The title of the topic they asked me to speak about was "Trends in Software as a Service Platforms". I searched around for some ideas and came across two recent posts predicting trends in SaaS for 2008, one by Phil Wainewright "Eight Reasons SaaS Will Surge in 2008" and Jeff Kaplan's post "Top Ten Reasons Why On-Demand Services in 2008". I decided to borrow liberally from these (thanks Phil and Jeff) and mash these two together (along with a couple of thoughts of my own) and present "8 Trends in Software as a Service Platforms" to an audience made up of CTOs and VPs of engineering and development for software companies in the Utah area.

In preparation for the presentation, my boss (Martin Plaehn) at Bungee Labs suggested I write up my presentation as notes blog them afterward, so here they are.

8 Trends in Software as a Service Platforms

SaaS is just part of the web mega-trend
Mainstream opinion says “Yes” to SaaS
Software vendors stampede into SaaS
All is being virtualized
Explosion of Web APIs
Economic factors favor SaaS
Enterprise and SMB IT embraces SaaS
SaaS platforms proliferate (PaaS)

1. SaaS is just part of the web mega-trend

Most of us have witnessed and many of us have been a part of the transformation in the way goods and services have been digitized, virtualized, delivered and consumed. Software, the data behind that software and the functionality that software provides is no different - software is subject to the very same transformational forces.

Just think about how even a class of product that is natively digital - such as software - has been transformed in the way it is delivered and consumed. For prosperity's sake, I've still got a few of those ZX81 software cassettes stashed away somewhere, gathering dust, looking ever more antiquated with each passing year. How will today's mode of software delivery and use look to us in a few years from now?

The web wants to connect things, and that's interesting. But connecting and interacting with "live" data, information and remote functionality make things more interesting.

At the fundamental level, the web connects things. It connects people to people, businesses to businesses, and people to businesses. Since the early 90's, the web has enabled the connection of so many things to so many other things at an ever accelerating rate, and yet we crave even more connectivity. But we increasingly also want the ability to interact with those things.

And it is the nature of these connected things that have changed since the early internet. The early web was good at connecting to static views of information and accessing limited and rigid functional services, very much a read-only mode. Then, as we learned a) the ability to read more dynamic-type information - at least regularly updated, and b) access richer remote functionality, we created whole new opportunities for ourselves. Next, we grew our ability read and write against dynamic, near real-time data and information and to program against remote functionality to create a new class of web applications leveraging those capabilities - and hence a new order of business and experiential opportunities have emerged. Some label this as "Web 2.0".

At its essence, it is the "liveness" of these real-time read-write data, information and functional sources available as "always on" services and the increasing ease to connect to, interact with - specifically change those resources available as live, programmable services that allows us to create new value out of those resources, opening up brand new market opportunities for businesses and the compelling, rich "live" end-user experiences of tomorrow.

2. Mainstream opinion says “Yes” to SaaS

Not surprisingly, Wall Street loves the the predictability of subscription services. It's good for cash flow, forecasting and business planning.

The venture firms also relish the opportunities that are opening up in a software as services-oriented economy. The ability to circumnavigate the incumbent software players with new disruptive technologies and propositions that are significantly easier to try and access for prospective customers compared to traditional software evaluation, along with usage and subscription-based business models verses the old licensing model makes investing in services-based software companies very compelling propositions from the venture firms' point of view. We should also see healthy M&A activity based on these similar opportunities in the coming year.

And then there's the trend for offshore / IT business process outsourcing. These providers will surely get in the game and make their plays through investments in and acquisitions of SaaS vendors that align well with their current core businesses.

Add to that the excitement we're reading about the SaaS space from the IT Analysts, journalists and bloggers, plus the new book by Nick Carr (author of “IT Doesn’t Matter”) - delivered by Amazon to me last week: “The Big Switch: Rewiring the World, from Edison to Google”. I think there's little doubt Carr's excellent analysis of the computing industry as an analogy to the electricity industry's shift to a utility model will be on business bestseller list for much of 2008. His messages resonates with corporate executives and end-users agree with him:

IT is a needless hassle,
it should be as easy as electricity and
be as reliable as a utility

3. Software vendors stampede into SaaS

The Big Software Players are following the early SaaS successes

CRM as a case in point. If you've been following the CRM software market, you'll know about the noises Oracle-Siebel, SAP and Microsoft started to make in the 2007 about what they are are lining up for the 2008 in terms of CRM as a service. Their efforts to emulate Salesforce.com's success delivering CRM as SaaS will be key strategic bets from the incumbents' point of view - and loud, price and functionally competitive propositions from the point of view of their existing and prospective customers.

CRM is just one of the multiple horizontal solution categories to transform from on-premise with traditional licensing model to a service-based delivery and subscription-based revenue model. ERP, supply chain, e-commerce, HR and many more...the horizontal solution list goes on. And then there are the vertical solution players...

Here's another data point to consider regarding the move by traditional software vendors to a SaaS model:

“15-20% of application ISVs have already either begun new skunk works initiatives or gained access to SaaS assets and development experience through M&A activity”
(Source: Key Trends in SaaS: 2008 and Beyond, Saugatuck Technology)

4. All is being virtualized

Virtualization is a technology trend.

Virtualization enables hardware as a service. The demand for virtual machines met by hypervisor software (VMWare, Xen, Hyper-V) and the success of Amazon's Elastic Compute Cloud (EC2) in the last couple of years point to a continuation of further virtualization of applications and hardware. Virtualization is accelerating the move from traditional on-premise software to services.

Virtualization is a business trend.

We continue to become a mobile workforce. The younger entrants into the workforce in service-oriented economies expect and want to be always connected. It's very hard work, if not impossible to get your traditional on-premise applications and centralized servers sitting behind a firewall to serve today's mobile workers. SaaS and managed services meet the needs square on.

5. The explosion of Web APIs is upon us

According to ProgrammableWeb.com, there are 559 commercial and public APIs available today, most of these are new and there are plenty more to come. How many will we see go live this year? And how many private web APIs are there and will be developed and consumed in the coming year?

Data from ProgrammableWeb.com

6. Economic factors favor SaaS

On-premise software requires upfront capital investments
To lower costs, many companies hold back on their capital investments to mitigate their risks, especially in recessions
Adopting on-demand services on a pay-as-you-go basis will be a perfect sourcing strategy for businesses seeking greater cost-controls and flexibility – the utility model

All well and good, but the real economic value of SaaS is that fact that it unleashes new value of previously isolated data silos and functionality.

7. Enterprise and SMB embraces SaaS

When it comes to IT, who doesn't like

Low-maintenance?
Low cost?
Low-resource profile?

IT and business folk like these things, and externally delivered SaaS applications deliver these benefits.

8. SaaS platforms proliferate (PaaS)

The more mainstream SaaS becomes the more the large vendors will be forced to offer effective platforms for ISVs, enterprises and SMBs.

If the move by the software vendors from traditional on-premise software to a services model is to be successful, they will need to provide programmable interfaces - not just end-user interfaces - to their services for their customers. Customers need and want the ability to access, intergrate and create new value out of live, programmable data, information and functionality living in the cloud. And in turn these same customers will want their custom-developed composite applications and integrated data available as programmable services - yet more APIs.

Customers want to unleash new value of previously isolated data silos and functionality through the development of their own applications programmed against those resources. And in turn these same customers will want their own custom-developed composite applications and newly integrated data available as end-user interfaces and as programmable services - yet more APIs. These customer needs will drive the software market to provide platforms to provide businesses and developers with with end-to-end:

programmable services and data integration
application development, testing and collaboration tools
deployment and scalable delivery

...all as a service with a utility model.

(hey...I needed to mentioned Bungee Connect just the once ;-).

2008 will mark a the proliferation of such offerings as "platforms as services" (or PaaS) through 2009, where then the consolidation will begin. Interesting SaaS and PaaS times ahead.

Update 2/20/2008: see "Time to Define "Platform as as Service" (PaaS)

The presentation seemed to go down pretty well and we had lots of interesting discussion throughtout. One of the topics we discussed was data security in a SaaS world. Don Kleinschnitz (VP, Development at Symantec) followed up with a mail linking to his blog covering Security 2.0 topics.

Again - thanks to Phil Wainewright and Jeff Kaplan for their post and to Martin for suggesting I blog this.

REST / ROA / Astoria / SOA

alexbarnett — Thu, 24 May 2007 22:46:00 GMT

Jon Udell, rightly so, is getting excited about the new book RESTful Web Services, by Leonard Richardson and Sam Ruby - a book I myself got excited about in November (REST Web Services and ROA) when I wrote at the time:

"My bet is that this project, the book and the concept of ROA as an effort to standardize aspects of RESTful architecture design is going to do very well, because I agree this is needed. Example provided:

""REST" is an abused term. Many alleged REST services (such as Flickr's, which says REST right on the website) are actually HTTP+POX: service-oriented APIs that happen to use nothing more the basic technologies of the web. Classifying services as resource-oriented or service-oriented makes it easy to see which ones are more and which less RESTful, without wasting time on minutiae.""

(more links to reviews here)

I don't know if John, Leonard or Sam have spent any time to look at the Astoria project (think - RESTful data services) - see links here, but would love to know what they think - in particular regarding the query syntax (good? bad? too complex? too simple?) and where it sits on the ROA (WOA?) to SOA continuum (WOA links) they discussed.

I'll be at MIX07

alexbarnett — Sat, 28 Apr 2007 13:34:00 GMT

I'll be heading down to MIX07 tomorrow. Let me know if you want to hook up (ping me at )

There will be a torrent of Microsoft product announcements over the next few days - Sam Sethi has made a number of predictions on what some of these will be.

One of the sessions I'll be looking forward to is Pablo Castro's talk that Sam picked up on (I used to work with Pablo in Microsoft's Data Programmability team). If you are going to MIX07, DO NOT miss this:

Accessing Data Services in the Cloud
Speaker(s): Pablo Castro - Microsoft

Come learn about new Microsoft technologies that enable you to make your data available over the Web through a simple REST interface and using open formats such as plan XML, JSON or even RDF. We also discuss the underlying entity framework that makes it easy to model, publish, and program against your data over the Web.

Does REST need a WSDL?

alexbarnett — Tue, 13 Mar 2007 22:15:00 GMT

First read this (written by Marc Hadley)

"This article describes the Web Application Description Language (WADL). An increasing number of Web-based enterprises (Google, Yahoo, Amazon, Flickr - to name but a few) are developing HTTP-based applications that provide access to their internal data using XML. Typically these applications are described using a combination of textual protocol descriptions combined with XML schema-based data format descriptions; WADL is designed to provide a machine processable protocol description format for use with such HTTP-based Web applications, especially those using XML."

Then read this by Sam Ruby:

"Those that merely attempt to produce compliant WSDL based on the available specifications often find problems such as these. But the siren call for viewing the programmable web as merely a serialization format seems unstoppable: the current incarnation is called WADL."

Then read this: Automatic Multi Language Program Library Generation for REST APIs, by Thomas Steiner

"The question of describing (REST) web services in a machine-readable way other than WSDL has been raised before^[9]. However, often the motivation behind was more to get rid of WSDL rather than actually solving the REST description issues. Many suggestions are more or less ad hoc inventions designed to solve particular problems. It is to be noted that with WSDL 2.0 it is possible to describe REST services^[10], but here we want to focus on some examples of non-WSDL approaches. As Sun Microsystem's Norman Walsh writes^[11]: "We know the hard things are possible, we just have to make the easy things easy." "

Then see this, also by Thomas Steiner.

What do you think?

-shutting down comments on this post due to spam-

Hyperdata, interoperable data web and other loosely related concepts

alexbarnett — Wed, 24 Jan 2007 02:12:00 GMT

Apologies in advance. I'm attempting to couple loosely related threads here...I have no conclusion.

Leigh Dobbs originally published Connecting Social Content Services using FOAF, RDF and REST for XTech 2005, where he analysed a sample of eight APIs (including the usual suspects - del.icio.us, Last.fm and Upcoming).

Dobbs speculated as to why those services lacked 'hypermedia' support.

Hypermedia: links and pointers to other resouces that are included in a service response to allow further discovery and interaction of those other resources. Or, to paraphrase Danny Ayers, 'Hypermedia should be hyperdata and Hyperdata is 'the Semanitc Web'.

Back to Dobbs (my bold):

"Lack of linking can be attributed to three factors. Firstly, discussion of the REST style have, to date, been largely centred on correct use of HTTP rather than the additional benefits that acrue from use of hypermedia. Secondly, the RPC style that the majority of the services follow, promotes a view of the API as a series of method calls, rather than endpoints within an hypertext of data. Thirdly, the use of API keys prohibits free publishing of links, as given URL is only suitable for use by a single application, the one to which the key was assigned."

Mike Dierken (a 'Senior Troublemaker') last week provided another reason why the desired 'hypermedia' (or 'hyperdata') support might be missing from those services analysed by Dobbs...again, my bold:

"This review notes that nearly all services do not use hypermedia which I think is unfortunate but understandable. I've always had a problem resolving the desire to be flexible in allowing the internal data identifiers to be used in many situations and the desire to be trivally easy for clients to access other resources by simply using links - the mashup problem. One issue I have is that the server-side software that generates the representation might not know all the possible resources made available by sibling services. Think of a US postal zip-code - if you have a service that provides weather based on zip code, should that representation also be responsible for linking to all other services - either provided by your system or some other server - that could potentially take in a zip-code? My approach is to return both direct links to known resources (tagged appropriately) as well as the short-form of the identifier, the plain zip-code for example. Microformats sort of do this, but it's a style that isn't well applied by data services."

I'll give Danny the final word (for this post at least ;-) on Hyperdata:

"I'm still convinced that working at the syntax/grammar level without any common data model/language (i.e. semantics) is fundamentally flawed when it comes to global interop. On the web interop starts with URIs for ~~concepts &~~ resources and their relations, not angle brackets. [PS. reworded - amounts to the same, but sounds less abstract]"

More <loosely> related reading:

Semantic Web & Data Integration by Kingsley Idehen
Network-Oriented Architecture and Perfect Programming by Jason Kolb
Every webservice is an island and Understanding the Future of Data: Data 2.0 by Alex James (also see Data 2.0, a response by Danny Ayers to Alex James' post)

REST Web Services and ROA

alexbarnett — Sat, 04 Nov 2006 17:58:00 GMT

REST Web Services, a book in progress by Leonard Richardson and Sam Ruby piqued my interest yesterday. From the introduction:

"We want to restore the World Wide Web to its rightful place as a respected architecture for distributed programming. We want to shift the focus of "web service" programming from a method-based Service-Oriented Architecture that just happens to use HTTP as a transfer protocol, to a URI-based Resource-Oriented Architecture that uses the technologies of the web to their fullest."
"...a lot of people have gotten the impression that REST just means "whatever you want to do, so long as you don't use SOAP". That it's a sloppy no-methodology used to justify bad design, malformed XML, and, in particularly troublesome cases, Extreme Programming."

I've come across this perspective a few times and it is justifiable given some of the APIs I've seen described as RESTful. The intro goes on:

"To counter this, REST advocates have come up with a new term, "HTTP+POX", to describe URI-based web services that aren't RESTful. But that just brings back the arguments about what REST is and isn't. Is it like pornography, where you only know REST when you see it? Or is it like communism, where if a service fails it must not have really been REST? Can a service be somewhat RESTful, or is that like being somewhat pregnant? How many resources can dance on the head of a pin?
...We're writing a book to codify the folklore, define what's been left undefined, and try to move past the theological arguments."

My bet is that this project, the book and the concept of ROA as an effort to standardize aspects of RESTful architecture design is going to do very well, because I agree this is needed. Example provided:

""REST" is an abused term. Many alleged REST services (such as Flickr's, which says REST right on the website) are actually HTTP+POX: service-oriented APIs that happen to use nothing more the basic technologies of the web. Classifying services as resource-oriented or service-oriented makes it easy to see which ones are more and which less RESTful, without wasting time on minutiae."

In this post, Leonard further explains the idea behind the book:

"I got the idea for a REST book when I started seriously trying to figure out what was and wasn't REST. I noticed that though certain best practices showed up repeatedly in REST folklore, they never really progressed beyond that point. I decided to write a book that would set down the folklore and hopefully create some new canon, some common ground. I got Sam Ruby to agree to do the hard parts. And when I started work on the book I discovered a basic rule of thumb, a framing device that really focuses on what's important about REST, and makes it easy to tell what's RESTful and what's not.
We call this framing device the Resource-Oriented Architecture (we're not the first, but the other uses are compatible with ours), and I'm going to be writing about it a lot more in NYCB. It's too good to keep hidden in a book along with a bunch of implementation details."

Further Reading

Replacing Service Oriented Architecture with Resource Oriented Architecture by Alex Bunardzic

Danny Ayers' comment on Alex's post regarding the role RDF / SPARQL has to play in the ROA world is worth a read
Abhijit Nadgouda's analysis of Alex's ROA thoughts - *OA - The Different Web Programming Paradigms

WOA VS ROA, by Sam Ruby "The very notion of a link has become practically inexpressible and virtually unthinkable in the vernacular of SOA"
Let's REST! A bunch of links to RESTful resources I've collated here and there (on my old blog) - a good starting point if you're not familiar with all this RESTful business

Alex Barnett Podcasts

admin — Sat, 23 Sep 2006 19:58:00 GMT

Alex Barnett Podcasts - I like podcasting, here are the links to them.

2008 - Podcasts for the Bungee Line

Nate Bowler, CTO of @Task, July 20 2008

@task (or AtTask) is a Utah-based tech company providing a comprehensive, web-based project and portfolio-management package delivered in both a SaaS and on-premise model with a very rich web API set. We talked with Nate about the evolution of their web services design and @task's future product plans in light of the market opportunities presented by the availability of the increasing number of 3rd party programmable web services."

Steve Bjorg, Founder and CTO of MindTouch, June 20 2008

"Prior to founding MindTouch and Steve worked in advanced strategies at Microsoft focusing on distributed systems and web services. We talked with Steve about MindTouch platform, its rich set of web APIs and the implications of a programmable wiki. But MindTouch goes beyond providing open source wiki collaboration and content management - it's delivering a leading edge application integration and development platform called MindTouch Deki. Michael Coté, an industry analyst with RedMonk (analyst firm) picked up on both the podcast interview and news of the latest release of MinTouch Deki."

Update from John Musser of ProgrammableWeb, April 14 2008

"ProgrammableWeb’s John Musser returns to the Bungee Line to give us an update on the API action of early 2008. Alex and Ted apologize for the unfortunate audio treatment to the Bungee sound in the previous episode, promising “never again!” In related news, check out the new intro music for our “Cool Web Tips” segment."

Developer Community Management with Jono Bacon, March 14 2008

"There are few developer communities as large and distributed as that of Ubuntu, perhaps the most popular brand of GNU/Linux distributions available today. Jono Bacon is the first official community manager for Ubuntu. He joins to tell us what he has learned in his 18 months of working with this vast and disparate community."

Social Design with Joshua Porter, Jan 30 2008

"Joshua Porter is a usability consultant, web designer, researcher and blogger specializing in the art of social design for the web whose experience includes five years at world-renowned User Interface Engineering. Josh’s blog (Bokardo.com) is a must-read favorite for UI and web designers and is finishing up his first book, to be published in the next few weeks (details below)."

Alan Lewis on eBay Desktop and eBay APIs, January 15 2008

"As product manager for eBay Desktop, Alan Lewis relies on the same web APIs that eBay makes available to all developers. In this edition of the Bungee Line, Alan tells us about what the eBay Desktop is, how it came about, and various details about eBay’s developer program and web APIs. We ask Alan about eBay’s position Oauth and on open source."

2007 - Podcasts for the Bungee Line

Toby Segaran on “Programming Collective Intelligence”, December 13 2007

"Since the publication of his O’Reilly book Programming Collective Intelligence: Building Smart Web 2.0 Applications, Toby Segaran has become well noted for his ability to explain easily-understandable algorithms for the kind of deeply complex problems involved in social applications. Toby joins Alex and Ted to discuss some of the high-level concepts that he tackles in his book."

Jon Aizen of Dapper.net, November 17 2007

"Jon Aizen joins Alex and Ted to explain how Dapper.net provides a no-fee tool for making almost any structured web site data accessible via a REST API. In a past life, Jon was involved in creating The Internet Archive. Jon also helps the Bungee Line introduce romantic intrigue into the podcast.

Punditry Alert! At the end of this show, Ted and Alex speculate a bit about Android, Google’s open source mobile device platform, the Apache License, and whether Robert Love is involved. Please consider this as another demonstration of Ted’s idiocy, brought to you by the Bungee Line."

Jeff Barr on Amazon Web Services (Part 2), October 7 2007

"In part 2 of our interview with Amazon Web Services evangelist Jeff Barr, Alex and Ted ask Jeff about Flexible Payment Service, virtual user group meetings in Second Life, the Startup Project, and pry at Jeff’s views of possible futures of technologies that developers might anticipate."

OAuth Podcast, with Chris Messina (aka FactoryJoe), Larry Halff (of Ma.gnolia) and Eran Hammer-Lahav, November 3 2007

"OAuth is a big idea, but is it a "solution looking for a problem to solve"? I don't think so. The problem for end users today is real, i.e. authorizing one service to access your data by another service for use by the first service, securely and with control. For developers wanting to develop apps and services that create value through the use of customer data stored on other services, there is no standardized means set of protocols to lean on. Instead, developers need to waste time learning a new way for their app to be authorized to do so for each service provider, having to jump through the various specific means and idiosyncrasies of each service."

Jeff Barr on Amazon Web Services (Part 1), October 18 2007

"Developer evangelist for Amazon Web Services, Jeff Barr tells Alex and Ted about how he became a native Amazonian, his recent visit to “The Business of API’s Conference,” and a bunch of stuff on Amazon Web Services, including: Mechanical Turk, EC2, and S3. Additionally, Jeff explains the newly announced S3 Service Level Agreement*."

Interview with Jeremy Zawodny of Yahoo! - Part 1 and Part 2, October 1 2007

Part 1

"Yahoo!'s Zimbra acquisition, the Yahoo! Mail Web Services APIs, Jeremy's take on the Business Week article discussing Yahoo! Openness, the fruits of Yahoo! Hack Days and the Internal Yahoo! Hack Days initiative, Yahoo! Geocoding API, Yahoo! User Interface (YUI) Library, Yahoo!'s AJAX API for Maps"

Part 2

"What Mash lets you do, Hadoop and Yahoo!'s formal involvement, the WebOS meme, something Jeremy feels strongly about :-) That was fun. Watch out for the discussion on "Meta-API Providers"... More APIs...From b2c APIs to b2b APIs, plus Pipes and democratizing the mashupshpere"

Interview with John Musser of ProgrammableWeb.com, September 19 2007

"Topics covered include Facebook APIs, Amazon's recently launched Flexible Payment Service (FPS) , Google Base, Microsoft's Astoria and relational-data-in-the-cloud programming models and services, SaaS models and API SLAs, REST vs SOAP, "Closed is Still the Old Closed" and plenty more."

-

Older podcasts:

Search & Enjoy! (Podcast) The Power of Search and Recommendation, June 6 2006

"Speakers from Microsoft, Blinkx and Last.fm discussed issues of content regarding search, recommendation, the semantic web and the ownership of data in the Web 2.0 era at Content 2.0 on 6th June 2006."

-

Microformats Podcast, March 31, 2006

"Here's a great podcast for you. All about microformats..."

Guests: Tantek Çelik, Dan Connolly and Rohit Khare. I think it's safe to say these guys know a thing or two about the web and microformats.

-

OPML Podcast, March 10, 2006

"It's all about the draft OPML 2.0 spec and a few other things thrown in such as structured blogging, OPML tools, namespaces and microformats."

Guests: Joshua Porter, Adam Green and John Tropea.

--

OPML and Reading Lists Podcast with Danny Ayers and Adam Green, Feb 12, 2006

"Last year Dave Winer started to push the idea of Reading Lists for RSS. More recently, the idea of Dynamic Reading Lists and Feed Grazing (or Grazing Lists / Glists) has been kicking around.

Its likely that Reading Lists support will become a common feature of Feed Readers / Aggregators."

Guests: Danny Ayers, Adam Green and Joshua Porter

--

Attention podcast : Attention with Steve Gillmor, Feb 08, 2006

"Steve has been leading Attention conversation for some time now. In 2003 he, along with David Sifry (CEO of Technorati), initiated the attention.xml efforts and has since taken on the role as president of the non-profit Attention Trust."

Guests: Steve Gillmor and Joshua Porter

--

MSN Search Champs podcast - Privacy conversation Jan 26 2006

"I attended the MSN Search Champs today....and what a day. Given the recent news and concerns around the data MSN Search, Yahoo and AOL provided to the government, there was a session set up where the 57 bloggers / online experts at MSN Search Champ were invited to discuss the topic with senior MSN management (Senior VP Yusuf Mehdi and VP Chris Payne)."

Guests: Fred Oliveira, Dion Hinchcliffe, Joshua Porter, Chris Pirillo, Thomas Vander Wal and Brady Forrest.

--

Attention podcast: RSS feedreaders and aggregators Jan 22, 2006

"I asked two of the RSS industry's leading lights to join me for a call and share their perspective on the question of where Attention is going with respect to RSS feedreaders and aggregators: Nick Bradbury creator FeedDemon, part of Newsgator (Nick also developed Homesite - sold to Macromedia - and Topstyle) and Kevin Burton of Tailrank (also co-founder Rojo)."

Guests: Nick Bradbury, Joshua Porter and Kevin Burton

--

Structured Blogging podcast with Marc Canter and Joe Reger, Dec 16, 2005

"You might have heard of the Structured Blogging initiative announced earlier this week by Marc Canter and others...there was certainly plenty of buzz and reaction to the news, but not all the reaction was rosy."

Guests: Marc Canter and Joshua Porter

--

Identity Podcast with Kim Cameron and Dick Hardt, Dec 09, 2005

"A couple of weeks ago Joshua and I had a conversation about attention data (as podcasts).

In that conversation we kept touching on the topic of online identities and their management, so we thought we'd invite two pioneers of the identity space, Dick Hardt and Kim Cameron, to a podcast session and discuss how they saw the connections between these two related topics: attention and identity."

Guests: Dick Hardt, Kim Cameron and Joshua Porter

--

OPML = Attention Data, Attention Engines and Tailrank, Nov 12, 2005

"Although we met briefly last week, Kevin Burton and I didn't manage to get enough time to discuss some of the things on our mind at the time, so we got a Skype call together and posted it as a podcast (.mp3, 42mb).

We focused the discussion around what he calls Meme Engines and I call Attention Engines, Tailrank (Kevin's latest project), OPML, RSS and Attention.xml"

Guests: Kevin Burton

--

Web 2.0 podcast, July 01, 2005

"Richard MacManus of Read/WriteWeb and I had a Skype chat this evening and recorded the call Talked about Web 2.0, attention.xml, a bit about RSS, APIs and more."

Guest: Richard MacManus

-

Attention podcast with Joshua Porter, Nov 26, 2005

"About OPML, Attention, and empowering people."

Guest: Joshua Porter

--