Alex Barnett blog : Data

Why OData Matters (IMHO)

alexbarnett — Fri, 19 Mar 2010 15:24:00 GMT

Earlier this week I was in the MIX10 crowd as Douglas Purdy announced the Open Data Protocol (it was a great presentation - summarized here).

I want to share with you why I think OData could be a very big deal…But before we go there...let's start with the basics...

What is OData? Where Did OData Come From?

To understand the history of how OData came to be, you need to understand how project "Astoria" came to be...I won't go over that again as this is already pretty well documented. ~~Astoria~~ OData has come a long way since.

In a nutshell: Today, OData builds on a few conventions, popularized by AtomPub (see OData AtomPub Format), to using REST-based data services. These services allow resources, identified using Uniform Resource Identifiers (URIs) and defined in an abstract data model (see OData URI Conventions, to be read and edited by web clients using simple HTTP messages (see OData Operations).

An Open Data Protocol for the Web

OData offers a standardized way for programmable data to be made available across the web and in turn allowing "consumers" of that data to rely on a set of conventions to be followed that in turn allows many interesting things to happen if widely adopted...

...And to this end:

As announced, OData has been released by Microsoft under the Open Specification Promise (OSP) "to allow anyone to freely interoperate with OData implementations" . Since then, the W3C has invited the OData team to Bring OData to a W3C Incubator (I haven't seen a public response yet but I urge the team to do so.).
OData is not a Microsoft-only thing and it won’t succeed if it is. The originating philosophy was about transparency in the design process, with an Open end-point as the goal - not a .NET lock-in play (“agree on standards and compete on implementation”). This approach has already yielded an initial set of clients, servers, services, and tools. Today, a number of OData SDKs and libraries are available for .NET, Java, PHP, iPhone (Objective-C) and more – and there’ll be more coming.
There is a small but growing ecosystem of non-Microsoft "producers" and "consumers" - (where OData "producers" include Netflix’s catalog as OData and the VanGuide (a social map of Vancouver Open Data) and Public data from the city of Edmonton as OData) and OData “consumers” - standard web browsers, RIA "data explorers" – such as Sesame OData Browser, and the client libraries mentioned above – p.s. somebody build a javascript-only data browser please!)

Where can OData take us?

The clue is in the OData icon (next to the RSS feed icon. Can you see the similarities?

The big idea here is that in the same way we have the "RSS" feed icon, we'll get used to seeing the "OData" icon on commercial and non-commercial websites everywhere (especially for government-related data). So in the same way you know today that the RSS icon means "get an XML feed for this content", the "OData" icon means "get this web data" - you'll know (and your client will know) what to expect in terms of how to read in, and navigate through and query structured web data sets - and in many cases write against them - using a common syntax.

Q: Right, But So What?

A1: Open Government OData. From Open Government Data Principles

The Internet is the public space of the modern world, and through it governments now have the opportunity to better understand the needs of their citizens and citizens may participate more fully in their government. Information becomes more valuable as it is shared, less valuable as it is hoarded. Open data promotes increased civil discourse, improved public welfare, and a more efficient use of public resources.

That’s great, but it needs to be practicable…And number 5 of the 8 Principles of Open Government Data sensibly states that the data should be (via David Eaves):

5. Machine processable

Data is reasonably structured to allow automated processing.

It'll be down to each government agency (and local government) as to how they decide to implement this principle, but wouldn't it be great if they agreed to a standard (and a powerfully simple, web-oriented one at that)? This is what Jon Udell concluded here:

"The open data movement, in government and elsewhere, aims to help people engage with and participate in processes represented by the data. When you publish data in a fully articulated way, you build a framework for engagement, a trellis for participation. This is a huge opportunity, and it’s what most excites me about OData"

A2: To ODatarize your data is to RESTify your data.

As more data-oriented web APIs come online, each team responsible for the design of each web API is confronted by the same kinds of questions, and each team answers these in their own particular way. Increasingly, “RESTful” is a design goal of web APIs. Great…but what does that mean? How do you expose the data, the relationships between the entities inside the model, and what should the querying syntax look like? Unfortunately, there are as many answers to these questions as there are RESTful web APIs. And there needn’t be.

For to ODatarize your data is to RESTify your data. Do spend the time at the value layer - figure out the way your developers / consumers want to see the data and expose it that way. Do make it easy for devs / consumers to learn / navigate about the data and use it. Do not make them learn about the unique idiosyncrasies you’ve built into your API (or those that leak out of your originating store) :-)

From a developer’s standpoint, OData is ultimately about productivity.

For the OData “Production Developer”: Point at your data store – define your entity model and map it to the data model you already have (so your developers consume / program against the data that makes most sense to them – effectively ORM’ing) and expose as an OData service, inheriting: all the REST characteristics; entity relationship self-discovery; and querying goodness.
For the OData “Consuming Developer”: If you know the web API is OData…great! Pick up a client library, get to the API end-point (data.foo.org/blah.svc). Point and Shoot: Traverse the data model, query it (and bookmark as needed – it’s a URI)…play!

(see links at the bottom of this post to technical content that provides details on all this)

A3: Since the announcement, I’ve seen quite a bit of excitement around the web (especially Twitter) by developers who see the potential here…there is plenty of experimentation going on. At Intuit, my team is also experimenting with ODatarizing some of our data services, exploring how it might be applied across a number of our cloud based data services. And when our team’s architect Tweets that “Looks like #odata is going to be a good fit for our data services”, I know there’s something interesting going on here…

So I encourage you to find out more about OData and get involved.

More Resources

An easy way to set up an OData feed from your SQL Azure database - Stephen Forte
OData interoperability with .NET, Java, PHP, iPhone and more - MSFT Interop team
Microsoft, Bring OData to a W3C Incubator - W3C blog -
Silverlight 4 + RIA Services - Ready for Business: Exposing OData Services-4-ria-services-ready-for-business-exposing-odata-services.aspx - Brad Abrams (MSFT)
Open Data for the Open Web - Douglas Purdy (MSFT)
Producing and consuming OData feeds: An end-to-end example - Jon Udell (MSFT)
OData for collaborative sense-making - Jon Udell
The three laws of open government data – David Eaves
8 Open Government Data Principles
Open Data Protocol (OData) with Pablo Castro - Hanselminutes podcast interview with Pablo Castro (MSFT – Architect for OData)
OData, a RESTful contender for your API - Andy Thompson
Announcing Sesame Data Browser - Fabrice Marguerie
Retrieving Data with the OData SDK for PHP - Brian Swann
Got SQL Azure? Then you've got OData - OData blog
Implementing only certain aspects of OData-only-certain-aspects-of-odata.aspx - Pablo Castro
Every SharePoint 2010 server is a Data Services server - Pablo Castro
Microsoft Creates New OData.org Website for Open Data Protocol – CMS Wire
Microsoft delivers updates on OData, Houston, Dallas - ZDNet
Timeline of Project Astoria - Pablo Castro
How ADO.NET Data Services came to be (formerly known as Project Astoria) - Alex Barnett
previous Astoria / OData posts - Alex Barnett

Connecting Clouds: Intuit Partner Platform and Windows Azure

alexbarnett — Wed, 20 Jan 2010 16:33:00 GMT

How about I break my blogging “hiatus” by sharing some cool stuff the Intuit Partner Platform team has been working on for a little while that involves Windows Azure?

OK, then… :-)

This morning, Intuit and Microsoft have announced very cool news for Intuit and Microsoft developers and for Small Businesses…

From the IPP team blog post, Alex Chriss, Director of IPP (er…my boss):

“Today, we’re thrilled to announce an alliance between Intuit and Microsoft that brings IPP a giant step closer to our ecosystem vision. Starting today, SaaS developers can access the beta of the Windows Azure SDK for IPP - a set of tools, code samples, and services, designed to make it easy for developers of SaaS applications developed on Windows Azure to federate those SaaS apps on to IPP and sell them to millions of Small Businesses in the Intuit Workplace App Center.”

Connecting these two clouds has been fun and it’s just the start…the bits we’re releasing today is a beta…but fully functional: it provides everything an Azure developer needs to federate their apps on to IPP. In the v1.0 release of the Windows Azure SDK for IPP (expected to launch sometime in February) will also include built-in support for IPP’s Intuit Data Services, the web API that allows those SaaS apps to fully integrate with QuickBooks customer data and program against the common data model and cloud repository that all Intuit Workplace App Center leverage…this is how the SaaS apps from different vendors as well as Intuit’s SaaS offerings work together at the data level.

So the developer story is pretty awesome.

The other aspect to the news released this morning is about the plan to federate Microsoft Business Productivity Online Standard Suite (BPOS) on to IPP and become one of the great set of apps available on Intuit Workplace App Center. BPOS includes a set of messaging and collaboration solutions hosted by Microsoft, and consists of Exchange Online, SharePoint Online, Office Live Meeting, and Office Communications Online.

I recorded a short interview with Jeff Collins, Group Architect for IPP, and Jarred Keneally, Developer Support Engineer to talk about what’s in the SDK and what’s coming soon. Enjoy!

Some more info links:

Official Press Release at Microsoft.com
IPP Team blog post: Intuit Partner Platform + Windows Azure = Win for Small Businesses
IPP Dev Center – Windows Azure SDK for IPP
Intuit Workplace App Center

Update - some reactions to the news coming in now:

Dennis Howlett - ZDNet - Intuit and Microsoft partner: more PaaS to put in your aaS environment

My take: Intuit is joining a growing band of apps vendors that see PaaS as a way of delivering all sorts of aaS functionality, expanding its reach, developing deep domain expertise and helping it accelerate growth. These are bold ambitions and fit well with the idea that a single cloud platform should provide the ecosystem framework needed to achieve these goals. There is no reason why the SMB market should not benefit from these initiatives so at this level it is good to see both Microsoft and Intuit step up to the plate of opening up access to a large group of developers.

Ben Kepes - CloudAve - Intuit and Microsoft Sign Deal to Serve SMBs

"This really is massive news for anyone involved in small or medium business – be it as a business themselves or in anyway selling technology products or services to SMBs....APIs are great – wonderfully valuable things that allow applications to work together. But a common data model of the sort that the IPP is built around, is even better, allowing applications to be built from the start around an underlying and consistent model of data."

Sam Diaz - Between The Lines - Microsoft and Intuit become cloud partners

"The idea, of course, is to link Microsoft’s business applications to the financial data that’s found within Quickbooks to help businesses operate more efficiently. For months, Intuit has been working to push the cloud and open its arms to developers....In July, Intuit launched an open-source community where users could share information to enhance the apps on Intuit’s platform. Prior to that, the company announced Federated Applications, which allows developers to use any programming language, host those apps on any cloud infrastructure and connect them to Intuit’s platform, marketing them to business customers who use Intuit products."

Mary Jo Foley - Microsoft partners with Intuit to shore up Redmond's small-business cloud play

“Customers don’t want a one-point small-business solution. They want a whole suite,” said Walid Abu-Hadba, Microsoft Corporate Vice President of the Developer & Platform Evangelism. He said that this kind of partnership with Intuit was an example of how Microsoft plans to address the needs of small-business developers and customers.”

Dave Rosenberg, CNET,

“According to Alex Chriss, director of IPP, the technology took just a few months to build because the strategy was already aligned. The integration is based on a common data model that is an extension of Quickbooks data. The data model lives in the cloud and developers are able to use the SDK to integrate between the desktop, cloud, and other applications in the ecosystem…For well over a year I've been suggesting that Microsoft needs to take advantage of it's massive developer base to make any real progress with Azure. While this deal with Intuit is still a bit of a baby step, there are many other applications, both online and off, that could use Azure for a variety of purposes. This bodes well not just for Microsoft but for the cloud in general.”

The Microsoft Blog - Microsoft, Intuit team up to encourage cloud apps

“Azure is Microsoft's new platform as a service (PaaS) offering, designed to automatically manage and scale applications hosted on Microsoft's public cloud. Fueled by Microsoft's vast network of partners familiar with its programming languages, Azure will compete against similar offerings such as Google App Engine and Force.com.”

Oh...and we made techmeme :-)

More...

Phil Wainewright - Why Microsoft and Intuit need each other's clouds

“Cloud platforms share many of the ingredients of conventional software platforms, but they add several crucial new ingredients. One thing that hasn’t changed is the need to build momentum among developers and customers for the platform. Intuit and Microsoft have plenty of both, which guarantees attention for what they’ve announced today. But the tie-up between these two giants is important too for the light it shines on the special characteristics of cloud platforms and how they change the game in so many ways for ISVs, developers and platform vendors…

...the link-up combines Microsoft’s strengths in developer tools and functional scope with Intuit’s advanced skills and investment in service delivery on IPP”

Joining Intuit

alexbarnett — Sun, 21 Sep 2008 22:02:00 GMT

Firstly - thanks to everyone who has reached out to me in the last three weeks via email, phone calls and comments since I shared the news of my pursuit for the next adventure - I have really appreciated everyone's support and interest in my next steps.

The great news is I'll be joining Intuit as a Group Manager working in a fast growing start-up team responsible for leading the Intuit Partner Platform. Below is a snippet and some links to blog posts and articles that should give you a fairly good idea about where my focus will be a week from now.

Although I'll be working from the Orem office (Utah) to start off with, the plan is to ultimately move to the greater Boston area - another new adventure. I've visited Boston three times in the last year or so and have loved it more and more with each visit, so watch out Boston!

This is a fantastic opportunity for me personally - the team has ambitious goals and an amazing set of existing assets to leverage (see some of the numbers below) in becoming a significant player in the cloud computing space. I look forward to sharing stories of my new journey with you.

Intuit Partner Platform Opens to Developers (Sept 15 2008)

"The Intuit Partner Platform not only gives developers the opportunity to build Web-based applications, but successful SaaS businesses by taking the complexity out of managing infrastructure, hosting, user management, integration and billing. Now developers can focus on developing innovative on-demand software solutions that solve unique and important problems for the four million small and mid-market businesses across the U.S. that use QuickBooks and the 25 million employees that work in those companies.

The platform-as-a-service offering allows developers to combine the powerful open source Flex framework and Adobe Flex Builder and the proven database of Intuit QuickBase to build rich Internet applications that work with QuickBooks data. They can also leverage Adobe AIR to provide additional desktop-like functionality in their applications, such as pop-up notifications, local file system access, local data storage, and the ability to create a fully branded user experience.

"We have now accepted more than 1,000 developers into the program and it is exciting to hear their ideas and energy about what they want to build," said Alex Chriss, business leader for the Intuit Partner Platform. "Customers will benefit greatly from the imagination and expertise that developers use to solve problems facing their specific industries." "

Web 2.0 Expo NYC - Intuit Connected Services (Sept 19 2008)
Intuit Makes SaaS Play Guns For Salesforce (Sept 18 2008)
Intuit’s flexy PaaS (Aug 14 2008)
Intuit enters the PaaS wars (April 28 2008)
Intuit’s Radical New Flex + QuickBase Cloud Platform (April 16 2008)

How ADO.NET Data Services came to be (formerly known as Project Astoria)

alexbarnett — Thu, 21 Aug 2008 02:16:00 GMT

Pablo Castro has recounted some of his timelined memories about how "Project Astoria" evolved from a lunch time conversation to bits in .NET 3.5 SP1 and Visual Studio 2008 SP1 now known as ADO.NET Data Services Framework). Nice write up.

Three memories of my own to add to the story:

1. I was reading up on the whole REST thing in the summer of 2006 - its origins, philosophy and design patterns. I knew there was something interesting going on and some potential dots to join, but I wasn't sure which dots...So I collated and circulated a bunch of research / links to the team, then blogged the links (I liked How I explained REST to my wife. More recently see Explaining REST to Damien Katz). I got a few proverbial (and some literal) blank stares as I shared my enthusiasm for REST, asking how we could apply the ideas to the various projects we were working on. It was Pablo, and (as Pablo attests) Britt Johnston (now a PUM for SQL Business) who were able to develop the initial conceptual leaps into something more concrete like a Think Week Paper and a prototype demo.

2. When it came to brainstorming the code name, the team agreed on a "cloud" theme. A number of proposals were floated around along with their rationales, including "cumulus" and "cirrus". We were then advised that city and town code names were legal-safe. So there we were, struggling to agree on some city or town name we all liked (or at least not hate nor be confused by..."how about Nameless?"...), and then Mike Pizzo's proposal came in: "Astoria - hey, it's the cloudiest city in the USA!" (at least it was in 2006). Sold.

3. I think my favorite memory of all is the reaction Gary Flake provided (of Microsoft's Live Labs) to the prototype Pablo demo'd at one of the pitch meetings: "As God himself would have designed it!" Dr Flake exclaimed..."Cool", I thought to myself - "but does that mean no REST for the wicked?"

Designing Web APIs - Twitter Learnings

alexbarnett — Thu, 01 May 2008 11:08:00 GMT

Although I made it to Web 2.0 Expo in San Francisco last week, I didn't make it to a session Matt McAlister blogged about by Twitter’s Alex Payne and Michael Migurski of Stamen Design who presented learnings from the perspective of an API provider.

But I can see the slide deck discussing the Twitter API and so can you:

| View | Upload your own

More Web 2.0 session slides available here. Recommended:

Videos of sessions here. Check out Clay Shirky's session, author of Here Comes Everybody: The Power of Organizing Without Organizations (a good read btw).

Astoria at MIX08 (REST in Vegas)

alexbarnett — Wed, 05 Mar 2008 02:11:00 GMT

As much as I'd love to make it to MIX08 this week, time will not allow me...But if I were, then I'd be going to the following three sessions related to the Project Formerly Known as Astoria:

Wed, March 5th - RESTful Data Services with the ADO.NET Data Services Framework
Fri, March 7th - Accessing Windows Live Services via AtomPub
Fri, March 7th - Building RESTful Real World Applications with the ADO.NET Data Services Framework

What I'm reading...

alexbarnett — Mon, 21 Jan 2008 17:06:00 GMT

There are a whole bunch of interesting posts / stuff I find on the net that I bookmark on del.icio.us (at least, I think they are interesting). Over the years I've been experimenting with different ways of sharing these with you. My most recent solution has been to include my del.icio.us links within my feed as seperate items. The problem with this approach is I haven't had a permalinked way of publishing these to my blog with a way to easily edit prior to publishing...also, having the daily summaries del.icio.us format in a feed is lame.

So I going to try something new. Instead of having the daily del.icio.us link summaries published as RSS items within my Feedburner feed, I'm going to publish these as blog posts. It should make things more economical from the consumption point of view (I don't think the "Links for 2008-01-20 [del.icio.us]" blah blah feed item titles are pretty). To do this, I have:

disabled the del.icio.us feed syndication from Feedburner (the Link Splicer)
have installed Josh Leggard's Insert Feed Content plugin for Windows Live Writer - this lets me populate a draft blog post with the latest items from any feed (my del.icio.us feed in this case - I'll still use the service for bookmarking) that I can then include / edit / add more commentary before I post to my blog - along with a custom post title.

Better me thinks. I like Harry Pierson's and Assaf's Labnotes style of providing links with commentary...over time I hope to emulate these.

So here goes - this first effort will be larger than future posts like this...shorter in the future.

OpenID - Getting Traction

Telegraph to become OpenID provider
"The Telegraph will soon become the first newspaper in the world, and the first British media company, to become an OpenID provider."
Yahoo! Announces Support for OpenID; Users Able to Access Multiple Internet Sites with Their Yahoo! ID
Sweet!
Stories we want to see in 2008
"Google, Microsoft, Yahoo, AOL, Endorse Support OpenID and OAuth"

Cool UI / Vizualization and Useful Bits

external hard disk treemap
"an external hard disk that shows the content of the hard disk on its outside skin."
Fascinating new way of entering text
"Dasher is a really fascinating interface that allows you to write by browsing through letters using a finger, mouse or some other pointing devise."
How Does FeedDemon Calculate Attention?
Nick Bradbury: "FeedDemon's algorithm for determining a feed's attention rank has changed since I first wrote about it, but it's still very simple."
using delicious on your iphone
Perfect. All I need is an iPhone now.
wmIRC.com - IRC client for Windows Mobile Smartphone and Pocket PC / Phone Edition
"IRC for when you're on the move."

SaaS Stuff

Scalable Media Hosting with Amazon S3
Amazon S3 101.
How to package up the SaaS platform
Phil Wainwright: "Sun’s intervention gives MySQL’s open source database an aura of greater enterprise readiness than it previously had, backed up by fully accountable support offered on a traditional commercial basis."
Updates to URI Syntax in Dec 2007 ADO.NET Data Services CTP
Astoria gets URI syntax updates.
The Idea of Software as a Service Platform
I still don’t see desktop GIS being replaced by web services anytime soon.
600 Web APIs
Programmable Web's 600 web APIs.
Enter the Internet Cloud
"The Internet cloud [is] where the distributed and programmable network of services across the globe will serve all the data, resources and functionality we will ever use." Good quote ;-)
Your Data in the Cloud - URL-based computing, SimpleDB, Astoria, etc.
Michael Cote: "the question for Astoria, SimpleDB, and all these “the non-relational database” databases isn’t so much a question of a good idea or not, but the way the technology is packaged and delivered."
12 predictions for Enterprise Web 2.0 in 2008
Dion Hinchcliffe on the worlds of SOA, SaaS, and Web 2.
Eight reasons SaaS will surge in 2008
Phil Wainwright: "The coming year is going to be a pivotal one for anyone involved in software-as-a-service."
The Big Switch: Rewiring the World, from Edison to Google
New book by Nicholas Carr, author of Does IT Matter? "A hundred years ago, companies stopped producing their own power with steam engines and generators and plugged into the newly built electric grid." Next - everything software.
Is Red Hat's New Development Environment Destined for an Amazon or IBM Cloud?
Dana Gardner: "Tools in the clouds."

Developer Cults and Dataheads

Sriram Krishnan: Amazon SimpleDB - Technical Overview
"I love the data model for SimpleDB."
The Cults of Programming
"In my experience with various programmers over the years, I've realized that most of them fall into one of several cults which describe their behavior." Via Assaf.
Scaling Twitter: Making Twitter 10000 Percent Faster
How many of the 15 Twitter employees are dedicated to managing all this?
Eventual Consistency Is Not That Scary
"Get ahead of the curve and understand for your application what the consistency requirements will be."
Eventually Consistent
"Recently there has been a lot of discussion about the concept of eventual consistency in the context of data replication."
Make Money Fast - Introducing Amazon DevPay
"With DevPay, developers can focus on being creative and innovative while dispatching the less-than-glamorous aspects of dealing with bank accounts, credit cards, and so forth to us."
Facebook Right, Scoble Wrong: Social Network Interoperability and the O'Reilly Social Graph FOO Camp
Dare Obasanjo: "The data portability folks want to make it easy for you to jump from service to service."
ADO.NET Data Services (Astoria) Transforms SQL Server into an Atom Store
Wow.
Rails 2.0: It's done!
Er, Rails 2.
Volta Fundamentals
Erik Meijer's latest. In essence Volta is a recompiler.

Uncategorizable But Good.

Lessons from Star Wars
Johnnie Moor's pointer: "Stephen Anderson shares his presentation about what designers can learn from the making of Star Wars."

8 Trends in Software as a Service Platforms

alexbarnett — Sun, 13 Jan 2008 17:40:00 GMT

To kick off the new year, I presented to around 40 or 50 members of Utah Technology Council (UTC) last week. The title of the topic they asked me to speak about was "Trends in Software as a Service Platforms". I searched around for some ideas and came across two recent posts predicting trends in SaaS for 2008, one by Phil Wainewright "Eight Reasons SaaS Will Surge in 2008" and Jeff Kaplan's post "Top Ten Reasons Why On-Demand Services in 2008". I decided to borrow liberally from these (thanks Phil and Jeff) and mash these two together (along with a couple of thoughts of my own) and present "8 Trends in Software as a Service Platforms" to an audience made up of CTOs and VPs of engineering and development for software companies in the Utah area.

In preparation for the presentation, my boss (Martin Plaehn) at Bungee Labs suggested I write up my presentation as notes blog them afterward, so here they are.

8 Trends in Software as a Service Platforms

SaaS is just part of the web mega-trend
Mainstream opinion says “Yes” to SaaS
Software vendors stampede into SaaS
All is being virtualized
Explosion of Web APIs
Economic factors favor SaaS
Enterprise and SMB IT embraces SaaS
SaaS platforms proliferate (PaaS)

1. SaaS is just part of the web mega-trend

Most of us have witnessed and many of us have been a part of the transformation in the way goods and services have been digitized, virtualized, delivered and consumed. Software, the data behind that software and the functionality that software provides is no different - software is subject to the very same transformational forces.

Just think about how even a class of product that is natively digital - such as software - has been transformed in the way it is delivered and consumed. For prosperity's sake, I've still got a few of those ZX81 software cassettes stashed away somewhere, gathering dust, looking ever more antiquated with each passing year. How will today's mode of software delivery and use look to us in a few years from now?

The web wants to connect things, and that's interesting. But connecting and interacting with "live" data, information and remote functionality make things more interesting.

At the fundamental level, the web connects things. It connects people to people, businesses to businesses, and people to businesses. Since the early 90's, the web has enabled the connection of so many things to so many other things at an ever accelerating rate, and yet we crave even more connectivity. But we increasingly also want the ability to interact with those things.

And it is the nature of these connected things that have changed since the early internet. The early web was good at connecting to static views of information and accessing limited and rigid functional services, very much a read-only mode. Then, as we learned a) the ability to read more dynamic-type information - at least regularly updated, and b) access richer remote functionality, we created whole new opportunities for ourselves. Next, we grew our ability read and write against dynamic, near real-time data and information and to program against remote functionality to create a new class of web applications leveraging those capabilities - and hence a new order of business and experiential opportunities have emerged. Some label this as "Web 2.0".

At its essence, it is the "liveness" of these real-time read-write data, information and functional sources available as "always on" services and the increasing ease to connect to, interact with - specifically change those resources available as live, programmable services that allows us to create new value out of those resources, opening up brand new market opportunities for businesses and the compelling, rich "live" end-user experiences of tomorrow.

2. Mainstream opinion says “Yes” to SaaS

Not surprisingly, Wall Street loves the the predictability of subscription services. It's good for cash flow, forecasting and business planning.

The venture firms also relish the opportunities that are opening up in a software as services-oriented economy. The ability to circumnavigate the incumbent software players with new disruptive technologies and propositions that are significantly easier to try and access for prospective customers compared to traditional software evaluation, along with usage and subscription-based business models verses the old licensing model makes investing in services-based software companies very compelling propositions from the venture firms' point of view. We should also see healthy M&A activity based on these similar opportunities in the coming year.

And then there's the trend for offshore / IT business process outsourcing. These providers will surely get in the game and make their plays through investments in and acquisitions of SaaS vendors that align well with their current core businesses.

Add to that the excitement we're reading about the SaaS space from the IT Analysts, journalists and bloggers, plus the new book by Nick Carr (author of “IT Doesn’t Matter”) - delivered by Amazon to me last week: “The Big Switch: Rewiring the World, from Edison to Google”. I think there's little doubt Carr's excellent analysis of the computing industry as an analogy to the electricity industry's shift to a utility model will be on business bestseller list for much of 2008. His messages resonates with corporate executives and end-users agree with him:

IT is a needless hassle,
it should be as easy as electricity and
be as reliable as a utility

3. Software vendors stampede into SaaS

The Big Software Players are following the early SaaS successes

CRM as a case in point. If you've been following the CRM software market, you'll know about the noises Oracle-Siebel, SAP and Microsoft started to make in the 2007 about what they are are lining up for the 2008 in terms of CRM as a service. Their efforts to emulate Salesforce.com's success delivering CRM as SaaS will be key strategic bets from the incumbents' point of view - and loud, price and functionally competitive propositions from the point of view of their existing and prospective customers.

CRM is just one of the multiple horizontal solution categories to transform from on-premise with traditional licensing model to a service-based delivery and subscription-based revenue model. ERP, supply chain, e-commerce, HR and many more...the horizontal solution list goes on. And then there are the vertical solution players...

Here's another data point to consider regarding the move by traditional software vendors to a SaaS model:

“15-20% of application ISVs have already either begun new skunk works initiatives or gained access to SaaS assets and development experience through M&A activity”
(Source: Key Trends in SaaS: 2008 and Beyond, Saugatuck Technology)

4. All is being virtualized

Virtualization is a technology trend.

Virtualization enables hardware as a service. The demand for virtual machines met by hypervisor software (VMWare, Xen, Hyper-V) and the success of Amazon's Elastic Compute Cloud (EC2) in the last couple of years point to a continuation of further virtualization of applications and hardware. Virtualization is accelerating the move from traditional on-premise software to services.

Virtualization is a business trend.

We continue to become a mobile workforce. The younger entrants into the workforce in service-oriented economies expect and want to be always connected. It's very hard work, if not impossible to get your traditional on-premise applications and centralized servers sitting behind a firewall to serve today's mobile workers. SaaS and managed services meet the needs square on.

5. The explosion of Web APIs is upon us

According to ProgrammableWeb.com, there are 559 commercial and public APIs available today, most of these are new and there are plenty more to come. How many will we see go live this year? And how many private web APIs are there and will be developed and consumed in the coming year?

Data from ProgrammableWeb.com

6. Economic factors favor SaaS

On-premise software requires upfront capital investments
To lower costs, many companies hold back on their capital investments to mitigate their risks, especially in recessions
Adopting on-demand services on a pay-as-you-go basis will be a perfect sourcing strategy for businesses seeking greater cost-controls and flexibility – the utility model

All well and good, but the real economic value of SaaS is that fact that it unleashes new value of previously isolated data silos and functionality.

7. Enterprise and SMB embraces SaaS

When it comes to IT, who doesn't like

Low-maintenance?
Low cost?
Low-resource profile?

IT and business folk like these things, and externally delivered SaaS applications deliver these benefits.

8. SaaS platforms proliferate (PaaS)

The more mainstream SaaS becomes the more the large vendors will be forced to offer effective platforms for ISVs, enterprises and SMBs.

If the move by the software vendors from traditional on-premise software to a services model is to be successful, they will need to provide programmable interfaces - not just end-user interfaces - to their services for their customers. Customers need and want the ability to access, intergrate and create new value out of live, programmable data, information and functionality living in the cloud. And in turn these same customers will want their custom-developed composite applications and integrated data available as programmable services - yet more APIs.

Customers want to unleash new value of previously isolated data silos and functionality through the development of their own applications programmed against those resources. And in turn these same customers will want their own custom-developed composite applications and newly integrated data available as end-user interfaces and as programmable services - yet more APIs. These customer needs will drive the software market to provide platforms to provide businesses and developers with with end-to-end:

programmable services and data integration
application development, testing and collaboration tools
deployment and scalable delivery

...all as a service with a utility model.

(hey...I needed to mentioned Bungee Connect just the once ;-).

2008 will mark a the proliferation of such offerings as "platforms as services" (or PaaS) through 2009, where then the consolidation will begin. Interesting SaaS and PaaS times ahead.

Update 2/20/2008: see "Time to Define "Platform as as Service" (PaaS)

The presentation seemed to go down pretty well and we had lots of interesting discussion throughtout. One of the topics we discussed was data security in a SaaS world. Don Kleinschnitz (VP, Development at Symantec) followed up with a mail linking to his blog covering Security 2.0 topics.

Again - thanks to Phil Wainewright and Jeff Kaplan for their post and to Martin for suggesting I blog this.

SQL Server 2005 Driver for PHP

alexbarnett — Tue, 09 Oct 2007 21:57:00 GMT

Who woulda thunk it? The CTP for the SQL Server 2005 Driver for PHP will be available October 11:

"The SQL Server Driver for PHP is designed to enable reliable, scalable integration with SQL Server for PHP applications deployed on the Windows platform. The Driver for PHP is a PHP 5 extension that allows the reading and writing of SQL Server data from within PHP scripts. It provides a procedural interface for accessing data in all Editions of SQL Server 2005 and SQL Server 2000 (including Express Edition), and makes use of PHP features, including PHP streams to read and write large objects."

CNET reports...

"Microsoft revealed some fruits of a partnership that that was announced a year ago with Zend, which develops and commercializes the open-source PHP scripting language for creating dynamic Web pages. Bill Staples, a Microsoft product unit manager, announced four moves at the ZendCon conference here:

• Microsoft is releasing a preview version of a software connector that lets PHP run atop the SQL Server 2005 database. "This is a Microsoft-developed and supported PHP driver for accessing SQL Server data from within a PHP application," Staples said."

Ah...stat time now...(my bold):

"About 70 percent of PHP developers use Windows, said Andi Gutmans, who along with Zeev Suraski are Zend's co-founders and co-CTOs. But when it comes to deploying the applications for use, customers use Linux in about 80 percent to 90 percent of cases, Suraski said."

Playing with Zoho DB

alexbarnett — Wed, 03 Oct 2007 19:30:00 GMT

Via Read/WriteWeb, I heard of Zoho DB this morning. Am playing around and I like it.

I've uploaded my LibraryThing book catalog into a new Zoho DB and can publish one of the views I've defined as an embedded view into this blog post, providing some basic interaction - see below.

Within Zoho, I can create views based on SQL Queries over the data - there is support for Oracle, SQL Server, DB2, Sybase, MySQL, PostgreSQL, Informix and ANSI SQL dialects. Very nice. However, what I'm trying to figure out is the web API story for Zoho DB....I can't seem to find the documentation for it, but will keep looking.

Update: Raju Vegesna, evangelist at Zoho, got in touch to let me know there are no APIs today, but the team is planning to provide these.

ADO.NET Entity Framework video- 1 year later

alexbarnett — Thu, 30 Aug 2007 14:04:00 GMT

Some old Data Programmability teammates of mine, Britt Johnston, Pablo Castro and Mike Pizzo (well, Mike's not that old), are interviewed by Charles Torre in this Channel 9 video to discuss their progress, challenges and customer feedback in the last year since announcing the first ADO.NET Entity Framework CTP, including some details on project Astoria and LINQ to Entities support, details on the Beta 2 and Tools CTP released earlier this week (the entity model designer is v.cool), plus the future of the Entity Framework.

P.S. Michael - in case you're reading: reconsider the shorts for the next interview :-P

Geo-crime mashups

alexbarnett — Sat, 25 Aug 2007 12:36:00 GMT

I think I've hit a personal first: a depressing mashup. Oakland Crimestopping is a Flash-based vizualization tool overlaying reported crime data in Oakland, CA, by type (from aggrevated assault to murder to burglary) and time on a Virtual Earth map.

Fortunately, I live nowhere near Oakland, but if I did I think I'd try to live on a pontoon on Lake Merritt. This is what July 27 to August 24 2007 looks like:

To fuel an ongoing level of anxiety, you can subscribe to email alerts or to customized RSS feeds based on specific queries. Example - track all the fun and games going on within 1/2 mile of High St.

This post by the one of the site's developers, Michal Migurski, mentions future areas to cover might include San Francisco and Berkeley. Check out the juicy implementation details.

Two more crime maps tools you can get depressed about are Portland's CrimeMapper (e.g. a generated report on crimes on Richmond, Portland in the last 12 months) and LA Times' Homicide map (536 murders in 2007 and counting).

(via information aesthetics)

Two thoughts on the Facebook Data Store API

alexbarnett — Fri, 17 Aug 2007 17:41:00 GMT

Two thoughts on the Facebook Data Store API:

1. One

2. Two

Interview with Peter Spiro - Building Great Databases

alexbarnett — Sat, 11 Aug 2007 15:37:00 GMT

This is great video from Channel 9 - an interview with Peter Spiro, a Technical Fellow and Distinguished Engineer at Microsoft. It's probably an understatement to describe Peter's career path as "unconventional", a path that lead to him eventually to accept a role at Microsoft in 1994 within the database team as an architect, going on to manage the dev team and building SQL Server's technical team to where it is today. This is after studying Forestry and stint as a charcoal maker in Mali, West Africa.

In the interview, Peter discusses lessons learnt through his various career experiences, including the people / management skills acquired, the technical challenges involved in solving complex software problems, thoughts on leadership, mentoring & coaching and his approach to recruiting and growing technical talent. He also gets to draw his favorite data structure in the interview.

The question asked by by Bill Hill toward the end of the interview was unusual "What did nature teach you about software?"

And yes, the beard is real and according to Sam Druker is probably around 15 years old. Btw, Sam (my old boss during my stint with Data Programmability team) also appears in the video describing how Peter <didn't> recruit him into the team.

Five out of five for this one.

Astoria: Data Services for the Web - Part 2

alexbarnett — Sat, 04 Aug 2007 02:17:00 GMT

Back in April, the Data Programmability team at Microsoft announced "Astoria": Data Services for the Web, an incubation project exploring programming models for data over the web. As part of the announcement, the team deployed an online implementation of Astoria here, providing some sample read-only datasets for you to play with (see this post for examples). They also mentioned that plans were underway to allow web developers to define their own data model and have their own data stored as read-write relational data service "in the cloud", programmable over a RESTful API.

What's New

Today, Mike Flasko, Astoria Program Manager announced the delivery of this second piece of Astoria online service, where

"web developers can create custom structured data stores (up to 100MB in size) on the web and access them from anywhere that they have Internet access. These data services can then be the storage or data source for mashups, or the backing store for Internet enabled applications, or be applied to any other scenario where a rich data service on the web is required, independently of where it is hosted. "

(Astoria Program Manager? This is incubation project is going somewhere... :-)

You create your own data model for the Astoria online service by web-based tool that lets you define your data model (entities, properties and associations). Be warned - it's a little rough at the edges, but good enough. Once you've you've defined your data model the service generates the data store for you and hands you an URL...Yes! Your very own read/write REST interface to your own data store.

So now I've got my own relational data service going (note: will you require a username / password, which I ain't going to give you :-) at https://astoria.sandbox.live.com/users/alexrest/alexrest.rse

And all this without requiring the installation of Microsoft bits - no Visual Studio, no SQL Server, nada!

I'll be experimenting a little more with the new service and hope to share some interesting possibilities in the future, but I think it is fair to say that this incubation project has some seriously interesting and broad potential. Congrats to the team for pulling this off!

More at:

Astoria team blog
Pablo Castro's blog
Elisa Johnson's blog
(8/6/07) Mary Jo Foley - Microsoft Code Name of the Day: Astoria
(8/8/07) Jamie Thomson has been playing and got his own service running

Previous Astoria posts: