Alex Barnett blog : OpenSource, RIA

Why OData Matters (IMHO)

alexbarnett — Fri, 19 Mar 2010 15:24:00 GMT

Earlier this week I was in the MIX10 crowd as Douglas Purdy announced the Open Data Protocol (it was a great presentation - summarized here).

I want to share with you why I think OData could be a very big deal…But before we go there...let's start with the basics...

What is OData? Where Did OData Come From?

To understand the history of how OData came to be, you need to understand how project "Astoria" came to be...I won't go over that again as this is already pretty well documented. ~~Astoria~~ OData has come a long way since.

In a nutshell: Today, OData builds on a few conventions, popularized by AtomPub (see OData AtomPub Format), to using REST-based data services. These services allow resources, identified using Uniform Resource Identifiers (URIs) and defined in an abstract data model (see OData URI Conventions, to be read and edited by web clients using simple HTTP messages (see OData Operations).

An Open Data Protocol for the Web

OData offers a standardized way for programmable data to be made available across the web and in turn allowing "consumers" of that data to rely on a set of conventions to be followed that in turn allows many interesting things to happen if widely adopted...

...And to this end:

As announced, OData has been released by Microsoft under the Open Specification Promise (OSP) "to allow anyone to freely interoperate with OData implementations" . Since then, the W3C has invited the OData team to Bring OData to a W3C Incubator (I haven't seen a public response yet but I urge the team to do so.).
OData is not a Microsoft-only thing and it won’t succeed if it is. The originating philosophy was about transparency in the design process, with an Open end-point as the goal - not a .NET lock-in play (“agree on standards and compete on implementation”). This approach has already yielded an initial set of clients, servers, services, and tools. Today, a number of OData SDKs and libraries are available for .NET, Java, PHP, iPhone (Objective-C) and more – and there’ll be more coming.
There is a small but growing ecosystem of non-Microsoft "producers" and "consumers" - (where OData "producers" include Netflix’s catalog as OData and the VanGuide (a social map of Vancouver Open Data) and Public data from the city of Edmonton as OData) and OData “consumers” - standard web browsers, RIA "data explorers" – such as Sesame OData Browser, and the client libraries mentioned above – p.s. somebody build a javascript-only data browser please!)

Where can OData take us?

The clue is in the OData icon (next to the RSS feed icon. Can you see the similarities?

The big idea here is that in the same way we have the "RSS" feed icon, we'll get used to seeing the "OData" icon on commercial and non-commercial websites everywhere (especially for government-related data). So in the same way you know today that the RSS icon means "get an XML feed for this content", the "OData" icon means "get this web data" - you'll know (and your client will know) what to expect in terms of how to read in, and navigate through and query structured web data sets - and in many cases write against them - using a common syntax.

Q: Right, But So What?

A1: Open Government OData. From Open Government Data Principles

The Internet is the public space of the modern world, and through it governments now have the opportunity to better understand the needs of their citizens and citizens may participate more fully in their government. Information becomes more valuable as it is shared, less valuable as it is hoarded. Open data promotes increased civil discourse, improved public welfare, and a more efficient use of public resources.

That’s great, but it needs to be practicable…And number 5 of the 8 Principles of Open Government Data sensibly states that the data should be (via David Eaves):

5. Machine processable

Data is reasonably structured to allow automated processing.

It'll be down to each government agency (and local government) as to how they decide to implement this principle, but wouldn't it be great if they agreed to a standard (and a powerfully simple, web-oriented one at that)? This is what Jon Udell concluded here:

"The open data movement, in government and elsewhere, aims to help people engage with and participate in processes represented by the data. When you publish data in a fully articulated way, you build a framework for engagement, a trellis for participation. This is a huge opportunity, and it’s what most excites me about OData"

A2: To ODatarize your data is to RESTify your data.

As more data-oriented web APIs come online, each team responsible for the design of each web API is confronted by the same kinds of questions, and each team answers these in their own particular way. Increasingly, “RESTful” is a design goal of web APIs. Great…but what does that mean? How do you expose the data, the relationships between the entities inside the model, and what should the querying syntax look like? Unfortunately, there are as many answers to these questions as there are RESTful web APIs. And there needn’t be.

For to ODatarize your data is to RESTify your data. Do spend the time at the value layer - figure out the way your developers / consumers want to see the data and expose it that way. Do make it easy for devs / consumers to learn / navigate about the data and use it. Do not make them learn about the unique idiosyncrasies you’ve built into your API (or those that leak out of your originating store) :-)

From a developer’s standpoint, OData is ultimately about productivity.

For the OData “Production Developer”: Point at your data store – define your entity model and map it to the data model you already have (so your developers consume / program against the data that makes most sense to them – effectively ORM’ing) and expose as an OData service, inheriting: all the REST characteristics; entity relationship self-discovery; and querying goodness.
For the OData “Consuming Developer”: If you know the web API is OData…great! Pick up a client library, get to the API end-point (data.foo.org/blah.svc). Point and Shoot: Traverse the data model, query it (and bookmark as needed – it’s a URI)…play!

(see links at the bottom of this post to technical content that provides details on all this)

A3: Since the announcement, I’ve seen quite a bit of excitement around the web (especially Twitter) by developers who see the potential here…there is plenty of experimentation going on. At Intuit, my team is also experimenting with ODatarizing some of our data services, exploring how it might be applied across a number of our cloud based data services. And when our team’s architect Tweets that “Looks like #odata is going to be a good fit for our data services”, I know there’s something interesting going on here…

So I encourage you to find out more about OData and get involved.

More Resources

An easy way to set up an OData feed from your SQL Azure database - Stephen Forte
OData interoperability with .NET, Java, PHP, iPhone and more - MSFT Interop team
Microsoft, Bring OData to a W3C Incubator - W3C blog -
Silverlight 4 + RIA Services - Ready for Business: Exposing OData Services-4-ria-services-ready-for-business-exposing-odata-services.aspx - Brad Abrams (MSFT)
Open Data for the Open Web - Douglas Purdy (MSFT)
Producing and consuming OData feeds: An end-to-end example - Jon Udell (MSFT)
OData for collaborative sense-making - Jon Udell
The three laws of open government data – David Eaves
8 Open Government Data Principles
Open Data Protocol (OData) with Pablo Castro - Hanselminutes podcast interview with Pablo Castro (MSFT – Architect for OData)
OData, a RESTful contender for your API - Andy Thompson
Announcing Sesame Data Browser - Fabrice Marguerie
Retrieving Data with the OData SDK for PHP - Brian Swann
Got SQL Azure? Then you've got OData - OData blog
Implementing only certain aspects of OData-only-certain-aspects-of-odata.aspx - Pablo Castro
Every SharePoint 2010 server is a Data Services server - Pablo Castro
Microsoft Creates New OData.org Website for Open Data Protocol – CMS Wire
Microsoft delivers updates on OData, Houston, Dallas - ZDNet
Timeline of Project Astoria - Pablo Castro
How ADO.NET Data Services came to be (formerly known as Project Astoria) - Alex Barnett
previous Astoria / OData posts - Alex Barnett

Joining Intuit

alexbarnett — Sun, 21 Sep 2008 22:02:00 GMT

Firstly - thanks to everyone who has reached out to me in the last three weeks via email, phone calls and comments since I shared the news of my pursuit for the next adventure - I have really appreciated everyone's support and interest in my next steps.

The great news is I'll be joining Intuit as a Group Manager working in a fast growing start-up team responsible for leading the Intuit Partner Platform. Below is a snippet and some links to blog posts and articles that should give you a fairly good idea about where my focus will be a week from now.

Although I'll be working from the Orem office (Utah) to start off with, the plan is to ultimately move to the greater Boston area - another new adventure. I've visited Boston three times in the last year or so and have loved it more and more with each visit, so watch out Boston!

This is a fantastic opportunity for me personally - the team has ambitious goals and an amazing set of existing assets to leverage (see some of the numbers below) in becoming a significant player in the cloud computing space. I look forward to sharing stories of my new journey with you.

Intuit Partner Platform Opens to Developers (Sept 15 2008)

"The Intuit Partner Platform not only gives developers the opportunity to build Web-based applications, but successful SaaS businesses by taking the complexity out of managing infrastructure, hosting, user management, integration and billing. Now developers can focus on developing innovative on-demand software solutions that solve unique and important problems for the four million small and mid-market businesses across the U.S. that use QuickBooks and the 25 million employees that work in those companies.

The platform-as-a-service offering allows developers to combine the powerful open source Flex framework and Adobe Flex Builder and the proven database of Intuit QuickBase to build rich Internet applications that work with QuickBooks data. They can also leverage Adobe AIR to provide additional desktop-like functionality in their applications, such as pop-up notifications, local file system access, local data storage, and the ability to create a fully branded user experience.

"We have now accepted more than 1,000 developers into the program and it is exciting to hear their ideas and energy about what they want to build," said Alex Chriss, business leader for the Intuit Partner Platform. "Customers will benefit greatly from the imagination and expertise that developers use to solve problems facing their specific industries." "

Web 2.0 Expo NYC - Intuit Connected Services (Sept 19 2008)
Intuit Makes SaaS Play Guns For Salesforce (Sept 18 2008)
Intuit’s flexy PaaS (Aug 14 2008)
Intuit enters the PaaS wars (April 28 2008)
Intuit’s Radical New Flex + QuickBase Cloud Platform (April 16 2008)

Podcast interviews - smart people in the world of the web

alexbarnett — Sun, 20 Jan 2008 15:41:00 GMT

One of the fun parts of my job at Bungee Labs is to partner up with Ted and interview some smart people in the world of the web. We publish these as a podcast series (the Bungee Line - podcast feed here) over on the BCDN blog.

If you have ideas about someone you think we should interview, let me know! We're focusing on topics we think web developers might be interested in the worlds of software as a service and web app development, in particular profiling web apis. Related topics are good too.

I've listed out below our most recent podcasts below...plenty more in the works (previous podcasts are listed here). Hope you like :-)

Alan Lewis on eBay Desktop and eBay APIs

"As product manager for eBay Desktop, Alan Lewis relies on the same web APIs that eBay makes available to all developers. In this edition of the Bungee Line, Alan tells us about what the eBay Desktop is, how it came about, and various details about eBay’s developer program and web APIs. We ask Alan about eBay’s position Oauth and on open source."

Toby Segaran on “Programming Collective Intelligence”

"Since the publication of his O’Reilly book Programming Collective Intelligence: Building Smart Web 2.0 Applications, Toby Segaran has become well noted for his ability to explain easily-understandable algorithms for the kind of deeply complex problems involved in social applications. Toby joins Alex and Ted to discuss some of the high-level concepts that he tackles in his book."

Jon Aizen of Dapper.net

"Jon Aizen joins Alex and Ted to explain how Dapper.net provides a no-fee tool for making almost any structured web site data accessible via a REST API. In a past life, Jon was involved in creating The Internet Archive. Jon also helps the Bungee Line introduce romantic intrigue into the podcast.

Punditry Alert! At the end of this show, Ted and Alex speculate a bit about Android, Google’s open source mobile device platform, the Apache License, and whether Robert Love is involved. Please consider this as another demonstration of Ted’s idiocy, brought to you by the Bungee Line."

Jeff Barr on Amazon Web Services (Part 2)

"In part 2 of our interview with Amazon Web Services evangelist Jeff Barr, Alex and Ted ask Jeff about Flexible Payment Service, virtual user group meetings in Second Life, the Startup Project, and pry at Jeff’s views of possible futures of technologies that developers might anticipate."

Jeff Barr on Amazon Web Services (Part 1)

"Developer evangelist for Amazon Web Services, Jeff Barr tells Alex and Ted about how he became a native Amazonian, his recent visit to “The Business of API’s Conference,” and a bunch of stuff on Amazon Web Services, including: Mechanical Turk, EC2, and S3. Additionally, Jeff explains the newly announced S3 Service Level Agreement*."

Alex Barnett Podcasts

admin — Sat, 23 Sep 2006 19:58:00 GMT

Alex Barnett Podcasts - I like podcasting, here are the links to them.

2008 - Podcasts for the Bungee Line

Nate Bowler, CTO of @Task, July 20 2008

@task (or AtTask) is a Utah-based tech company providing a comprehensive, web-based project and portfolio-management package delivered in both a SaaS and on-premise model with a very rich web API set. We talked with Nate about the evolution of their web services design and @task's future product plans in light of the market opportunities presented by the availability of the increasing number of 3rd party programmable web services."

Steve Bjorg, Founder and CTO of MindTouch, June 20 2008

"Prior to founding MindTouch and Steve worked in advanced strategies at Microsoft focusing on distributed systems and web services. We talked with Steve about MindTouch platform, its rich set of web APIs and the implications of a programmable wiki. But MindTouch goes beyond providing open source wiki collaboration and content management - it's delivering a leading edge application integration and development platform called MindTouch Deki. Michael Coté, an industry analyst with RedMonk (analyst firm) picked up on both the podcast interview and news of the latest release of MinTouch Deki."

Update from John Musser of ProgrammableWeb, April 14 2008

"ProgrammableWeb’s John Musser returns to the Bungee Line to give us an update on the API action of early 2008. Alex and Ted apologize for the unfortunate audio treatment to the Bungee sound in the previous episode, promising “never again!” In related news, check out the new intro music for our “Cool Web Tips” segment."

Developer Community Management with Jono Bacon, March 14 2008

"There are few developer communities as large and distributed as that of Ubuntu, perhaps the most popular brand of GNU/Linux distributions available today. Jono Bacon is the first official community manager for Ubuntu. He joins to tell us what he has learned in his 18 months of working with this vast and disparate community."

Social Design with Joshua Porter, Jan 30 2008

"Joshua Porter is a usability consultant, web designer, researcher and blogger specializing in the art of social design for the web whose experience includes five years at world-renowned User Interface Engineering. Josh’s blog (Bokardo.com) is a must-read favorite for UI and web designers and is finishing up his first book, to be published in the next few weeks (details below)."

Alan Lewis on eBay Desktop and eBay APIs, January 15 2008

"As product manager for eBay Desktop, Alan Lewis relies on the same web APIs that eBay makes available to all developers. In this edition of the Bungee Line, Alan tells us about what the eBay Desktop is, how it came about, and various details about eBay’s developer program and web APIs. We ask Alan about eBay’s position Oauth and on open source."

2007 - Podcasts for the Bungee Line

Toby Segaran on “Programming Collective Intelligence”, December 13 2007

"Since the publication of his O’Reilly book Programming Collective Intelligence: Building Smart Web 2.0 Applications, Toby Segaran has become well noted for his ability to explain easily-understandable algorithms for the kind of deeply complex problems involved in social applications. Toby joins Alex and Ted to discuss some of the high-level concepts that he tackles in his book."

Jon Aizen of Dapper.net, November 17 2007

"Jon Aizen joins Alex and Ted to explain how Dapper.net provides a no-fee tool for making almost any structured web site data accessible via a REST API. In a past life, Jon was involved in creating The Internet Archive. Jon also helps the Bungee Line introduce romantic intrigue into the podcast.

Punditry Alert! At the end of this show, Ted and Alex speculate a bit about Android, Google’s open source mobile device platform, the Apache License, and whether Robert Love is involved. Please consider this as another demonstration of Ted’s idiocy, brought to you by the Bungee Line."

Jeff Barr on Amazon Web Services (Part 2), October 7 2007

"In part 2 of our interview with Amazon Web Services evangelist Jeff Barr, Alex and Ted ask Jeff about Flexible Payment Service, virtual user group meetings in Second Life, the Startup Project, and pry at Jeff’s views of possible futures of technologies that developers might anticipate."

OAuth Podcast, with Chris Messina (aka FactoryJoe), Larry Halff (of Ma.gnolia) and Eran Hammer-Lahav, November 3 2007

"OAuth is a big idea, but is it a "solution looking for a problem to solve"? I don't think so. The problem for end users today is real, i.e. authorizing one service to access your data by another service for use by the first service, securely and with control. For developers wanting to develop apps and services that create value through the use of customer data stored on other services, there is no standardized means set of protocols to lean on. Instead, developers need to waste time learning a new way for their app to be authorized to do so for each service provider, having to jump through the various specific means and idiosyncrasies of each service."

Jeff Barr on Amazon Web Services (Part 1), October 18 2007

"Developer evangelist for Amazon Web Services, Jeff Barr tells Alex and Ted about how he became a native Amazonian, his recent visit to “The Business of API’s Conference,” and a bunch of stuff on Amazon Web Services, including: Mechanical Turk, EC2, and S3. Additionally, Jeff explains the newly announced S3 Service Level Agreement*."

Interview with Jeremy Zawodny of Yahoo! - Part 1 and Part 2, October 1 2007

Part 1

"Yahoo!'s Zimbra acquisition, the Yahoo! Mail Web Services APIs, Jeremy's take on the Business Week article discussing Yahoo! Openness, the fruits of Yahoo! Hack Days and the Internal Yahoo! Hack Days initiative, Yahoo! Geocoding API, Yahoo! User Interface (YUI) Library, Yahoo!'s AJAX API for Maps"

Part 2

"What Mash lets you do, Hadoop and Yahoo!'s formal involvement, the WebOS meme, something Jeremy feels strongly about :-) That was fun. Watch out for the discussion on "Meta-API Providers"... More APIs...From b2c APIs to b2b APIs, plus Pipes and democratizing the mashupshpere"

Interview with John Musser of ProgrammableWeb.com, September 19 2007

"Topics covered include Facebook APIs, Amazon's recently launched Flexible Payment Service (FPS) , Google Base, Microsoft's Astoria and relational-data-in-the-cloud programming models and services, SaaS models and API SLAs, REST vs SOAP, "Closed is Still the Old Closed" and plenty more."

-

Older podcasts:

Search & Enjoy! (Podcast) The Power of Search and Recommendation, June 6 2006

"Speakers from Microsoft, Blinkx and Last.fm discussed issues of content regarding search, recommendation, the semantic web and the ownership of data in the Web 2.0 era at Content 2.0 on 6th June 2006."

-

Microformats Podcast, March 31, 2006

"Here's a great podcast for you. All about microformats..."

Guests: Tantek Çelik, Dan Connolly and Rohit Khare. I think it's safe to say these guys know a thing or two about the web and microformats.

-

OPML Podcast, March 10, 2006

"It's all about the draft OPML 2.0 spec and a few other things thrown in such as structured blogging, OPML tools, namespaces and microformats."

Guests: Joshua Porter, Adam Green and John Tropea.

--

OPML and Reading Lists Podcast with Danny Ayers and Adam Green, Feb 12, 2006

"Last year Dave Winer started to push the idea of Reading Lists for RSS. More recently, the idea of Dynamic Reading Lists and Feed Grazing (or Grazing Lists / Glists) has been kicking around.

Its likely that Reading Lists support will become a common feature of Feed Readers / Aggregators."

Guests: Danny Ayers, Adam Green and Joshua Porter

--

Attention podcast : Attention with Steve Gillmor, Feb 08, 2006

"Steve has been leading Attention conversation for some time now. In 2003 he, along with David Sifry (CEO of Technorati), initiated the attention.xml efforts and has since taken on the role as president of the non-profit Attention Trust."

Guests: Steve Gillmor and Joshua Porter

--

MSN Search Champs podcast - Privacy conversation Jan 26 2006

"I attended the MSN Search Champs today....and what a day. Given the recent news and concerns around the data MSN Search, Yahoo and AOL provided to the government, there was a session set up where the 57 bloggers / online experts at MSN Search Champ were invited to discuss the topic with senior MSN management (Senior VP Yusuf Mehdi and VP Chris Payne)."

Guests: Fred Oliveira, Dion Hinchcliffe, Joshua Porter, Chris Pirillo, Thomas Vander Wal and Brady Forrest.

--

Attention podcast: RSS feedreaders and aggregators Jan 22, 2006

"I asked two of the RSS industry's leading lights to join me for a call and share their perspective on the question of where Attention is going with respect to RSS feedreaders and aggregators: Nick Bradbury creator FeedDemon, part of Newsgator (Nick also developed Homesite - sold to Macromedia - and Topstyle) and Kevin Burton of Tailrank (also co-founder Rojo)."

Guests: Nick Bradbury, Joshua Porter and Kevin Burton

--

Structured Blogging podcast with Marc Canter and Joe Reger, Dec 16, 2005

"You might have heard of the Structured Blogging initiative announced earlier this week by Marc Canter and others...there was certainly plenty of buzz and reaction to the news, but not all the reaction was rosy."

Guests: Marc Canter and Joshua Porter

--

Identity Podcast with Kim Cameron and Dick Hardt, Dec 09, 2005

"A couple of weeks ago Joshua and I had a conversation about attention data (as podcasts).

In that conversation we kept touching on the topic of online identities and their management, so we thought we'd invite two pioneers of the identity space, Dick Hardt and Kim Cameron, to a podcast session and discuss how they saw the connections between these two related topics: attention and identity."

Guests: Dick Hardt, Kim Cameron and Joshua Porter

--

OPML = Attention Data, Attention Engines and Tailrank, Nov 12, 2005

"Although we met briefly last week, Kevin Burton and I didn't manage to get enough time to discuss some of the things on our mind at the time, so we got a Skype call together and posted it as a podcast (.mp3, 42mb).

We focused the discussion around what he calls Meme Engines and I call Attention Engines, Tailrank (Kevin's latest project), OPML, RSS and Attention.xml"

Guests: Kevin Burton

--

Web 2.0 podcast, July 01, 2005

"Richard MacManus of Read/WriteWeb and I had a Skype chat this evening and recorded the call Talked about Web 2.0, attention.xml, a bit about RSS, APIs and more."

Guest: Richard MacManus

-

Attention podcast with Joshua Porter, Nov 26, 2005

"About OPML, Attention, and empowering people."

Guest: Joshua Porter

--