Alex Barnett blog : Astoria

Why OData Matters (IMHO)

alexbarnett — Fri, 19 Mar 2010 15:24:00 GMT

Earlier this week I was in the MIX10 crowd as Douglas Purdy announced the Open Data Protocol (it was a great presentation - summarized here).

I want to share with you why I think OData could be a very big deal…But before we go there...let's start with the basics...

What is OData? Where Did OData Come From?

To understand the history of how OData came to be, you need to understand how project "Astoria" came to be...I won't go over that again as this is already pretty well documented. ~~Astoria~~ OData has come a long way since.

In a nutshell: Today, OData builds on a few conventions, popularized by AtomPub (see OData AtomPub Format), to using REST-based data services. These services allow resources, identified using Uniform Resource Identifiers (URIs) and defined in an abstract data model (see OData URI Conventions, to be read and edited by web clients using simple HTTP messages (see OData Operations).

An Open Data Protocol for the Web

OData offers a standardized way for programmable data to be made available across the web and in turn allowing "consumers" of that data to rely on a set of conventions to be followed that in turn allows many interesting things to happen if widely adopted...

...And to this end:

As announced, OData has been released by Microsoft under the Open Specification Promise (OSP) "to allow anyone to freely interoperate with OData implementations" . Since then, the W3C has invited the OData team to Bring OData to a W3C Incubator (I haven't seen a public response yet but I urge the team to do so.).
OData is not a Microsoft-only thing and it won’t succeed if it is. The originating philosophy was about transparency in the design process, with an Open end-point as the goal - not a .NET lock-in play (“agree on standards and compete on implementation”). This approach has already yielded an initial set of clients, servers, services, and tools. Today, a number of OData SDKs and libraries are available for .NET, Java, PHP, iPhone (Objective-C) and more – and there’ll be more coming.
There is a small but growing ecosystem of non-Microsoft "producers" and "consumers" - (where OData "producers" include Netflix’s catalog as OData and the VanGuide (a social map of Vancouver Open Data) and Public data from the city of Edmonton as OData) and OData “consumers” - standard web browsers, RIA "data explorers" – such as Sesame OData Browser, and the client libraries mentioned above – p.s. somebody build a javascript-only data browser please!)

Where can OData take us?

The clue is in the OData icon (next to the RSS feed icon. Can you see the similarities?

The big idea here is that in the same way we have the "RSS" feed icon, we'll get used to seeing the "OData" icon on commercial and non-commercial websites everywhere (especially for government-related data). So in the same way you know today that the RSS icon means "get an XML feed for this content", the "OData" icon means "get this web data" - you'll know (and your client will know) what to expect in terms of how to read in, and navigate through and query structured web data sets - and in many cases write against them - using a common syntax.

Q: Right, But So What?

A1: Open Government OData. From Open Government Data Principles

The Internet is the public space of the modern world, and through it governments now have the opportunity to better understand the needs of their citizens and citizens may participate more fully in their government. Information becomes more valuable as it is shared, less valuable as it is hoarded. Open data promotes increased civil discourse, improved public welfare, and a more efficient use of public resources.

That’s great, but it needs to be practicable…And number 5 of the 8 Principles of Open Government Data sensibly states that the data should be (via David Eaves):

5. Machine processable

Data is reasonably structured to allow automated processing.

It'll be down to each government agency (and local government) as to how they decide to implement this principle, but wouldn't it be great if they agreed to a standard (and a powerfully simple, web-oriented one at that)? This is what Jon Udell concluded here:

"The open data movement, in government and elsewhere, aims to help people engage with and participate in processes represented by the data. When you publish data in a fully articulated way, you build a framework for engagement, a trellis for participation. This is a huge opportunity, and it’s what most excites me about OData"

A2: To ODatarize your data is to RESTify your data.

As more data-oriented web APIs come online, each team responsible for the design of each web API is confronted by the same kinds of questions, and each team answers these in their own particular way. Increasingly, “RESTful” is a design goal of web APIs. Great…but what does that mean? How do you expose the data, the relationships between the entities inside the model, and what should the querying syntax look like? Unfortunately, there are as many answers to these questions as there are RESTful web APIs. And there needn’t be.

For to ODatarize your data is to RESTify your data. Do spend the time at the value layer - figure out the way your developers / consumers want to see the data and expose it that way. Do make it easy for devs / consumers to learn / navigate about the data and use it. Do not make them learn about the unique idiosyncrasies you’ve built into your API (or those that leak out of your originating store) :-)

From a developer’s standpoint, OData is ultimately about productivity.

For the OData “Production Developer”: Point at your data store – define your entity model and map it to the data model you already have (so your developers consume / program against the data that makes most sense to them – effectively ORM’ing) and expose as an OData service, inheriting: all the REST characteristics; entity relationship self-discovery; and querying goodness.
For the OData “Consuming Developer”: If you know the web API is OData…great! Pick up a client library, get to the API end-point (data.foo.org/blah.svc). Point and Shoot: Traverse the data model, query it (and bookmark as needed – it’s a URI)…play!

(see links at the bottom of this post to technical content that provides details on all this)

A3: Since the announcement, I’ve seen quite a bit of excitement around the web (especially Twitter) by developers who see the potential here…there is plenty of experimentation going on. At Intuit, my team is also experimenting with ODatarizing some of our data services, exploring how it might be applied across a number of our cloud based data services. And when our team’s architect Tweets that “Looks like #odata is going to be a good fit for our data services”, I know there’s something interesting going on here…

So I encourage you to find out more about OData and get involved.

More Resources

An easy way to set up an OData feed from your SQL Azure database - Stephen Forte
OData interoperability with .NET, Java, PHP, iPhone and more - MSFT Interop team
Microsoft, Bring OData to a W3C Incubator - W3C blog -
Silverlight 4 + RIA Services - Ready for Business: Exposing OData Services-4-ria-services-ready-for-business-exposing-odata-services.aspx - Brad Abrams (MSFT)
Open Data for the Open Web - Douglas Purdy (MSFT)
Producing and consuming OData feeds: An end-to-end example - Jon Udell (MSFT)
OData for collaborative sense-making - Jon Udell
The three laws of open government data – David Eaves
8 Open Government Data Principles
Open Data Protocol (OData) with Pablo Castro - Hanselminutes podcast interview with Pablo Castro (MSFT – Architect for OData)
OData, a RESTful contender for your API - Andy Thompson
Announcing Sesame Data Browser - Fabrice Marguerie
Retrieving Data with the OData SDK for PHP - Brian Swann
Got SQL Azure? Then you've got OData - OData blog
Implementing only certain aspects of OData-only-certain-aspects-of-odata.aspx - Pablo Castro
Every SharePoint 2010 server is a Data Services server - Pablo Castro
Microsoft Creates New OData.org Website for Open Data Protocol – CMS Wire
Microsoft delivers updates on OData, Houston, Dallas - ZDNet
Timeline of Project Astoria - Pablo Castro
How ADO.NET Data Services came to be (formerly known as Project Astoria) - Alex Barnett
previous Astoria / OData posts - Alex Barnett

How ADO.NET Data Services came to be (formerly known as Project Astoria)

alexbarnett — Thu, 21 Aug 2008 02:16:00 GMT

Pablo Castro has recounted some of his timelined memories about how "Project Astoria" evolved from a lunch time conversation to bits in .NET 3.5 SP1 and Visual Studio 2008 SP1 now known as ADO.NET Data Services Framework). Nice write up.

Three memories of my own to add to the story:

1. I was reading up on the whole REST thing in the summer of 2006 - its origins, philosophy and design patterns. I knew there was something interesting going on and some potential dots to join, but I wasn't sure which dots...So I collated and circulated a bunch of research / links to the team, then blogged the links (I liked How I explained REST to my wife. More recently see Explaining REST to Damien Katz). I got a few proverbial (and some literal) blank stares as I shared my enthusiasm for REST, asking how we could apply the ideas to the various projects we were working on. It was Pablo, and (as Pablo attests) Britt Johnston (now a PUM for SQL Business) who were able to develop the initial conceptual leaps into something more concrete like a Think Week Paper and a prototype demo.

2. When it came to brainstorming the code name, the team agreed on a "cloud" theme. A number of proposals were floated around along with their rationales, including "cumulus" and "cirrus". We were then advised that city and town code names were legal-safe. So there we were, struggling to agree on some city or town name we all liked (or at least not hate nor be confused by..."how about Nameless?"...), and then Mike Pizzo's proposal came in: "Astoria - hey, it's the cloudiest city in the USA!" (at least it was in 2006). Sold.

3. I think my favorite memory of all is the reaction Gary Flake provided (of Microsoft's Live Labs) to the prototype Pablo demo'd at one of the pitch meetings: "As God himself would have designed it!" Dr Flake exclaimed..."Cool", I thought to myself - "but does that mean no REST for the wicked?"

Astoria at MIX08 (REST in Vegas)

alexbarnett — Wed, 05 Mar 2008 02:11:00 GMT

As much as I'd love to make it to MIX08 this week, time will not allow me...But if I were, then I'd be going to the following three sessions related to the Project Formerly Known as Astoria:

Wed, March 5th - RESTful Data Services with the ADO.NET Data Services Framework
Fri, March 7th - Accessing Windows Live Services via AtomPub
Fri, March 7th - Building RESTful Real World Applications with the ADO.NET Data Services Framework

Podcast with John Musser of ProgrammableWeb.com

alexbarnett — Wed, 19 Sep 2007 16:38:00 GMT

A couple of weeks back John Musser of ProgrammableWeb.com joined me and Ted for a chat to discuss the state of web APIs and the API trends as he sees them.

We've now recorded the conversation and published as the first of a newly launched Bungee Line podcast series.

Topic covered include Facebook APIs, Amazon's recently launched Flexible Payment Service (FPS) , Google Base, Microsoft's Astoria and relational-data-in-the-cloud programming models and services, SaaS models and API SLAs, REST vs SOAP, "Closed is Still the Old Closed" and plenty more.

Thanks to John for his time.

ADO.NET Entity Framework video- 1 year later

alexbarnett — Thu, 30 Aug 2007 14:04:00 GMT

Some old Data Programmability teammates of mine, Britt Johnston, Pablo Castro and Mike Pizzo (well, Mike's not that old), are interviewed by Charles Torre in this Channel 9 video to discuss their progress, challenges and customer feedback in the last year since announcing the first ADO.NET Entity Framework CTP, including some details on project Astoria and LINQ to Entities support, details on the Beta 2 and Tools CTP released earlier this week (the entity model designer is v.cool), plus the future of the Entity Framework.

P.S. Michael - in case you're reading: reconsider the shorts for the next interview :-P

Astoria: Data Services for the Web - Part 2

alexbarnett — Sat, 04 Aug 2007 02:17:00 GMT

Back in April, the Data Programmability team at Microsoft announced "Astoria": Data Services for the Web, an incubation project exploring programming models for data over the web. As part of the announcement, the team deployed an online implementation of Astoria here, providing some sample read-only datasets for you to play with (see this post for examples). They also mentioned that plans were underway to allow web developers to define their own data model and have their own data stored as read-write relational data service "in the cloud", programmable over a RESTful API.

What's New

Today, Mike Flasko, Astoria Program Manager announced the delivery of this second piece of Astoria online service, where

"web developers can create custom structured data stores (up to 100MB in size) on the web and access them from anywhere that they have Internet access. These data services can then be the storage or data source for mashups, or the backing store for Internet enabled applications, or be applied to any other scenario where a rich data service on the web is required, independently of where it is hosted. "

(Astoria Program Manager? This is incubation project is going somewhere... :-)

You create your own data model for the Astoria online service by web-based tool that lets you define your data model (entities, properties and associations). Be warned - it's a little rough at the edges, but good enough. Once you've you've defined your data model the service generates the data store for you and hands you an URL...Yes! Your very own read/write REST interface to your own data store.

So now I've got my own relational data service going (note: will you require a username / password, which I ain't going to give you :-) at https://astoria.sandbox.live.com/users/alexrest/alexrest.rse

And all this without requiring the installation of Microsoft bits - no Visual Studio, no SQL Server, nada!

I'll be experimenting a little more with the new service and hope to share some interesting possibilities in the future, but I think it is fair to say that this incubation project has some seriously interesting and broad potential. Congrats to the team for pulling this off!

More at:

Astoria team blog
Pablo Castro's blog
Elisa Johnson's blog
(8/6/07) Mary Jo Foley - Microsoft Code Name of the Day: Astoria
(8/8/07) Jamie Thomson has been playing and got his own service running

Previous Astoria posts:

Ozzie's "Cloud OS" Raises More Questions than Answers

alexbarnett — Fri, 27 Jul 2007 20:04:00 GMT

Ray Ozzie's briefing this week provided quite a bit more detail around Microsoft's "Software Plus Services" strategy. It's definitely worth a read (or a look, and if you're feeling too lazy for either you can read Nick Carr's summary).

It's been a year since Ozzie took over the role as Chief Software Architect from Bill Gates, and I think it is exciting to see his influence further emerge throughout the business, architectural and experential direction of Microsoft. The 30 year old company needs this injection - a shot in the arm. And his vision is the right one. It is the only one that has any chance of seeing Microsoft through its need for growth.

However, as the Ozzie's "Cloud OS" story slowly becomes more concrete, the future influence that Microsoft will have throughout the software and internet services ecosystem is becoming less clear.

Yes, we know Software as a Service (Saas) is becoming an increasingly significant trend, and we know that the enabling role Web Services (SOAP and REST based) has to play as part of the overall move to a distributed computing model is becoming ever more central, and we know that the browser will continue to further its dominance as the primary interface between humans and data, functionality and people, but what is not so clear is how many "major players" there will be in that future, what their roles will be, nor what the roles of the "everyone elses" will be.

Microsoft Partners have been assured a place by Microsoft's side in this future, but does anyone really know? How will all this fall out? How will Microsoft's traditional partner profile fit into Ozzie's new brave future? What kind of ecosystems will emerge? Will Microsoft's ecosystem of tomorrow look radically different to its ecosystem of today? Who are the Microsoft partners of today who will find themselves competing head-to-head with Microsoft tomorrow? What will Microsoft's competition of the future even look like?

The answers to some of these question may surprise us. How many people, for example, would have imagined a just few years ago that search engine providers or an online bookseller or online university network would emerge to become a serious potential competitor in the computing and software space of Microsoft? Not many. In the second internet age Microsoft's future competition and partners can literally come from any direction at any time. And they often do. In many respects, the future looks bright, but I suspect that for many in the software / computing industry the future is also very cloudy indeed.

RESTpectful comparison of Microsoft's Astoria and Google Base Data APIs

alexbarnett — Fri, 13 Jul 2007 06:36:00 GMT

Dare has written up RESTpectful comparison of the programming models of Microsoft's Astoria and Google Base Data API for querying relational data over the web (RESTful data services).

Both these APIs also happen to support operations for changing the data - inserting, updating, and deleting - Google's details here and Astoria's in this .doc, where in addition Astoria allows associations between entities to be created and deleted). These aren't mentioned in Dare's analysis (I'm sure he knows of the ability to change the data via the APIs, not just query), however the article is worth a read nonetheless.

Astoria podcast - Jon Udell interviews Pablo Castro

alexbarnett — Tue, 03 Jul 2007 21:13:00 GMT

Here's something I've been looking forward to for a while - Jon Udell interviewing Pablo Castro on the topic of Astoria.

After the interview Jon wrote:

"I’m not even close to being an expert in the underlying data access technologies, including ADO.NET, the Entity Data Model, and LINQ, so parts of the discussion quite frankly went over my head. Nor am I yet familiar with the tooling that’s required to wrap this kind of services layer around a plain data source. But I’m 100% clear that it’s a good idea, and a great example of RESTful web services — a book that Pablo Castro says is “required reading” for members of the Astoria team."

If you like the sound of Astoria keep an eye on Pablo's blog.

REST / ROA / Astoria / SOA

alexbarnett — Thu, 24 May 2007 22:46:00 GMT

Jon Udell, rightly so, is getting excited about the new book RESTful Web Services, by Leonard Richardson and Sam Ruby - a book I myself got excited about in November (REST Web Services and ROA) when I wrote at the time:

"My bet is that this project, the book and the concept of ROA as an effort to standardize aspects of RESTful architecture design is going to do very well, because I agree this is needed. Example provided:

""REST" is an abused term. Many alleged REST services (such as Flickr's, which says REST right on the website) are actually HTTP+POX: service-oriented APIs that happen to use nothing more the basic technologies of the web. Classifying services as resource-oriented or service-oriented makes it easy to see which ones are more and which less RESTful, without wasting time on minutiae.""

(more links to reviews here)

I don't know if John, Leonard or Sam have spent any time to look at the Astoria project (think - RESTful data services) - see links here, but would love to know what they think - in particular regarding the query syntax (good? bad? too complex? too simple?) and where it sits on the ROA (WOA?) to SOA continuum (WOA links) they discussed.

Mashup design patterns

alexbarnett — Tue, 22 May 2007 16:27:00 GMT

<warning: the following post meanders, is unstrucutured and doesn't arrive at any conclusion>

In the middle of last year, I invited Dion to meet with me and other members of Microsoft's Data Programmability team to discuss REST in the context of data access over the web. Dion's perspective is one of the reasons our team was able to recognise the potential of a RESTful programming model against relational data, manifested today as Project Astoria.

I've been waiting to read Dion's thoughts on Project Astoria (via John Musser). It's an interesting take, contextualizing Astoria within the overall architectural trends we're seeing take place.

I liked this diagram and got me thinking (dangerous...):

Dion warns that the diagram like this could never be complete representation of the concepts discussed in his post and is work in progress. The mapping done here for the 3 types of applications (Composite Applications, Recombinant Software and Mashups) makes some sense to me. However, it reminds me that I have used the terms "Composite applications" and "Mashups" interchangeably (but never Recombinant Software) depending on who I'm speaking to and without really thinking through what I mean by these terms. To try and clarify my own thinking on this, here's my effort to do so...

Here are the Wikipedia definitions of Composite Applications, Recombinant Software and Mashups.

Composite applications

"the term composite application expresses a perspective of software engineering that defines an application built by combining multiple services. A composite application consists of functionality drawn from several different sources within a service oriented architecture (SOA). The components may be individual web services, selected functions from within other applications, or entire systems whose outputs have been packaged as web services (often legacy systems)."
Recombinant software
<none>
Mashup
"A mashup is a website or application that combines content from more than one source into an integrated experience."

So nothing in Wikipedia for "Recombinant software", but in searching for a definition I did come another post by Dion written in June 2006 - "Is IBM making enterprise mashups respectable?". Here Dion asserted:

"Accepting that information, visuals, and software can be remixed and combined at multiple levels in an application stack means that there are (at least) five places that mashups can take place."

Dion then went on to define 5 mashup types:

Presentation Mashup: This is the shallowest form of mashup in the sense that underlying data and functionality don’t meet. Information and laout is retrieved and either remix or just placed next to each other. Many of the Ajax desktops today fall into this category and so do portals and other presentation mashup techniques.
Client-Side Data Mashup: A slight deeper form of mashup is the data mashup which takes information from remote Web services, feeds, or even just plain HTML and combines it with data from another source. New information that didn’t exist before can result such as when addresses are geocoded and display on a map to create a visualization that could exist without the underlying combination of data.
Client-Side Software Mashup: This is where code is integrated in the browser to result in a distinct new capability. While a component model for the browser is only now being hashed out as part of Open Ajax, there is considerable potential in being able to easily wire together pieces of browser-based software into brand new functionality.
Server-Side Software Mashup: Recombinant software is probably easier right now on the server since Web services can more easily use other Web services and there are less security restrictions and cross domain issues. As a result, server-side mashups like those that in turn use things like Amazon’s Mechanical Turk or any of the hundreds of open Web APIs currently available, are quite common.
Server-Side Data Mashup: Databases have been linking and connecting data for decades, and as such, they have relatively powerful mechanisms to join or mashup data under the covers, on the server-side. While it’s still harder to mashup up data across databases from different vendors, products like Microsoft SQL Server increasingly make it much easier to do. This points out that many applications we have today are early forms of mashups, despite the term. Of course, the more interesting and newer aspects of mashups happen above this level.

Back to the plot...

When speaking with an "enterprisey" architect I use the term "Composite applications". "Enterprisey" architects think and talk in terms of governance, security, scale and complexity. They also like to hear and use technical sounding terms that confound non-IT folks ;-)

When speaking with a developer, I tend to use the term "Mashups". Developers are smart and lazy - they want to write fewer lines of code, do more with less and love to impress. In my mind at least, the term "mashup" implies a speedy development / test / deploy cycle, design flexibility and simplicity in the sense that its "less hard to do".

Now, if I'm talking to a developer who works in the enterprise context (enterprise as in a large business) the term "enterprise mashup" can connect the "enterprisey" values - governance, security, scale but without the complexity AND with the "mashup" values - speed, flexibility and simplicity.

Reviewing the 5 mashup design patterns above, there is reasonably defined line: Client-side vs. Server-side. Some of the fall into the WOA space - referring to Dion's diagram above:

"Another important item: The bottom of the overlapping circle contains a cryptic acronym near the edge of the circle: WOA. This stands for Web-Oriented Architecture, a concept that I’ve written about several times here and here in this blog. It’s an idea that basically states that software that goes naturally with the “grain” of the Web, extending the core infrastructure of the Web in natural ways, works the best."

Interview with Pablo Castro on Astoria - Data Services for the Web

alexbarnett — Tue, 01 May 2007 05:49:00 GMT

I caught up with Pablo Castro, who announced Codename Astoria - Data Services for the Web earlier today at MIX07.

Here's the video of our chat together.

In the meantime, there have been a number of blog posts commenting on the Astoria announcement via TechMeme:

John Musser at ProgrammableWeb.com:

"Although it’s an early experimental release, it’s quite an interesting approach using a very RESTful set of patterns and infrastructure for web data services."

Michael Coté of Redmonk:

"If you’re interested in REST, SOA (hopefully, of the REST type), SaaS, or (though I shudder to type it) “the semantic web,” take a look at Astoria yourself. Like I said, it’s all at the “labs”/”project” stage now. But, that means there’s a chance to get in there and influence what the final release turns out to be. Does the interface and use model “work”? Do you like it?"

Danny Ayers:

"They've totally gone to town on URIs and HTTP access - very much leaning towards REST. What's exposed is proper web stuff. There's a fair bit of innovation there that looks useful, it'll be interesting to see what RESTafarian gurus make of it. So the insides will be good for MS-heads - great, anything that encourages building web-friendly services has to be a good thing."

Alex James:

"Combine REST with a conceptual model and well you get Data 2.0.

Astoria leverages REST + Entity Data Model to expose data to the web.

Very cool."

Microsoft Codename "Astoria": Data Services for the Web

alexbarnett — Tue, 01 May 2007 01:28:00 GMT

Pablo Castro has announced a very cool project, Microsoft Codename "Astoria": Data Services for the Web.

So, if:

You build data-aware web applications (are there web devs that don't?)

You are interested in data access over the web (data over the web is what makes the web interesting)

You have been asking: "What is the programming model for data over web?" (I have)

You want more that simple file storage in the sky - you want relational data in the cloud you can program against - read and write (every developer I've spoken to wants this)

You like REST ('nuff said)

Then...read on.

Codename "Astoria" is the project I was working on before I left Microsoft. Astoria was probably the single hardest reason for me to leave as the momentum of the project was really picking up speed and the potential of Astoria was being grokked by a number of teams inside Microsoft. So to see Pablo's presentation today at MIX07 was nothing short of glorious.

On to "What is Astoria?"

Astoria is the cloudiest city in North America. Astoria is also the codename for an incubation project started some months ago attempting to answer the following questions: if you could provide a dead-simple way of programming against a relational data store that resides on the internet, what should the programming model look like? Could it be simpler than SOAP-based data access programming?

There are two parts (well, more - but we'll start with two) to the Astoria story.

The first part is the more traditional Microsoft CTP thing. What the downloadable bits allow you to do is to very rapidly provide a REST-based API - URLs- to an existing SQL Server database. That's right - REST to SQL in a jiffy! You download the bits for Visual Studio Orcas, do a couple of wizzy things and before you know it you have your data available through http. If you want, can then apply your own security model, constraints, etc. Downloads info and links here.

The second part is what I consider to be the coolest thing in data over web the today. The Microsoft team has deployed an online implementation of Astoria here, providing some sample read-only datasets for you to play with. There are plans to allow anyone to define their own data model and then have your data stored the same way as read-write, but it's not there today, and when that goes online it'll be for experimental use only, not the real-deal, yet.

There are currently four datasets for you to play with:

Northwind Data Service
Northwind is a classic database example, so we had to make it part of the sample set for the Astoria online service.
AdventureWorks Data Service
AdventureWorks is the new sample database that comes with Microsoft SQL Server 2005 and has a rich schema and a lot of data.
Encarta Data Service
The Microsoft® Encarta® team was nice enough to let us use a subset of their Encarta Encyclopedia article base and expose it as a data service.
TagSpace Data Service
The folks at Microsoft TagSpace were also kind enough to let us borrow their data (without user information, of course); this data service exposes the Tags, Tagged Items, Bookmarks and the relationships between them.

Let's look at the TagSpace Data Service as an example. Here's the entry point (data service root): http://astoria.sandbox.live.com/tagspace/tagspace.rse

What you see returned are the entities you can begin to browse though as an XML payload back (you can request JSON or simple RDF too): The three entities: Tags, TaggedItems and UserBookmarks.

<?xml version="1.0" encoding="utf-8" ?>
- <DataService xml:base="http://astoria.sandbox.live.com/tagspace/tagspace.rse">
- <TagSpaceEntities uri=".">
<Tags href="Tags" />
<TaggedItems href="TaggedItems" />
<UserBookmarks href="UserBookmarks" />
</TagSpaceEntities>
</DataService>

So, let's look at the all the tags. Simple add "/tags" to the end of the entry point URL: http://astoria.sandbox.live.com/tagspace/tagspace.rse/tags

One of the tags listed in the XML playload returned is vista:

- <Tag uri="Tags[vista]">
<TagName>vista</TagName>
<CreatedDate>11/6/2006 11:40:31 AM</CreatedDate>
<TaggedItems href="Tags[vista]/TaggedItems" />

Now let's replace "/tags" with "/Tags[vista]/TaggedItems" in the URL to get all the items tagged with 'vista': http://astoria.sandbox.live.com/tagspace/tagspace.rse/Tags[vista]/TaggedItems

Now we see all the the items tagged 'vista'. In the payload back, we see that 'TaggedItem' is another entity that is made up of a number of elements: the item ID (in this case '2'), the item title, the item description and the url to the item:

- <TaggedItem uri="TaggedItems[2]">
<TaggedItemId>2</TaggedItemId>
<Title>How do I change Windows Vista Media Center's Record folder...</Title>
<Description>Well, currently I installed Windows Vista Ultimate on my PC. I have two harddrives, a 80gig, and a...</Description>
<Uri>http://beta.communities.microsoft.com/Forums/thread.aspx?ThreadId=6abaf29f-81e0-4cbb-bcab-93541e49f0ea&MessageId=ddf043cb-903f-437a-9aa3-730c92398b37</Uri>

Now I know the ID (the 'key') of a specific item, I can ask just for that item back by replacing "/Tags[vista]/TaggedItems" with TaggedItems[2]" in the URL hitting the data service: http://astoria.sandbox.live.com/tagspace/tagspace.rse/Tags[vista]/TaggedItems[2]

The underlining technology that allows the data to be viewed at entities is the ADO.NET Entity Framework, so the data model you are traversing is an Entity Data Model (the EDM consists of the high-level constructs “entities” and “associations”). The cool thing is, as a consumer of the data for your app (or as a human navigating through the URLs), you don't need to anything installed, you just hit the URLS and off you go. The payload returns enough information for you to then start navigating the data model and the data itself and use the data it for your apps.

The are a number of other very cool aspects to all this, and more detail is avaiable to read through in this .doc - Using Microsoft Codename Astoria, but to summarise just a few point here:

The data service is reachable over regular HTTP requests, and standard HTTP verbs such as GET, POST, PUT and DELETE are used to perform operations against the service.

The URL-based query syntax supports a number of operators. Examples:

eq
Equal
/Customers[City eq 'London']
ne
Not equal
/Customers[City ne 'London']
gt
Greater than
/Orders[OrderDate gt '1998-5-1']
gteq
Greater than or equal
/Orders[Freight gteq 800]
lt
Less than
/Orders[Freight lt 1]
lteq
Less than or equal
/Orders[OrderDate lteq '1999-5-4']

There are plenty more operators too. You can also create new entities using HTTP POST request and delete entites using HTTP DELETE against the URLs and manage associations between entities through URIs.

Powerful stuff, but to point out here: the team is looking for feedback. It's actually pretty unusual to have a project reveal so much at this early stage of a technology's development, but the team was itching to get feedback as early as possible in the design process.

If you are at MIX07, Pablo is repeating his session on Wedesday May 2, 11:30 AM - 12:45 PM, Lando 4201.

Can't wait for the team to update the service so we can start hosting our own data up there too...

Update - 11:19pm

I caught up with Pablo and video'd our chat together. Also, blog reactions so far have been positive. New post here.