Alex Barnett blog

Stuff

Microsoft Codename "Astoria": Data Services for the Web

Pablo Castro has announced a very cool project, Microsoft Codename "Astoria": Data Services for the Web.

So, if:

You build data-aware web applications (are there web devs that don't?)

You are interested in data access over the web (data over the web is what makes the web interesting)

You have been asking: "What is the programming model for data over web?" (I have)

You want more that simple file storage in the sky - you want relational data in the cloud you can program against - read and write (every developer I've spoken to wants this)

You like REST ('nuff said)

Then...read on.

Codename "Astoria" is the project I was working on before I left Microsoft. Astoria was probably the single hardest reason for me to leave as the momentum of the project was really picking up speed and the potential of Astoria was being grokked by a number of teams inside Microsoft. So to see Pablo's presentation today at MIX07 was nothing short of glorious.

On to "What is Astoria?"

Astoria is the cloudiest city in North America. Astoria is also the codename for an incubation project started some months ago attempting to answer the following questions: if you could provide a dead-simple way of programming against a relational data store that resides on the internet, what should the programming model look like? Could it be simpler than SOAP-based data access programming?

There are two parts (well, more - but we'll start with two) to the Astoria story.

The first part is the more traditional Microsoft CTP thing. What the downloadable bits allow you to do is to very rapidly provide a REST-based API - URLs- to an existing SQL Server database. That's right - REST to SQL in a jiffy! You download the bits for Visual Studio Orcas, do a couple of wizzy things and before you know it you have your data available through http. If you want, can then apply your own security model, constraints, etc. Downloads info and links here.

The second part is what I consider to be the coolest thing in data over web the today. The Microsoft team has deployed an online implementation of Astoria here, providing some sample read-only datasets for you to play with. There are plans to allow anyone to define their own data model and then have your data stored the same way as read-write, but it's not there today, and when that goes online it'll be for experimental use only, not the real-deal, yet.

There are currently four datasets for you to play with:

Northwind Data Service
Northwind is a classic database example, so we had to make it part of the sample set for the Astoria online service.

AdventureWorks Data Service
AdventureWorks is the new sample database that comes with Microsoft SQL Server 2005 and has a rich schema and a lot of data.

Encarta Data Service
The Microsoft® Encarta® team was nice enough to let us use a subset of their Encarta Encyclopedia article base and expose it as a data service.

TagSpace Data Service
The folks at Microsoft TagSpace were also kind enough to let us borrow their data (without user information, of course); this data service exposes the Tags, Tagged Items, Bookmarks and the relationships between them.

Let's look at the TagSpace Data Service as an example. Here's the entry point (data service root): http://astoria.sandbox.live.com/tagspace/tagspace.rse

What you see returned are the entities you can begin to browse though as an XML payload back (you can request JSON or simple RDF too): The three entities: Tags, TaggedItems and UserBookmarks.

<?xml version="1.0" encoding="utf-8" ?>

- <DataService xml:base="http://astoria.sandbox.live.com/tagspace/tagspace.rse">

- <TagSpaceEntities uri=".">

<Tags href="Tags" />

<TaggedItems href="TaggedItems" />

<UserBookmarks href="UserBookmarks" />

</TagSpaceEntities>

</DataService>

So, let's look at the all the tags. Simple add "/tags" to the end of the entry point URL: http://astoria.sandbox.live.com/tagspace/tagspace.rse/tags

One of the tags listed in the XML playload returned is vista:

- <Tag uri="Tags[vista]">

<TagName>vista</TagName>

<CreatedDate>11/6/2006 11:40:31 AM</CreatedDate>

<TaggedItems href="Tags[vista]/TaggedItems" />

Now let's replace "/tags" with "/Tags[vista]/TaggedItems" in the URL to get all the items tagged with 'vista': http://astoria.sandbox.live.com/tagspace/tagspace.rse/Tags[vista]/TaggedItems

Now we see all the the items tagged 'vista'. In the payload back, we see that 'TaggedItem' is another entity that is made up of a number of elements: the item ID (in this case '2'), the item title, the item description and the url to the item:

- <TaggedItem uri="TaggedItems[2]">

<TaggedItemId>2</TaggedItemId>

<Title>How do I change Windows Vista Media Center's Record folder...</Title>

<Description>Well, currently I installed Windows Vista Ultimate on my PC.  I have two harddrives, a 80gig, and a...</Description>

<Uri>http://beta.communities.microsoft.com/Forums/thread.aspx?ThreadId=6abaf29f-81e0-4cbb-bcab-93541e49f0ea&MessageId=ddf043cb-903f-437a-9aa3-730c92398b37</Uri>

Now I know the ID (the 'key') of a specific item, I can ask just for that item back by replacing "/Tags[vista]/TaggedItems" with TaggedItems[2]" in the URL hitting the data service: http://astoria.sandbox.live.com/tagspace/tagspace.rse/Tags[vista]/TaggedItems[2]

The underlining technology that allows the data to be viewed at entities is the ADO.NET Entity Framework, so the data model you are traversing is an Entity Data Model (the EDM consists of the high-level constructs “entities” and “associations”). The cool thing is, as a consumer of the data for your app (or as a human navigating through the URLs), you don't need to anything installed, you just hit the URLS and off you go. The payload returns enough information for you to then start navigating the data model and the data itself and use the data it for your apps.

The are a number of other very cool aspects to all this, and more detail is avaiable to read through in this .doc - Using Microsoft Codename Astoria, but to summarise just a few point here:

The data service is reachable over regular HTTP requests, and standard HTTP verbs such as GET, POST, PUT and DELETE are used to perform operations against the service.

The URL-based query syntax supports a number of operators. Examples:

eq

Equal

/Customers[City eq 'London']

ne

Not equal

/Customers[City ne 'London']

gt

Greater than

/Orders[OrderDate gt '1998-5-1']

gteq

Greater than or equal

/Orders[Freight gteq 800]

lt

Less than

/Orders[Freight lt 1]

lteq

Less than or equal

/Orders[OrderDate lteq '1999-5-4']

There are plenty more operators too. You can also create new entities using HTTP POST request and delete entites using HTTP DELETE against the URLs and manage associations between entities through URIs.

Powerful stuff, but to point out here: the team is looking for feedback. It's actually pretty unusual to have a project reveal so much at this early stage of a technology's development, but the team was itching to get feedback as early as possible in the design process.

If you are at MIX07, Pablo is repeating his session on Wedesday May 2, 11:30 AM - 12:45 PM, Lando 4201.

Can't wait for the team to update the service so we can start hosting our own data up there too...

--

Update - 11:19pm

I caught up with Pablo and video'd our chat together. Also, blog reactions so far have been positive. New post here.

Comments

TrackBack said:

# April 30, 2007 10:47 PM

Alex Barnett blog said:

I caught up with Pablo Castro, who announced Codename Astoria - Data Services for the Web earlier today

# May 1, 2007 12:16 AM

Sam Sethi said:

This is very good but it is the same as Google GData nd why don't Microsoft mention that it is basically Atom Publishing Protocol to SQLServer.

Both companies are going the same way with RESTful cloud stores accessible via HTTP verbs.

The only difference is Microsoft are using Silverlight to make .NET cross platform XAML APPS instead of trying to make IE cross platform.  Google is relying on Mozilla to make FF 3.0 a cross platform RIA platform with XUL.

Let the fun and games begin.  

# May 1, 2007 10:05 AM

alexbarnett said:

Sam - thanks for popping by. I have to disagree with your assertion that GData and Astoria are the same thing. I've got time to explain why I disagree right now, but will address this in a post soon.

# May 1, 2007 11:44 AM

TrackBack said:

# May 1, 2007 7:21 PM

TrackBack said:

# May 1, 2007 7:21 PM

TrackBack said:

# May 1, 2007 7:21 PM

TrackBack said:

# May 2, 2007 10:50 AM

Atteint de Javascriptite aigüe said:

Lors du Mix de las Vegas, Microsoft a annoncé de très nombreux projet, je me suis récemment interessé

# May 10, 2007 11:54 AM

Alex Barnett blog said:

&lt;warning: the following post meanders , is unstrucutured and doesn't arrive at any conclusion&gt;

# May 22, 2007 10:32 AM

SSIS Junkie said:

Just lately I&#39;ve been doing a bit of searching around to see what data-centric offerings, other than

# June 30, 2007 12:49 AM

Alex Barnett blog said:

Back in April, the Data Programmability team at Microsoft announced "Astoria": Data Services for the

# August 3, 2007 8:25 PM

Catalin's Blog said:

Cam greu m-am adunat sa scriu si despre ele dar backlogu.... si o sa incerc sa splitui pe evnts ca sa

# December 14, 2007 1:11 AM

Website Marketing said:

Microsoft Codename "Astoria": Data Services for the Web - Alex Barnett blog

# October 18, 2014 7:50 PM