Indistinguishable from Magic

Its been a while since I've got this excited about an API, but oh you should have seen my eyes light up ten minutes ago. I've been meaning to take a look at the BizTalk Services SDK as soon as it was first thrown into the wild a few months ago, but it all seemed to abstract for me to easily grok. Well I just got my head around it tonight and this is what it is in a nutshell (everyone loves bullet points):

• Firstly its got nothing technically to do with BizTalk. It does share some architectural concepts in terms of its pub-sub mechanisms, service composition and workflow orchestration, but they are totally different products.
  
  
• Its am implementation of what Microsoft is calling the 'Internet Service Bus' which, from my understanding, is a cloud-based infrastructure that is responsible for amalgamating different services, transforming data from those services, and then routing the data to different listeners.
  
  
• Its a very beta product that MS have thrown out there to see what people can do with it. There are currently no availability or uptime guarantees so obviously not ready for production use at this point.

The three main components of the system are:

1. BizTalk Services Identity Provider - A WS-Trust-compliant identity provider and Secure Token Service (STS) that you can use Cardspace to issue tokens with
   
   
2. BizTalk Connectivity Service - A network relay service that enables applications to communicate regardless of what firewalls and NAT'ed routers you've got in between them
   
   
3. BizTalk Workflow Services - A yet-to-be-released implementation of cloud-hosted Windows Workflow instances. Rock and roll :-)

At first glance it is challenging to see what you can achieve with these components, however the benefit becomes apparent as soon as you start to delve a bit. It is all very compelling from an architectural perspective however what really got my rocks off was when I started to delve in the relay service. What I really wanted to know was how they implemented a WCF-based eventing mechanism over HTTP. So I downloaded the SDK, fired up my favourite network sniffer, loaded up the EchoService sample application and got snooping...

It turns out the way it works is firstly the services starts up and registers itself with the relay service, basically creating a URI for its clients to connect to. The service creates a long running HTTP connection and sends a ping message using a binary format that I couldn't easily determine. This keeps the socket connection alive and ready for the next part.

Clients connect up to the same URI after authenticating with the Cardspace IdP, and then can issue requests to the EchoService via the Relay Service. The relay is then responsible for routing that request down the appropriate socket connection all the way back to the listening EchoService.

Clients can then subscribe to events that the service publishes, using exactly the same mechanism. They connect to the Relay Service, express their intent to listen for certain message types and then establish a socket connection. As soon as the matching message is issued by the publishing service, the relay then matches the messages up to any client that has subscribed to them.

This is not a design that has never been implemented before (I'm sure IMAP Push and ActiveSync Always-Up-To-Date implement a similar model) however giving this functionality to developers in such a simple framework using APIs that they are already used to (WCF) is the really innovative part.

My recommendation approach to understanding it is first go grab a coffee (always a good start...) then watch the Channel 9 video that explains it from a high level, then just download the SDK and start playing around with the sample apps. There is also a good article in the upcoming MS Architecture Journal (not yet online) that explains the concept of the Internet Service Bus from a higher level.

So to explain this post's title, yes it does seem indistinguishable from magic when you first look at it, but without gushing (too much) I would say BizTalk Services is one of the most coolest bits of kit that I've seen come out of the Connected Systems Division in a long time.