I'm sorry Dave, I'm afraid I can't do that

RSS Feed Homepage

Bits and Pieces
2007-12-02 13:54:00
This has been a pretty content-free zone lately. Most of that has been due to work pressure. I've got a few systems going live around now and, as always, the last 20% of each project takes 80% of the time. Anyway, I have been looking at a few things here and there that might be of interest... First off, I noticed that George Chrysanthakopoulos has announced on the MSRS CCR forum that work is underway to unify the CCR with the PFX libraries recently released to CTP, although no time-frames have been given. Additionally, George also states that they are working towards the integration of the CCR into the core .NET libraries (possibly via PFX?). This is excellent news all round. I read with interest the MSDN article on the Task Parallel library, which sits at the core of PFX. Actually, in my line of work, I believe the CCR would be more useful than PFX, as taming the asynchrony, whilst exploiting the latent concurrency tends to be the bigger (and harder) problem than parallelising the...
More About: Pieces , Bits

F# Goes 'Official'
2007-10-18 23:09:00
Joe Duffy points to the announcement that the Developer Division at Microsoft is folding F# into the 'official' set of supported .NET languages. This is great news.
More About: Official

Hiring...
2007-10-17 15:35:00
I don't normally write about my job, so I'm going to write about a different one that could be yours... My team has vacancies. We design, develop and support equity-derivatives trading systems for one of Australia's foremost financial institutions for markets at home and abroad. You'd like to work in a small, highly-skilled team of software developers and have some or all of the following: Strong C++ and optionally VB6 and/or .NET Significant experience of multi-threaded server application development SQL experience (preferably SQL Server) A solid understanding of IP-based communication protocols A working knowledge of equity-derivative products A working knowledge of client-side development You will be a self-starter who can work with minimum supervision on projects from inception through delivery and onto support, across the various architectural tiers. You will be comfortable communicating with the key business stake-holders, primarily traders and business managers...
More About: Hiring

F# and Message-Passing
2007-10-17 14:10:00
My inexperience with F# has caught me out. Robert Pickering has an article on asynchronous workflows (similar concept to CCR iterators) in which he mentions a forthcoming article on 'mailboxes', the message-passing feature I said didn't exist. I look forward to it.
More About: Message , Passing , Mess

Why COM STAs aren't completely awful
2007-10-10 14:30:00
...and why productivity and pragmatism are important too. The last two paragraphs are especially interesting.
More About: Awful

F# and DSS
2007-10-10 14:18:00
So I rushed ahead of myself a bit and though that I'd try and implement a DSS service in F#. I've posted below what I think is the equivalent code for the default DSS service you get when you create a new DSS simple service project. There are three salient points: I've collapsed the *Types and *Service into a single file. The support for CCR iterators in the handler methods is a little bit of a hack (my hack). It doesn't work. It doesn't even compile. (3) is clearly the most salient point. DSS requires that the Contract class in a service namespace expose a string literal called 'Identifier' that contains the contract URI. Unfortunately, the code listed below doesn't expose a string literal but rather a read-only property. CLR aficionados will know that the difference is that clients that refer to 'Contract.Identifier' via a literal end up with that literal in their own assembly so that if that literal changes in the source assembly, pre-existing dependent assemblies...

F# and CCR
2007-10-08 03:41:00
I mentioned a while back that I'd read Joe Armstrong's PhD thesis on building reliable distributed systems. Early on, Joe introduces Erlang, a language he and his team designed at Ericsson to build large-scale, fault-tolerant, concurrent systems. I've no background at all in functional languages, but I was struck by 4 things (in no particular order): The code seemed to have hit a sweet-spot of readability and brevity. It's a compact syntax. The support of functions as first-class types (a feature of functional languages), allowed for some simple but powerful constructs. A lot of type inference (and a lot less typing!) Language support for message-passing primitives. I won't expand particularly on these right now, but some of these features have crept into C#, notably limited type inference (i.e. the var keyword) and lambda functions (via succinct syntax for anonymous delegates). Anyway, whilst reading round Erlang and functional languages in general, I discovered F...

Crunched
2007-09-20 13:40:00
I was asked on the Australian DotNet Mailing List (send "subscribe dotnet" in the body of the email to: imailsrv@stanski.com) if I could provide the CCR posts as a document, to make it easier to print off. Sort of. My host (blogger) doesn't support downloads as far as I can tell, and Windows Live Writer (Beta) has no facility for saving posts as word documents, so it was a cut-and-paste job in the end. The document can be found on FileCrunch. It's a little rough and ready, in particular the spacing of the code samples is not all it could be. Any hints or tips on this greatly received.

Innovation book
2007-09-17 13:46:00
I've started reading Scott Berkun's 'The Myths of Innovation '. It's an interesting read, exposing the cultural myths that surround ideas, those who have them and those who capitalise on them. It's also an O'Reilly book without an animal on the front!
More About: Book

DSS Optimizations
2007-09-12 15:11:00
I was looking at some of the code generated by the dssproxy tool. When you define a service (by specifying types and operations), the dssproxy tool takes your service assembly and generates two associated assemblies, a proxy and a transform. The proxy is the assembly that clients of the service use to send and receive messages. It contains representations of the service types that can also serialize and deserialize themselves for transmission. I noticed that these representations have a small inefficiency on how they deal with the deserialization of List<T>s. Prior to the state of each T is the count of all Ts in the list. This information is used to determine how many Ts to expect in the stream. It could also be used to construct the list with an initial capacity, to avoid excessive re-allocation and copying for large numbers of T. The generated code doesn't currently do this. Fortunately, you do have the source for the proxies, so you can change it yourself if it really mat...

All (not) my own work
2007-08-22 23:33:00
I've not posted much of my own recently. Outside of work (which has been getting a little hectic lately), I've been working on a small, sample project that will hopefully be a good segway into Distributed Software Services, which provides an application model for highly-concurrent, message-passing based programs in .NET. That will come after I finish off covering the remaining CCR features, specifically causalities and maybe interop. I'd be happy to hear any suggestions for future posts that you think I've not covered, or not covered sufficiently well. I'm aware that after building up CCR iterators, I rushed through them rather quickly.
More About: Work

Just wing it
2007-08-22 15:23:00
Courtesy of Kiran, an interesting presentation by Edward A. Lee on why shared memory is broken and what is needed instead to 'make concurrency mainstream'. [I couldn't get it to go under Firefox, but IE had no bother. You may get different results.]
More About: Wing

Newsqueak and Payback Time
2007-08-16 13:18:00
An interesting video presenting Newsqueak, another language with message-passing built in. And Joe Armstrong looks forward to a golden era for Erlang developers. Am I just looking harder or is message-passing gaining momentum nowadays?
More About: Time , Payback

I'm back
2007-08-16 01:49:00
Obviously. Blogger suspended my account as their anti-spam bot identified this content as spam. So much for heuristics. Or maybe not depending on your point of view.
More About: Back

CCR 101 - Part 9: Iterators
2007-08-11 16:42:00
Iterators (or Iterative Tasks in CCR parlance) are one of the standout features of the CCR. Their design and implementation is an imaginative fusion of the message-passing paradigm and the iterator functionality introduced in C# 2.0. But whether or not you choose to sit and admire the aesthetics of the CCR, you would probably agree that it is what they enable that is truly impressive - the ability to write a logical sequence of actions, within a single method, without ever blocking the thread between those actions. Now obviously the strength of this functionality really shines through when each action results in some form of asynchronous I/O, e.g. a web request or a database write. In this case, the thread is freed up to service other tasks whilst the I/O completes, before returning to the next action in the sequence. But remember, in our CCR-based message-passing world, pretty much all I/O is performed asynchronously even if those messages are passed between CPU-bound services...
More About: Part

More Concurrency Reading...
2007-08-09 16:02:00
I just haven't had the time this week to write the CCR Iterators post. A hard deadline at work means that spare capacity is at a premium right now, and likely to be in the near future. In the meantime, a few articles related to concurrency. Probably preaching to the converted... One from Herb Sutter. A bit out of date now, but not dated. Also, a follow-up. An interesting introduction to Software Transactional Memory by Simon Peyton-Jones.
More About: Reading

Patrick Dussud on CLR Garbage Collection
2007-08-05 16:19:00
An interesting video on Channel9 featuring Patrick Dussud, now chief architect (and one-time principal developer) of the .NET garbage collector. There are lots of smart cookies working in the software industry, many without much fanfare. Kudos to Microsoft (and Patrick) for putting themselves forward. The rest of us benefit from this kind of effort. Indirectly, I also re-acquainted myself with Maoni Stephens blog (also CLR GC). Well worth a look for CLR aficionados. To those readers who maybe waiting for the next instalment of CCR 101, I intend to fulfill my promise to cover Iterators this week, work commitments allowing.
More About: Garbage

My Introduction to Erlang
2007-07-30 15:45:00
I've just finished reading Joe Armstrong's PhD thesis entitled 'Making reliable distributed systems in the presence of software errors' (this is becoming a bit of a habit). Anyway, I found it very clear and well presented. Although I don't know much about Erlang or switching systems, what Joe has to say about distributed software architectures has much in common with Decentralized Software Services (DSS). To paraphrase Joe's summary of the characteristics of Concurrency-Oriented Programming: Services are isolated from one another. A fault in one should not adversely affect another. Each service has an identifier, and is addressable by it. There is no shared state between services. Services interact via asynchronous messaging, which must be assumed to be unreliable. Services are units of concurrency. They run independently of one another.
More About: Introduction

CCR 101 - Part 8: Dispatchers
2007-07-30 15:15:00
I've not covered the Dispatcher class before in any detail, and in all the examples so far, I've only used DispatcherQueues via their default constructor which causes all tasks to be scheduled over the CLR thread-pool. The CCR Dispatcher class provides a means of scheduling tasks over your own custom thread-pool. Being in control of the thread-pool creation gives you some control over the properties of the threads themselves (albeit as a homogenous block - you can't set these properties on individual threads). These include: The name of the threads. This is a useful diagnostic property. Whether the threads will run in the background. Background threads will not keep the application active, should all the other foreground threads exit. The priority the threads will run at. The COM apartment state of the threads. This is useful for interop. First though, let's adapt our first sample for use with a standard dispatcher:class Sample0801 { static void Main(string[]...
More About: Part

CCR 101 - Part 7: Interleave
2007-07-28 16:07:00
The Interleave is essentially an arbitration with reader-writer scheduling semantics. It does this by splitting the Receiver tasks it arbitrates into 3 categories: Concurrent receivers. Any receivers in this group can be scheduled concurrently with any other receivers from this group, providing no exclusive receivers (see below) are active or pending. Exclusive receivers. A receiver from this group will only be scheduled when there are no other exclusive or concurrent receivers active. Teardown receivers. A receiver in this group will also run exclusively but causes the Interleave to 'retire'. Once a receiver in the group has been scheduled, all other receivers are abandoned and will not be scheduled. Note that none of these categories need have any receivers at all. There are a few more interesting things about Interleaves: Exclusive receivers have their order retained. Remember how we saw that a persistent Receive (without an outer arbitration to control the scheduling) woul...
More About: Part

CCR 101 - Part 6: Arbitrations (Join)
2007-07-26 15:47:00
Actually, we've covered Join s already. I just forgot to mention it. The MultipleItemReceives we covered in Part 4 are joins, i.e. they wait for all the ports to produce n messages. All Joins (like their name suggests) are just rendezvous points for a set of messages and are an important construct - without them, we wouldn't easily be able to gather the results of a parallelized operation. Whilst the most useful arbiters are typically those that allow you to deal with a concurrency problem whose size isn't known until runtime, there exists a join across two ports that is explicitly provided for by the Arbiter class (not unlike Choice in fact). The following sample takes a number, n, and a string from the console and prints that string n times.class Sample0601 { static void Main(string[] args) { using (DispatcherQueue dq = new DispatcherQueue()) { Port<int> intPort = new Port<int>(); Port<string> stringPort = new Port<s...
More About: Arbitration

Web Buffoon
2007-07-26 14:34:00
I notice that this blog doesn't render properly on IE6. I have figured out a fix of sorts (something to do with the text in a fixed-length DIV overflowing) with some help, but need to make sure it doesn't break the experience for IE7 and Firefox users, before I apply it. As my CSS skills fall somewhere short of useless, this may take a couple of days. I've also noticed that the code is considerably less legible on Firefox than on IE. I'll try and fix that too. Thank you for your patience.

CCR 101 - Part 5: Arbitrations (Choice)
2007-07-25 08:15:00
Our first 'proper' arbitration is Choice . It takes one or more (normally at least two to be useful) Receiver tasks and schedules the first eligible one. The others will never be scheduled. An example will suffice:class Sample0501 { static void Main(string[] args) { using (DispatcherQueue dq = new DispatcherQueue()) { Port<int> intPort = new Port<int>(); Port<string> stringPort = new Port<string>(); Port<EmptyValue> emptyPort = new Port<EmptyValue>(); Arbiter.Activate(dq, Arbiter.Choice( Arbiter.Receive(false, intPort, delegate(int i) { Console.WriteLine("{0} is an int.", i.ToString()); }), Arbiter.Receive(false, stringPort, delegate(string s) { Console.WriteLine("{0} is a string.", s); }), Arbiter.Receive(false, emptyPort, delegate(EmptyValue e) { Console.Wr...
More About: Part , Arbitration

CCR 101 - Part 4: Arbitrations (Receivers)
2007-07-24 08:18:00
So far we've seen how to queue simple tasks to a thread-pool and how to post messages to ports. In this tutorial, I'll introduce arbitrations and an important type in the CCR library, the Arbiter class. In the previous tutorial, I demonstrated how to post and retrieve messages to ports. Most of the time however, we won't explicitly dequeue messages, rather we'll define the code that should run when a message arrives on a port. The CCR then takes care of dequeueing the item for us. Receivers Receivers are simplest from of message handler. In fact, they're so simple they aren't really an arbitration at all. They simply process messages as they get dequeued from the port. Let's look at example:class Sample0401 { static void Main(string[] args) { using (DispatcherQueue dq = new DispatcherQueue()) { Port<string> port = new Port<string>(); Arbiter.Activate(dq, Arbiter.Receive(false, port, delegate(string item) { ...
More About: Part , Arbitration

CCR 101 - Part 3: Ports
2007-07-21 15:49:00
In a message-passing system there must exist some mechanism over which to pass messages. In CCR, ports are that mechanism. They are essentially simple FIFO queues. For your programming convenience, they are declared as the generic type, Port<T>, where T defines the type that can travel on that port. The following sample puts 10 strings on a port and then dequeues them and writes them to the console:class Sample0301 { static void Main(string[] args) { Port<string> port = new Port<string>(); // Enqueue the items for (int i = 0; i < 10; ++i) port.Post(i.ToString()); // Dequeue the items string item; while (port.Test(out item)) Console.WriteLine(item); Console.ReadLine(); }}As illustrated, we use the Post method to enqueue items, and the Test method to dequeue them. Test will return false if the port is empty and set the output parameter to its default value.PortSetsPortSets are just aggreg...
More About: Part , Part 3 , Ports

CCR 101 - Part 1.
2007-07-18 15:55:00
[This post is intended to provide an introduction to the CCR, complementing other introductory materials online (links at the end)]. So we agree that multi-threading programming is hard right? Or, at least that it gets hard quickly. Sure, if you only have a single task you want to execute in parallel with your main code, you can just spin up a thread, or maybe even use the BackgroundWorker class. But the thing about multi-threading is that things start to get tricky quite quickly: Can I easily cancel this task? This suggests some inter-thread communication. It also suggests that there are points in the long-running task where I can check for this pending cancellation. How can I detect if the task fails? More inter-thread communication is required. I also need to worry about the thread termination in the failure class. Actually, how do I detect if the task completes at all? Can it timeout? What if I'm writing a server application? I can't really spawn a thread fo...
More About: Part

CCR
2007-07-17 15:50:00
Others have blogged about it, others have written technical pieces about it. The Concurrency and Coordination Runtime, which currently forms part of Microsoft Robotics Studio seeks to enable concurrent/parallel programming through the coordination of a lightweight message-passing system. The CCR (and its distributed big-brother, DSS) represents a shift in thinking for developers like me. Previously, I used to bring my not-inconsiderable (ahem) threading knowledge to a problem, consider the costs of introducing parallelism and weigh-up whether the increased algorithmic complexity was worth the potential runtime improvement. [Not sure where I read this, but someone once asked if debugging code is harder than writing code, and you're being as clever as you can be when you're writing code, what chance have you of debugging it?] Spend sometime with the CCR and maybe like me, you'll basically start to worry a lot less about the how and more on the why. And their are plenty o...

Introduction
2007-05-11 15:32:00
Hi, and welcome to my blog. My name is Nick Gunn and I'm a software developer with about 15 years experience, largely in the financial markets in some capacity or other. Currently, I'm working in the equity markets area, mostly on server-side execution systems. This blog will be largely technical but probably not too domain-specific.From a technical perspective, I come from a largely C++ wintel background, but have always had (until relatively recently) at least a toe in the Unix world. I've been involved in some Java/Oracle projects in the past but seem to be largely concentrating on .NET nowadays. I don't have any particular axe to grind technology-wise, but, because of where my attention is focused these days, the topics posted here will have a largely wintel/.NET flavour. Within that, it'll probably focus on areas of server-side scalability and design, concurrency and parallelism. I know something of these areas, but don't claim to be an expert. I'll link to those people...
More About: Introduction , Intro