It’s still a research project at Microsoft and it has recently released a preview version of the framework which is designed to work well with the asynchronous features of the latest .NET CTP.

The goal of the framework is similar to PLinq in the sense that you construct a pipeline composed of blocks expressing individual operation (action, transformation, join, fork, …) with the notable difference that the model is push-based instead of pull-based.

Push-based basically means that, in the same way than the Reactive Extensions, you explicitly feed data to your pipeline instead of acquiring it from a source. Then the final data can be received or processed by a block at the other end of the chain.

All of that makes a nice model for asynchronicity that you can use both to send or receive data thanks to a number of extensions method in the framework (suffixed by Async) which returns a Task thus letting you setup continuations or use the new await keyword (also available in Mono master btw).

Under the hood the framework is built upon what ParallelFx provides like the concurrent collections and the task primitive meaning it’s also taking full advantage of the available parallelism on your machine.

The code is now available directly from Mono master and is built in the System.Threading.Tasks.Dataflow assembly with the 4.0 profile. While not totally complete, it’s still already usable.

MSDN doesn’t yet show the documentation so I uploaded the copy shipped with the preview.

You can also watch Stephen Toub introductory video for a tour of the framework (also embedded below).

It took around 20 years for CPU manufacturers to abandon the GHz war on the desktop and start going multicore.

In the mobile space and only 3 years after the first Android smartphone, we are already seeing dual-cores phones (with quad-cores around the corner). Recent tablets are on the same boat.

The brilliant thing is that with Monodroid and Monotouch, we have the possibility to harness this new found processing power today using all the existing ParallelFx constructs since they are part of both distributions (i.e. Tasks, continuations, parallel loops, PLinq and friends).

To illustrate that, I ported the now classic raytracer code from the Parallel sample gallery to Android using Monodroid. Code is on GitHub if you want to give it a spin.

While the application itself is painfully slow since all rendering is in software, it does exhibit the same speed improvement when you ran in parallel mode as with the desktop version on my dual-core smartphone. For iDevices lovers, Geoff also had a similar version running on an iPad 2.

If you are at Monospace and want to talk about that stuff (and ParallelFx in general), don’t hesitate to come and say hi. Anyhow, I’ll continue to post more examples of how you can use ParallelFx practically in your mobile apps.

PS: as you’ll see in the source, there are some hacks bundled because of a couple of Mono bugs but they will get fixed eventually.

 _________________________________
< All your moos are belong to us  >
 ---------------------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||
 

Upon my latest Twitter question and getting no further answer, I carried out even more (useful) Manos world domination by writing Moo, an online cowsay generator (code on github under beer license).

Cowsay is one of those obscure Perl thing from the dark hacker age that once plagued mails and terminals.

It basically consists of a variety of ASCII animals (not limited to cows actually) telling some text inside bubbles. Useless therefore essential.

All of that offering me the shameless plug of telling the world I’m attending

See you there!

After team WebMatrix was formed this week, I have reimplemented the Data part of WebMatrix, a cute layer on top of ADO.NET with support for the dynamic keyword (C# 4) so that you can directly call property equivalent to your column names.

Bonus point is that even though is has been shipped with a heavy slant towards SQL Server (and its new compact counterpart) since it’s implemented on top of stock System.Data.Common we can use any provider with it including the great old-fashioned SQlite provider.

Say I have the following SQlite database:

That is populated with these values:

You can now write code like this to query the database:

using System;
using WebMatrix.Data;

class WebMatrixSample
{
	public static void Main ()
	{
		var db = Database.OpenConnectionString ("Data Source=sqlite.db;Version=3;", "Mono.Data.Sqlite");

		var result = db.Query ("select * from Phonebook where Number glob '0*'");
		foreach (dynamic row in result)
			Console.WriteLine ("({0:D2}) {1}: [{2}]", row.Id, row.Name, row.Number);

		var entry = db.QuerySingle ("select * from Phonebook where Name = @0", "Rupert");
		Console.WriteLine ("Rupert number is {0} with id {1}", entry.Number, entry.Id);

		Console.WriteLine ("but really his number is just {0}",
		                   db.QueryValue ("select Number from Phonebook where Name = @0", "Rupert"));

		// And let's add my bank number
		db.Execute ("insert into Phonebook(Name, Number) values (@0, @1)", "Bank", "01xxxxxxx");
	}
}

WebMatrix.Data assembly is now available when you compile Mono from Git master.

Once upon a time

As a lot of people I started doing programming during my HTML days when I needed to add a bit of interactivity with some layers of PHP. Realizing that what I was coding was both ugly and useless, I moved on to other stuff and distanced myself from web development altogether.

Meanwhile, the Web became much more feature-full than what it was at the time and when I took another dive these past weeks I grasped how it had grew since with HTML 5, CSS3 or jQuery (yeah I know I was out of the wheel for a long time) to the point it was actually enjoyable.

But what really motivated me to come back was the recent release by my esteemed colleague Jackson of his Manos web framework which, even though designed for cartels, was at long last providing a simple environment and simple interface to read and output valid HTTP headers in .NET/Mono.

The use case

Since I never really bothered signing-up for these Twitter image sharing website for the two or three pics I post per year and since I had no need of a special app for a badass smartphone I don’t have, I was leaning towards something FOSS and simple I could deploy on my own server which of course I didn’t find (if I was mistaken, replace that part with « I love reinventing the wheel »).

Thus Apachaï was born (on GitHub). It’s a KISS solution to a basic workflow: « let me post a pic quickly and write my twitter message at the same time ».

It does so by leveraging Twitter’s SSO using code borrowed from Miguel de Icaza‘s TrollStation. It also have some nice goodies like simple image effects courtesy of Pinta (yay for easy code reuse).

Data management is handled by Redis because it wouldn’t have been wicked enough without some NoSQL sparkles (nice thing though). Plus Redis is so lightweight that it is not a big dependency really.

Oh and because eating your own soup is good, Apachaï uses ParallelFx’s Future and continuations to process long-running operations without blocking Manos itself so it’s an example of using ParallelFx in a web context. I also made a choice of not using any specific templating system, MVC or whatnot. The HTML is always served as a basic empty container and all specific elements are dynamically filled with Javascript and some JSON serialization routes (see why Manos and futures fit?).

Apachaï screenshot

Finally

So to summarize:

• Apachaï is a small and lightweight photo and picture sharing application (for services like Twitter) built on the Manos framework
• A public instance for your usage is available at apch.fr
• Code is hosted on GitHub at https://github.com/garuma/apachai
• I will be giving a talk about Manos during upcoming FOSDEM as Ruben once again secured a Mono room.

PS: the name comes from this character because that was what I was reading at the time.

The past few weeks, I have given some cycles to test and ultimately make sure existing ParallelFx softwares from the .NET world could run on our Mono implementation.

In the end, I selected three pieces of code that were using ParallelFx extensively: the Parallel Extensions sample gallery, Microsoft Biology Foundation and RavenDB (triggered by Rob Ashton work).

As a result, I’m happy to say I was able to run most of the three of them and they were of a great help pinpointing some non obvious corner case of the API. Practical consequence is that to also run them you will need Mono from master.

Following is a bunch of notes specific to each of these projects:

Parallel Extensions sample gallery

This project actually also contains the parallel extras library which provides a bunch of new constructs and extensions to the classic ParallelFx API that you can reuse in your own project.

At the moment though, you should be aware that the license used doesn’t allow reusing the code on a platform different than Microsoft Windows but this will hopefully be changed in the near-future to use a wider free/open-source license.

In this gallery, I was only able to test the projects that didn’t specifically use non-supported library in Mono such as WPF. For the rest of them, using our xbuild tool does the trick of compiling the code.

If you happen to have a software that use one of those code path I hadn’t had a chance to test and if you can share the application with me I can also give it a spin.

Microsoft Biology Foundation

This project is a heavy user of almost everything in the framework ranging from concurrent collections to PLinq and is quite a hungry memory and CPU eater.

Since I couldn’t find a proper application leveraging this framework, I simply turned to the performance test suite bundled in the project which works fine although some test eats up quite a lot of memory (as always, running under sgen helps here).

The code actually requires a couple of extra tweaks if you want to run the performance test suite correctly but for compiling the core framework itself, again xbuild should be sufficient.

RavenDB

Last but not least, RavenDB is an engine part of the NoSQL and document database movement but with a slant towards .NET (with Linq integration for instance).

Rob Ashton decided to try to run the beast on Linux and I gave him a hand to tackle the few incompatibilities between Mono and .NET result of which can be found in his GitHub repository: https://github.com/robashton/ravendb/tree/mono

The test suite is running well enough with a total of 531 tests passing out of 588. Some of the failure are perfectly normal (like the tests using Microsoft-only native librairies) and another good portion is probably casual portability mistake (like hardcoded path and such).

It would actually be much appreciated if those knowing how the framework work could help triage these failures and find what is Mono bugs and what is not so that we can fix them.

Some tests also depend on not implemented features in Mono so if you feel like it, you could also contribute these missing code pieces or provide a workaround in RavenDB.

Following the informal one-month internship I did last summer, I will be back at Novell in lovely Dublin offices this time for a full-blown seven months session starting in August.

As you can expect, my main area of work will be ParallelFx and other 4.0 goodness but I hope to do some other interesting work on the side.

I very much look forward to this new opportunity of working full time on the ever awesome Mono and with its talented hacker team.

See you around a (Guinness) pint!

One of the cool buzzword these days beside parallel computing is GPGPU as in General Purpose GPU.

The base idea of GPGPU is to allow a programmer to tape in the parallel processing capacities of a GPU to do something different than pixel crushing (even though it mostly boils down to that very application most of time).

GPGPU isn’t new and for a long time vendors have provided their own rather low-level toolkit to program arbitrarily a GPU. As often, someone stepped up one day and proposed a unified API on top of all the vendor specificities. In our case, that API is a standard defined by the Khronos group (think OpenGL) and is named OpenCL.

OpenCL defines both a high-level C API to manipulate GPU (plus other friends) and a C-like intermediate language that is compiled and ultimately run on the device. It’s actually the same model as shader programming.

In the pursuit of Mono world-domination and following the implementation of PLinq that strived to be an easy way for programmer to access the multicore architecture of our processors via Linq, I hereby introduce GLinq (unoriginal acronym for GPU Linq) which this time use your graphic card to execute Linq queries via OpenCL.

GLinq is still much in its infancy and currently only support the Linq’s Select operator but I hope to be able to implement several other operators (at least the classic Where-Aggregate combo). The end goal is also to provide a new set of operator to allow OpenGL code to access the results of the computation without doing a roundtrip between CPU and GPU.

As I said previously, GLinq has the same design goal as PLinq which was to provide a totally new execution model for programmer in the most transparent way possible and with little modification to existing code. In our case, this prerequisite is rather tricky since you are juggling between two totally different worlds.

GLinq tries to provide that seamless experience with two features. The first one is the use of expression trees and C# compiler lambda magic to automatically capture the sense of a C# expression and rewrite it to OpenCL intermediate language. The second one is an automatic mapping of C# types and methods (e.g. Math class functions) to corresponding OpenCL symbols without however sacrificing OpenCL specificities which are also exposed through a C# API (not complete).

Of course, the GLinq API is still a transcription of standard Linq so you use it in the same way via the GpuEnumerable class. At the moment, the only input support are Range and Repeat but that will easily be changed to support any kind of IEnumerable.

Below is an example of the kind of query you can already execute:

var query = GpuEnumerable.Range (-20, 50)
			 .Select ((i) => i * -2 + 3)
			 .Select ((i) => Math.Abs (i))
			 .Select ((i) => ExtraMath.Ldexp ((float)i, 2f))
			 .Select ((i) => (int)i);

foreach (var i in query)
	Console.Write ("{0}, ", i.ToString ());

From what you can see, there are a lot of select operators here. It comes from the fact that the compiler is only able to convert a pure expression lambda to its corresponding expression tree. You can circumvent this limitation by building yourself the expression tree or by using any compiler/interpreter that outputs expression trees (e.g. IronPython, IronRuby, Bechamel).

On a final note, there is an acknowledged bug in at least NVidia implementation of OpenCL which makes Mono doesn’t play nice with it. This is due to the fact that (most) OpenCL implementations use LLVM for code generation/optimization which in turns mess with Mono internal working.

This problem is going to be fixed in the next NVidia driver iteration but as a work-around you can apply this simple patch to mono. I put SIGWINCHSIGXFSZ there because it looks cool and seems generally unused but any other should do the trick.

Code in its proof-of-concept form is living here: http://git.neteril.org/glinq/

It's all about splitting work

At long last, PLinq has finally found its way into Mono trunk as what can be considered a preview release. The code has seen a major rewrite compared to the initial implementation done during earlier Google Summer of Code which lacked flexibility and correctness.

Public API is complete except some operator still not implemented. In addition and as implied by the preview title, the current code hasn’t been tested against all possible flavor of query so it might or might not work for you (i.e. deadlock). If that’s the case, don’t hesitate to fill a bug report with a query reproducing the problem. The more query I have, the more robust we can shape future release.

By the way, these PLinq performance tips still hold true.

The Ubuntu-ie folks are running a slightly tuned Ubuntu Global Jam here in Dublin on Saturday 27th March where basically everyone is welcome to drop by and slackwork on any project he wants.

The event has been dubbed OpenJam and every Mono hacker/enthusiast/bystander is most welcome to register on the Mono Corner of the Signup page.

More informations and day program is available on Laura’s blog.