Jelle Druyts

Keynote 1: Ready For Business (Andrew Lees)

The official keynote was a combination of a "show" and a talk, with Corporate VP Andrew Lees being regularly swapped out for a couple of (what I presume to be) actors in the typical Developer/IT Pro/User roles. I'm sure we've all been there as one of these roles, just not getting anything done from the other ones; but it's all about team work, and the speach was mostly about how the Microsoft products will help these teams work together. Along with a number of videos and flashy demos (not built in Avalon though, which is a shame) this was quite an entertaining keynote, but nothing shockingly new of course.

There was one pretty cool demo about WS-Management (a new systems management protocol) that showed MOM sending management queries to a non-bootable machine (I'm still puzzled by which stack or hardware component). And since it's an open specification, it even worked on a Sun Solaris which they demoed live on stage. Nice!

They also mentioned that Microsoft Learning is currently offering free skills assessments and free trainings on Visual Studio 2005 and SQL Server 2005. Really nice!

ARC302 - Building And Using A Software Factory (Steve Cook, Annie Matthewman)

While the presentation was pretty "dry", I got quite some thinking out of the whole Software Factory concept. I think of Software Factories as the two-steps-up evolution of the Enterprise Templates you might remember from a few years ago when Visual Studio .NET first came out. They're basically an abstraction from recurring enterprise-specific patterns, implemented as Domain Specific Languages (DSL's). In a basic sense, this boils down to project and code generation inside Visual Studio 2005, along with graphical editors and designers that drive this generation process. It can be as simple as providing an editor for the UIP Application Block's configuration file (which they demoed), to an entire code generation of your business layer based on a service description.

They used the Guidance Automation Toolkit (GAT) to build their DSL editor, so it's good to think of this as a "Factory for Factories". This is a tool that will mostly be used by architects defining and implementing DSL's for an enterprise scenario. At first sight, creating a DSL using the GAT seemed doable for one-way code generation (I also took a Hands-On-Lab on it), but I wonder how hard it would be to implement it in a two-way synchronized mannor. Nonetheless, impressive stuff, but it will take some significant effort to implement this into your current software development strategy.

ARC309 - Microsoft Visual Studio 2005 Team Edition for Software Architects: Developing Service-Oriented Systems (Eric Lee)

This was an excellent overview on the Team System features for architects, mainly about the four Distributed System Designers that will help you create executable as well as deployable designs: the Application, System, Logical Datacenter and Deployment Designer. There isn't really a lot more to say about it other than the fact that Eric Lee is also a very good, enthousiastic and knowledgeable speaker. Great talk!

CTS365 - Implementing "Indigo" Endpoints: Addresses, Bindings and Contracts (Christian Weyer, Steve Swartz)

Steve Swartz did a nice job of explaining Indigo endpoints. He outlined the three types of contracts: the data contract (schema and versioning), the message contract (SOAP headers), and the service contract (behaviour).

The way to version a data contract is stunningly easy in Indigo: just add the VersionAdded, MustUnderstand and IsOptional attributes to your data member and you're set to go:

[DataMember(VersionAdded=2, MustUnderstand=false, IsOptional=true]
public string SomeNewField

The Indigo runtime will now have two-way support for this member, i.e. the new type will still work in the old service, and the old type will still work in the new service.

The proposed way to version a service contract was to inherit your new ServiceContract interface from the old one:

[ServiceContract]
public interface ICalculator2 : ICalculator

I'm not sure if this is really the way to go (it feels too much like MSXML, MSXML2, MSXML3, ...) but I'll give it some more thought before I make up my mind about this.

Another point he made was that Exceptions are problems in code, whereas Faults are error responses from the server. Since these Faults are also a WSDL standard, there's explicit support for them in Indigo through the Fault class:

DivideByZeroFault f = new DivideByZeroFault("Cannot divide by zero");
throw new Fault<DivideByZeroFault>(f);

There was also a cool demo showing the fact that you can return a Stream from a service operation, allowing infinite streaming data to be passed back (e.g. audio broadcasting, or webcam streams).

Finally, on the behaviour part of Indigo services, developers tend to care mostly about the concurrency and instancing behaviours (can recurrant calls be made, is it a singleton or singlecall service, is session state available, ...), whereas deployers care more about throttling and metadata behaviours (how many concurrent calls and instances can I have, is the WSDL available online, is WS-MetadataExchange supported, ...). All these types of behaviours can be defined by simple attributes or configuration settings.

All in all, a very nice talk showing some advanced capabilities of Indigo.

CHT072 - Drilldown into Visual Studio 2005 Team System (Eric Lee)

This interactive Chalk & Talk session gave a good demonstration about Team System, with Eric Lee showing how to create Team Projects and work in them, while answering questions from the attendees. Again, this guy seems to know pretty well what he's talking about, so it was a nice wrap-up for the day.

(Apart from the fact that there were free drinks in the community hall afterwards, but I'd argue that the Dutch beer doesn't really count as free drinks )

Ok that's it, everybody stop talking smack about the DataSet. As self-elected chairman of the inexistent "DataSet Preservation Fund", I feel urged to respond to years of incorrect positioning, biased thinking and over-simplified black-and-white views. I won't allow the "enterprise community" to make the DataSet the Clippy of .NET, so to speak.

All joking aside, I've long been putting off writing about this topic since I wanted to make sure my opinion was balanced enough but David Boschmans pointed me at a recent MSDN article on Custom Entity Classes by Karl Seguin which strikes me as pretty typical so here's my two cents on the matter. I'll take the article as my guide to inject some of my opinions, so it should be helpful to have a quick scan of the article to see what it's all about. By the way, why this article appears in the ASP.NET Developer Center and not in an Enterprise Development Center or the like (think PAG) is unclear to me, since this kind of discussion should target a broader audience.

Introduction

First and foremost, as the disclaimer in the introduction states: the article only talks about untyped DataSets, not about strongly-typed DataSets. This is a very unfortunate limitation by the author in my opinion, since Typed DataSets do indeed solve most problems mentioned in the article, as well as provide extra benefits, and this type of simplification will only give more traction to the anti-DataSet camp.

Alleged Problems

• Lack Of Abstraction.
  The statement is made (a few times throughout the article actually) that a DataSet makes it impossible to decouple your code from the database structure. While true that you still deal with logical tables, columns and relationships, a Typed DataSet will make this much less visible because you can navigate all of these by just using strongly-typed properties and methods. The argument that changing a column name in the database schema would result in adapting your client code doesn't make sense, you can (and, for this reason, always should) use "AS" in your SQL query to abstract away the actual column name, so it won't "trickle down" to the calling layers. So if the UserId column would actually change to Id for example, the SQL statement would simply become:
  SELECT Id AS UserId, FirstName AS FirstName, LastName AS LastName FROM Users
• Weakly-typed.
  The obvious statement is made that regular DataSets are untyped, the issues mentioned by the author are of course 100% correct. Hence the invention of Typed DataSets. They will eliminate all the raised concerns. This (along with the next paragraph) also renders the "Why Are They Beneficial" section superfluous.
• Not object-oriented.
  Sure enough, if you create a Typed DataSet, you get a class that inherits from System.DataSet, which means you can't make it derive from another class anymore. But does this make it "not object-oriented"? This type of problem is often referred to in the Java world as not being a "Plain Old Java Object" (POJO), so in this case I'll refer to it as a "Plain Old .NET Object" (PONO). The problem seems big enough to have Indigo address this by not requiring your components to inherit from any base class (as is the case with Enterprise Services COM+ components, which need to derive from ServicedComponent). If this is not a serious issue for you (it's never been for me), nothing prohibits you from just subclassing that generated DataSet class to add functionality to it without resorting to external utility methods. In the upcoming 2.0 version of .NET, it will even be possible to add functionality to the class without the need for subclassing it by using the partial classes feature.
  The Scott Hanselman quote, taken from his rant against DataSets on service boundaries, makes a good point though: "DataSets are bowls, not fruit". I'm actually cool with Typed DataSets being "bowls with a picture of fruit on them", they're data containers (data transfer objects) after all. Custom entities aren't fruit either, they're another binary representation of the same concept, but they still don't smell like fruit.
• NULLs.
  A point is made that "dealing with NULLs in DataSets isn't the easiest thing, because every time you pull a value you need to check if it's NULL". While I certainly agree that NULLs can be a pain, you could partially deal with this through annotations on your Typed DataSet, which allow you to define the value returned if the field actually contains a NULL value. Besides, a field having a functional value or not (i.e. being NULL or not) is something that affects all layers, and how you handle this in code will not be very different when using DataSets or custom classes. This is certainly true for value types (while we wait for nullable types anyway, gimme int? now!).

Other points that make me go "hmmm"...

• "Never return a class from the System.Data or child namespace from the DAL". If you are convinced that System.Data is the namespace for all server-side database stuff, then you're right. But I don't think that's the case, the provider child namespaces (e.g. System.Data.SqlClient) are server-side. I would allow System.Data.DataSet to pass through customs without a problem here.
• On performance: "whatever processing time you are able to save probably doesn't amount to much compared to the difference in maintainability". This is really comparing apples and pears, as runtime and designtime have very different characteristics and requirements in a project. It certainly is true that a "blanket statement" is pretty useless here, but I would like to say that although the performance hit and XML overhead on the wire of a DataSet might be larger in absolute figures, you should be communicating with the service backend through chunky calls anyway and the main performance bottleneck will lie beyond the service facade most of the time anyway. But the best performance tweaks can only be done through one optimal path: measuring and interpreting live data.
• "You can certainly assign nothing/null to [value types], but this will assign a default value. If you just declare an integer, or assign nothing/null to it, the variable will actually hold the value 0." Excuse me? Have you actually tried to run the "int i = null" statement through a C# compiler? This largely diminishes my trust in the author's, eh, authority on the subject of object oriented programming.

Free Code

So, what do you get with Typed DataSets? Free code mostly.

• You can use the boilerplate DataTable.Select method for searching through a table (which is illustrated as "custom behaviour" for the custom entity classes - i.e. code you need to type or generate).
• The remark that this Select call isn't strongly typed is again correct, but if you define a key on the UserId field in your Typed DataSet for example, you will also get a free typed method called FindByUserId which will select rows based on that key.
• In ADO.NET 2.0, you'll get the data column names as strongly typed properties so you can use these in your code. The databinding example in the article actually isn't strongly typed itself: the call to DataBinder.Eval(Container.DataItem, "UserName") contains the column name as a plain string, so it would benefit from these properties here as well.
• If you model relationships between tables, you get free methods and properties to navigate between parent and child tables. You can even use databinding expressions to navigate through these relationships.
• As stated by the author, "the DataView's built-in support for sorting and filtering, although requiring knowledge of both SQL and the underlying data structure, is a convenience that is somewhat lost with custom collections". So this is basically free code again, and as an extra remark: of course you need to know the data structure, how would you build an application if you knew nothing at all about your data structures? The main point, again, is that you're not supposed to know the database schema here, just the logical data structure. And regarding sorting: if you call "UserName DESC" (which you can use as a sorting expression) knowledge of SQL and think that's too hard to grasp, then you can't really expect much from a developer anyway. There needs to be some way of pouring that sort expression into a language construct. Indeed, it would be better to add strongly typed methods to sort with a sort flag enumeration (Ascending/Descending) and a column name, but it's really not that bad, is it?
• Data binding is far better supported with DataSets than with any other mechanism in the .NET Framework. Developer productivity is indeed a project feature.

DataSet Issues

So it can't all be good, right? There are indeed some issues with the DataSet and I can only hope they are fixed in upcoming versions of the framework. But the main point I'm trying to convey here is that these don't suffice to radically kill DataSets or tell everyone they're all Bad and Evil. Some real problems:

• They behave bad on the wire when passing over a Web Service or otherwise serialized as XML. This is indeed a problem and a very good reason to avoid them, if (and only if) you need to interoperate with platforms other than the .NET runtime. Don't forget, a lot of times you have both ends under control so this is actually not always an issue.
• They have a bloated wire format, since the XML representation can become very large very fast. If this is really a problem in your case and causes a measurable performance hit, you could opt for gzip compression over the HTTP transport protocol, or rethink the way you are using the services. The more or less "fixed" overhead will also become smaller in relative figures if the communication with the backend service is chunky in stead of chatty, so that's certainly an important point of attention. And you can consider using the GetChanges method to only include a diffgram of the changes since the DataSet was initially populated - but beware that this is incredibly platform-dependant though.

Good Points

Fortunately, the author makes some other very valid points as well.

• "I realize that code generation sounds like something of a dream. But when properly used and understood, it truly is a powerful arsenal in your bag of tools - even if you aren't doing custom entities". This is very, very true.
• "Making the switch to custom entities and collections shouldn't be a decision you make lightly". This is even more true. It has a major impact on the way your applications and services will be developed.
• "Jimmy Nilsson has an overview of some of these alternatives in his 5 part series Choosing Data Containers for .NET (part 1, 2, 3, 4, and 5)". These articles are quite good, indeed. Recommended reading!

Conclusion

I think part of the real discussion here is: do we go for a full Object Oriented domain model (most likely with some kind of Object/Relational mapping tool that handles lazy loading, automatic persistence, etc.) or do we build a (Typed) DataSet model where you create different ad-hoc DataSets according to each separate use case (which means there is no unified "one fits all" model). I personally think the full OO model is very hard to maintain in a large corporate domain with many projects and dependencies. Combine this with Service Orientation where it's much harder to "deploy" your domain model to the outside world (since you can only share contract and certainly not type) and I tend to think the latter is more flexible. Either that or you'll have to re-map the service boundary messages to your domain model internally, which causes even more work (although this might be a good approach if that fits your existing internal OO model).

To conclude, I believe in one thing: choose the right tool for the job. Given two viable options, choose the option that would solve the problem best under the constraints at hand. As Andres Hejlsberg often puts it when comparing performance between C++ and C#: given infinite time and resources, you'll do better in C++. Given infinite time and budget, I might agree that custom entity classes are the best possible solution. Unfortunately, that's not very realistic in real-world projects. Developer productivity and shipping should be considered two very important "features" of your project, and if DataSets help you accomplish these, then I would certainly advise (or at least consider) using them.

Yes! I just got my first Indigo sample up and running!

I started off with a clean Windows XP SP2 installation, and I just installed the March CTP of Avalon/Indigo/WinFX (build 2.0.50110) without anything else - no Visual Studio .NET, no IIS, no nothing - so it's definitely back to basics here while I'm downloading the February CTP of Visual Studio .NET 2005... On the other hand, it's quite a lot of fun to just try and get anything working without any major productivity booster such as Visual Studio, IntelliSense or any form of actual documentation installed. For now, I'll consider notepad and ildasm.exe my best friends in the Indigo space. Long time no see, welcome back guys

The first thing to try was to get any of the prepackaged samples working. I unzipped the AllSamples file in the "C:\Program Files\Microsoft SDKs\WinFX\samples" directory and seeing I had no Visual Studio installed I just tried to run MSBuild on the solution files. Although I think this should work, it was complaining to me about not being able to copy some files or something. Weird.

I then decided to take Clemens Vasters' Indigo Hello World sample and see what I could make of that. I just copied it into a new file and ran the compiler on it:

csc /r:"C:\Program Files\Microsoft Indigo Preview\System.ServiceModel.dll" IndigoHelloWorld.cs

Unfortunately, I got an error straight away: the BasicProfileHttpBinding class wasn't found. So here's ildasm to the rescue already. As it turns out, they recently must have renamed the class to BasicProfileBinding, so a simple change to the sourcefile fixes the problem and we're all set and compiled. Step 1 complete! The server code is now just:

using System;
using System.ServiceModel;

namespace IndiHello
{
  [ServiceContract]
  public class Hello
  {
    [OperationContract]
    public string SayHello(string name)
    {
      return "Hello " + name;
    }
  }

  class Program
  {
    static void Main(string[] args)
    {
      ServiceHost<Hello> host = new ServiceHost<Hello>(new Uri("http://localhost/hello"));
      host.AddEndpoint(typeof(Hello), new BasicProfileBinding(), "ep");
      host.Open();
      Console.WriteLine("Press ENTER to quit");
      Console.ReadLine();
      host.Close();
    }
  }
}

So with my newborn Indigo service now up and running on my machine, I figured I should be able to make this do something, right? So because I couldn't really think of anything else, I just took the ASMX approach, which is to surf to the endpoint (http://localhost/hello/ep) and observe the magic: Indigo provides a very similar model to ASMX with an informative html page if you access the endpoint via HTTP/GET and is so kind as to give a hint what you could do next. Not only does it tell you how to generate a client proxy for the service, but it also gives some basic info about how to use the proxy afterwards.

So off I am building a client proxy using svcutil.exe (think wsdl.exe for Indigo) on the wsdl:

"C:\Program Files\Microsoft Indigo Preview\svcutil.exe" http://localhost/hello/ep?wsdl

This just creates a proxy class in a file named tempuri.org.cs (since I haven't changed the default namespace, it just takes the tempuri.org namespace as the base name) with the contract interface and some kind of channel interface (I'm not sure what that does yet but I'll find out sooner or later):

[System.ServiceModel.ServiceContractAttribute()]
public interface Hello
{
  [System.ServiceModel.OperationContractAttribute(Action=http://tempuri.org/Hello/SayHello, ReplyAction="http://tempuri.org/Hello/SayHelloResponse")]
  [return: System.ServiceModel.MessageBodyAttribute(Name="SayHelloResult", Namespace="http://tempuri.org/")]
  string SayHello([System.ServiceModel.MessageBodyAttribute(Namespace="http://tempuri.org/")] string name);
}

public interface HelloChannel : Hello, System.ServiceModel.IProxyChannel
{
}

public partial class HelloProxy : System.ServiceModel.ProxyBase<Hello>, Hello
{
  public HelloProxy()
  {
  }

  public HelloProxy(string configurationName) : 
    base(configurationName)
  {
  }

  public HelloProxy(System.ServiceModel.Binding binding) : 
    base(binding)
  {
  }

  public HelloProxy(System.ServiceModel.EndpointAddress address, System.ServiceModel.Binding binding) : 
    base(address, binding)
  {
  }

  public string SayHello(string name)
  {
    return base.InnerProxy.SayHello(name);
  }
}

Looks good. Next up: adding a test driver. Simple enough, the service's informative html page shows me how to create a simple console app that uses the proxy:

HelloProxy proxy = new HelloProxy();
string result = proxy.Hello("some data");
System.Console.WriteLine("The Hello service returned '" + result + "'");
proxy.Close();

So I just whipped that into the generated proxy class file, and tried to compile the bunch:

csc /out:Client.exe /r:"C:\Program Files\Microsoft Indigo Preview\System.ServiceModel.dll" tempuri.org.cs

Too bad, there seems to be a little bug in the way the service generates the html page (I bet you didn't spot that just now, neither did I), since the service operation isn't called "Hello", it's "SayHello". So a small change here as well, run the compiler again - which seems to give me an actual executable this time - and we're ready to try and run the client:

C:\IndigoTest>Client.exe
Unhandled Exception: System.InvalidOperationException: Could not find Channel element for contract Hello, Client, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null
   at System.ServiceModel.Design.ConfigLoader.LoadChannelDescription(ChannelDescription channelDescription, ContractDescription contract, String configurationName, EndpointAddress address)
   at System.ServiceModel.Design.ChannelLoader..ctor(Type contractType, String configurationName, EndpointAddress address)
   at System.ServiceModel.ChannelFactory`1..ctor(String configurationName)
   at System.ServiceModel.ProxyBase`1..ctor()
   at Test.Main(String[] args)

Whoops, that's not good. Here's that Channel thing again. But then again, I didn't really provide an actual uri where the client could locate the service. So I decide to use one of the generated HelloProxy constructors that seems to make sense: the one with an EndpointAddress and Binding.

A quick look in Ildasm at the EndpointAddress class reveals a constructor that takes a uri string, so I'll just take that and then paste in the same binding as I used on the server side:

HelloProxy proxy = new HelloProxy(
    new System.ServiceModel.EndpointAddress( "http://localhost/hello/ep" ),
    new System.ServiceModel.BasicProfileBinding() );
string result = proxy.SayHello("some data");
System.Console.WriteLine("The Hello service returned '" + result + "'");
proxy.Close();

Throwing the compiler at the code creates me a new executable, so I run that and voila:

C:\IndigoTest>Client.exe
The Hello service returned 'Hello some data'

Wicked! My first Indigo client is talking to my first Indigo service over a basic profile HTTP binding, without even having IIS installed. Ain't that something...

Here are the files if you want to try it out yourself: IndigoHelloWorld.cs (service), tempuri.org.cs (client)

In retrospect, I could also have used the Indigo CTP SDK docs Don Box pointed at, but, well, it just didn't come to my mind. I was so happy to see ildasm again, I guess I didn't want to trade our new-found relationship for some long-distance relationship with a remote webserver

Now that I have this basic stuff out of the way, I can start digging in the articles, posts and documentation for Indigo to get a better hands-on view of the API.

There seems to be a bug in the .NET Soap serialization stack when passing a DataSet with an expression column in it that uses a relation. Allow me to elaborate But don't run off just yet if you don't care about the serialization stack, there's some useful stuff in there as well

Imagine a typed DataSet "TestDataSet" that you're passing from your Service layer to your client UI layer through an ASP.NET Web Service. Say you have a table Employee and a table Company in it, and you want to show the employees in a DataGrid (e.g. WinForms but that doesn't really matter here) with the name of the company in a separate column. The service backend doesn't care about your needs to display the company name inline with the rest of the data, so all you have is the relation CompanyEmployee between the two tables.

For display purposes, you can add a column to the Employee table with its Expression set to "Parent(CompanyEmployee).Name" and it will display just fine: it calculates the new column by navigating to the parent row through the CompanyEmployee relationship and then takes the Name column from that row. Bingo!

TestDataSet testData = TestServiceAgent.GetTestData();
testData.Employee.Columns["ID"].ColumnMapping = MappingType.Hidden;
testData.Employee.Columns["CompanyID"].ColumnMapping = MappingType.Hidden;
testData.Employee.Columns.Add( "CompanyName", typeof( string ), "Parent(CompanyEmployee).Name" );
employeeGrid.DataSource = testData.Employee;

Extra advantages: this value is automatically updated if the parent changes and since it's a calculated column, you can't edit the company name in the grid. Also note that you can hide columns by setting their mapping type to Hidden. That's just a simple trick you can use on the client side.

The problem arises when you use this modified DataSet to update your entity in the backend again: you probably have some SaveTestData method on your service that accepts a TestDataSet to be saved and you just want to pass the changes of the modified DataSet along to it.

TestServiceAgent.SaveTestData( testData.GetChanges() );

Too bad, it will blow up in your face:

Exception: System.Web.Services.Protocols.SoapException: Server was unable to read request. --->
System.InvalidOperationException: There is an error in XML document (1, 12311). --->
System.NullReferenceException: Object reference not set to an instance of an object.
   at System.Data.LookupNode.Bind(DataTable table, ArrayList list)
   at System.Data.DataExpression.Bind(DataTable table)
   at System.Data.DataExpression..ctor(String expression, DataTable table, Type type)
   at System.Data.DataColumn.set_Expression(String value)
   at System.Data.Merger.MergeSchema(DataTable table)
   at System.Data.Merger.MergeTableData(DataTable src)
   at System.Data.Merger.MergeDataSet(DataSet source)
   at System.Data.DataSet.Merge(DataSet dataSet, Boolean preserveChanges, MissingSchemaAction missingSchemaAction)
   at TestProject.TestDataSet.ReadXmlSerializable(XmlReader reader) in C:\TestProject\TestDataSet.cs:line 166
   at System.Data.DataSet.System.Xml.Serialization.IXmlSerializable.ReadXml(XmlReader reader)
   at System.Xml.Serialization.XmlSerializationReader.ReadSerializable(IXmlSerializable serializable)
   at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReader1.Read14_SaveTestData()
   --- End of inner exception stack trace ---
   at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle)
   at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader)
   at System.Web.Services.Protocols.SoapServerProtocol.ReadParameters()
   --- End of inner exception stack trace ---
   at System.Web.Services.Protocols.SoapServerProtocol.ReadParameters()
   at System.Web.Services.Protocols.WebServiceHandler.Invoke()
   at System.Web.Services.Protocols.WebServiceHandler.CoreProcessRequest()
   at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message,
        WebResponse response, Stream responseStream, Boolean asyncCall)
   at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
   at TestProject.TestServiceProxy.SaveTestData(TestDataSet testData) in c:\testproject\testserviceagent.cs:line 291
   at TestProject.TestServiceAgent.SaveTestData(TestDataSet testData) in c:\testproject\testserviceagent.cs:line 162

As you can see, the Soap/XML deserialization stack has some problems with the expression column. And this isn't just because the parent row could be missing in case you only sent the changed rows (which you should do using the GetChanges() method on the DataSet to get the diffgram): it happens even if you send the full DataSet back. If there's an expression column that doesn't use a relation, then there's no problem.

There's a post on .NET 247 called "Serialize DataSet with ExpressionColumn" that shows somebody else running into this issue, and if you kept your foreign languages polished you can also check it out on the Russian GotDotNet site. But that's all I found on it so far, and none offer an explanation or solution.

A possible workaround would be to remove all expression columns again before submitting the DataSet back to the Service layer, but that's pretty harsh, not too transparent, and it can mess up your UI if you're still using the DataSet on-screen.

What I consider to be much better, is to create a new instance of the typed DataSet and merge the changes of the DataSet you're working with into it - making sure you ignore anything not in the DataSet's schema by using MissingSchemaAction.Ignore. This removes all the added clutter you don't need on the backend anyway and it makes sure you get a clean and valid copy of your typed DataSet out of the door.

TestDataSet changes = new TestDataSet();
changes.Merge( testData.GetChanges(), true, MissingSchemaAction.Ignore );
TestServiceAgent.SaveTestData( changes );

This way, everything works and the universe is happy again.

And if you're worried about the cost of the extra DataSet you just created: it's on the client side so I'd consider the performance hit negligible as long as you don't constantly send this type of changes to the backend. Then again, if you are constantly doing this, then there's probably something wrong with the way you're using your service

Today was frustration day. I got all cooked up a few times and got mad at

• my next-door neighbour (for playing loud music at 2:30 AM)
• some printer (for paperjamming 66% of the time)
• Visio (for generally sucking; well either that or I'm just too much of an idiot to understand how to properly draw a UML sequence diagram in there)
• aspnet_wp (for not finding some system dll's in its own runtime cache until I nuke the process)
• SourceSafe (see Visio; and for the reason below)
• Visual Studio .NET 2003 (for pestering me with the "Object type cannot be converted to target Type" CopyLocal bug again)

Luckily, I also picked up what could be the trick to avoid an all-too-common SourceSafe mess.

If you're getting a new solution from SourceSafe that includes an ASP.NET Web Site or Web Service you probably encountered this as well: chances are that you don't keep your sources under IIS's wwwroot but under your local SourceSafe working directory and then create VRoots (Virtual Roots) in IIS to the proper directory. If you did that before you opened the solution in Visual Studio .NET for the first time, you're doomed with a feeble attempt of SourceSafe to create the web for you, again. It notices that there's already a web with that name, creates a new one with the same name and a "_1" suffix and then complains that it can't bind to the proper web site. Here's a trick that worked for us:

• Create the directory but keep it empty (don't get a local copy out of SourceSafe)
• Create the VRoot to that directory with the proper name
• Make sure Anonymous Access is enabled on the VRoot
• Open the solution in Visual Studio .NET, SourceSafe should be getting the files in the right directory now
• Disable Anonymous Access on the VRoot if required

If you have other or better solutions, don't hold back and share them I've seen a trick via David to avoid some other problems by making SourceSafe treat a web project as a class library, but I think the friction is pretty high on that one.

And I'll still be holding my breath for the source control system included with Visual Studio 2005 Team System of course. That, and the fact that Visual Studio .NET 2005 doesn't need IIS anymore, or creates web sites under wwwroot by default. Shoosh, now be gone SourceSafe.

As it happens, I'll be co-hosting a TechNet Evening session on "Customizing Collaborative Solutions with InfoPath 2003 SP1 and Windows SharePoint Services" with Yves Kerwyn on October 27. Woohoo, that should be fun So if you're interested in InfoPath, XML, Web Services and SharePoint, be sure to register for the session and come say hi!

There should be lots of demos and I fully intend to make sure Murphy's Law is upheld... So there's a big chance you get to point and laugh at me in public - don't miss this unique opportunity!

I finally took the time last week to take my third Microsoft exam: "Developing XML Web Services and Server Components with Microsoft Visual C# and the Microsoft .NET Framework" (#70-320). I'm glad to see they're giving you your score afterwards again (I never knew how much I got for the WinApp and WebApp exams) because I came out with a score of 952/1000

So that makes me a Microsoft Certified Application Developer; two more exams to go to become a Microsoft Certified Solution Developer. After I achieve that, I fully expect to become a NZCSH (New Zealand Certified Sheep Herder), a required step towards becoming the GCUU (Globally Certified Uomo Universale) - a title last given to Leonardo da Vinci but I've had my eye on it for a while now.

By the way, there's a new elective security exam you can take to become MCSD.NET: "Implementing Security for Applications with Microsoft Visual (Whatever) .NET" (#70-330 or #70-340), with an accompanying new course "Implementing Security for Applications" (#2840) which again shows the big security push Microsoft is committed to. Goodness!

InfoPath 2003 is a pretty cool product with lots of potential, but it's obvious that it's still a little young for the real world. Here are some issues I found when developing with InfoPath out-of-the-box:

• You can't make fields required easily if the datasource is a (readonly) webservice since you can't change that definition and you can't set the validation to "cannot be blank" anymore. Apparently, this is by design: "Notice that the Cannot be blank option is disabled; this property is always disabled in the InfoPath user interface when your data source is an external data source". Why is that?
• Publishing is a pain; I just wanted to publish to a website on my local machine and after a lot of head-scratching I found that you actually have to deploy to a network share. The first thing you do is set the directory (the virtual directory on your drive) and then the web url from which the form will be accessible.
• Copying the form to another location - e.g. from develepment to acceptation or production - can't be done easily since the originating location is baked in so you have to redeploy it. Alternatively you could try to script the process to extract the xsn file (which is really just a cab file), update the location and repack it.
• You can't prefill a form with data from a WebService easily, if you don't want the user to click a button first then you'll have to resort to scripting. This looks like a very common scenario though, so I'd expect this to be a lot easier.
• Data binding the controls to the fields can be tedious, a drag & drop mode would be easier.
• There's no support for WSE so I can't use WS-Security; another option would be to use custom SoapHeaders but there's also no support for that.
• There's no password control, so I can't prompt the user for his credentials without having him expose his password to potential neckbreathers.
• It would be nice if InfoPath worked a bit more like ClickOnce: click an xsn in your browser and have the the form open immediately in stead of getting the standard Open/Save dialog. Of course, it's still a cab file so it could be harmful to just allow this without the ClickOnce wonders of .NET security - but it would provide a much more transparant user experience.
• Copy/paste doesn't work between instances of InfoPath, not even for plain text. So it's pretty hard to copy part of an existing form to another one.

Note that this is based on a standard InfoPath 2003 installation, I haven't tried the new InfoPath SP1 Preview yet (which might solve some issues but certainly not all of them).

It's been quite a while since the last progress report on WeFly247.NET, but we've been very busy getting a more or less stable build out for the rest of the team to review - luckily that also means a lot of work has been done in the mean time

Last time, I left off with a "standalone" Passenger web site that looked like it was designed by a developer (well, it was ), but since then I've updated the look and feel to match the great designs from Elevator Digital and connected it all to the business tier. The website is pretty much done right now, it's all talking to the backend over Web Services through smart Agents that handle security and custom authorization Soap headers.

The home page:

The "panels" (flight info, destination info, crew info, ...) on the homepage are Web Parts so you can customize the whole lot and look only at what you would want to see during your flight. The Flight Map on the homepage is a dynamic image (easy as pie with the new .asix handler and the DynamicImage control in ASP.NET 2.0) which displays the the route the plane has already traveled and where the plane is at now.

There's also an InfoPath form that allows you to fill in your Customs & Immigration information; it also pulls and pushes data from and to the backend through Web Services (but since InfoPath doesn't talk Soap headers yet, it's not automatically secured yet).

The Duty Free Shop:

So with that part pretty much done, I've started working on the Maintenance application, which is a Smart Client application for the pilots that can be used on a Tablet PC. They use it to go over a checklist before takeoff (there are no gas stations in mid-air so better check if you're fueled up) and to monitor some statistics in-flight (to make sure you won't be fined for speeding). There's some custom controls here with transparency, and the whole app is supported by smart Office documents: the new ActiveDocumentHost control allows you to embed those directly into your WinForms app! If there's a problem with one of the readings (e.g. your altitude is too low), you can look at an Excel sheet showing the data and you have a Research Pane ready to help you diagnose the problem before you submit a problem report in Word.

The Maintenance Application:

So we've already covered quite a lot of the scenarios but there's still a lot of work (especially on the Flight Attendant app on Pocket PC) - so I'm off coding again!

ASP.NET 2.0 features a whole new DataBinding infrastructure, which allows you to define and consume data sources declaratively. Sounds fancy, and it is. If you look at last week's coding summary, you can see this in action:

<asp:ObjectDataSource ID="infoData" Runat="server" TypeName="WeFly247.UI.Proxies.PassengerWebService" SelectMethod="GetFlightData" />
<asp:GridView ID="info" Runat="Server" DataSourceID="infoData" />

As you can see, there's that new GridView control, and in stead of binding it in code (you know, set the DataSource property and don't forget to call DataBind!) you just set a DataSourceID. This is a reference to some data source object, which can now be just a control as any other (under the "more markup! less code!" motto).

Now there are a number of data source controls: SqlDataSource (accesses SQL Server), AccessDataSource (accesses Access (whew)), SiteMapDataSource (reads a sitemap file), DataSetDataSource and XmlDataSource (which speak for themselves), and last but certainly not least: the ObjectDataSource.

Now this is what I call goodness. They've finally come to their senses and realized that not everyone uses a database on the client tier. They've heard themselves say "Web Services" one time too many I guess, and this ObjectDataSource is the solution for everyone who has data living on another tier. What it does is create a new instance of the type you specify, and then call a method on it to get the data. After that, the object is disposed again but your disconnected data remains alive in the data source control.

Possibly (I'm just guessing - hoping actually - here, I haven't checked since I can't use Reflector to decompile this yet ) they'll cache the object and method information (not the object itself, the documentation states clearly that it's disposed of) somehow to prevent every call from having the overhead of using reflection to find the type and the method to call. Anyway.

So in our case this little piece of markup just creates a new Web Service proxy for the passenger Web Service, calls the GetFlightData method on it (which effectively sets the whole SOAP story in action, talk about hidden complexity) and uses the returned data for data binding.

Sounds easy, and easy usually means limited. Fortunately, they've thought well about it, and provided quite a range of extra options. DataSource controls support parameters to be passed in or out, through parameter controls which can fetch their values from other controls, form fields, cookies, the new ASP.NET Profile, the query string, the session, or you can build your own. (A literal parameter value would be nice to pass in a constant though, but it doesn't seem to be there...). For example, this one fetches the user's culture from his Profile, and passes that along to the GetProducts method:

<asp:ObjectDataSource ID="advertisementData" Runat="server" TypeName="WeFly247.UI.Proxies.PassengerWebService" SelectMethod="GetProducts">
 <SelectParameters>
  <asp:ProfileParameter Name="cultureName" Type="String" PropertyName="Culture" />
 </SelectParameters>
</asp:ObjectDataSource>

And in case you were worried about two-way data, DataSource controls also support updating, deleting and inserting through other properties - again with parameters like before. For a lot more information, look at the excellent MSDN article on databinding in ASP.NET Whidbey by Early & Adopter.

So this is one of the reasons I'm pretty impressed with ASP.NET 2.0. More reasons are bound to follow - stay tuned

"WSE 2.0 is now Microsoft.Web.Services2 and the config section is called <microsoft.web.services2>." [Via Early Adopter]

"WSE major versions are not simply incremental feature additions, they are different libraries. As such, they deserve different names." Sure, there are some good technical reasons to append the version number to the namespaces, dll's and config sections, but I don't like the smell of this. Stuff like "msxml.dll, msxml2.dll, msxml3.dll, msxml4.dll" springs to mind. Granted, those are libraries with incremental feature additions - they're not simply "different libraries" as is the case here.

It's sort of like calling your successor product "NT" (which in Windows-land stands for New Technology - or doesn't it?), "Next-Generation", or "The Revenge" for that matter.

I'm not calling this change dll hell, but it still feels icky. I wish that .NET would even have nice solutions for these kinds of "higher-level" versioning issues.

Web Services so rock: I'm teaching a .NET Framework course this week and in the Web Services module I thought I'd give a live demo of the wsdl.exe tool that generates proxy classes for Web Services. Normally live demos are not a good idea but hey sometimes you can really get something cool out of them so I figured I'd give it a go.

So I point my brower at xmethods.org to look up an interesting Web Service and I stumbled upon one called InstantMessageAlert that claims it can send instant messages to any of the four major IM services: MSN, AIM, ICQ and Yahoo! Messenger.

So I sign in to Yahoo, take the wsdl, generate a proxy class, call the SendYahoo method and BANG! Nothing happens Ok so that was a setback. Then I popped up MSN Messenger and called the SendMSN method - et voila: an instant message appears from out of the blue telling me how Web Services rule. And thereby giving me extra credit for my evaluation later on