I have finally updated the Linq Expressions DebuggerVisualizer to work in VS 2010.

There were a lot of new kind of Expressions introduced so I added support for them too.

You can see it in action with data from the MSDN samples for the SwitchExpression class and the TryExpression class.

You can download the Visualizer here

All controls that inherit from ListControl (BulletedList, CheckBoxList, DropDownList, ListBox and RadioButtonList) have some annoying behavior in common:

You can only bind the control to a single property using the DataTextField property. For example, if you have a Customer class with properties Id, Name, Surname, etc and you want to show the full name of a customer in any of the ListControls, you have several options:

1. When you get the data from the database, return another column with the name and surname.
2. Create a property (f.e. FullName) that returns what you want to show.
3. Use a DataSet and add a computed column.
4. Manually add ListItems with the appropriate values performing the data bind manually.
5. Create a simple class that will be bound (instead of the original one) just to fix the problem.

The truth is that I don’t like any of those options, all of them are hacks.

For nested properties you have the similar problems. You can bind to Id or Name, but you can’t bind to Address.City or Address.Zip.

Today I faced this problem again and I decided to investigate a more innovative solution. The problem itself is in the controls that inherit from ListControl that are very strict in the binding options. So I took a look of how the controls perform the binding to see if I could do anything to overcome those limitations. At first sight I thought I was loosing my time but after a deeper study I found that it was amazingly easy to fix those problems in a very sleek way.

The data binding of the controls is performed in the PerformDataBinding method. The code at ListControl.PerformDataBinding uses DataBinder.GetPropertyValue to retrieve the value to show in the control if you set the DataTextField property. However, DataBinder.GetPropertyValue doesn’t handle nested properties. DataBinder.Eval is a better choice because it does handle nested properties. So using it we solve one problem. The other problem can be solved easily as well. As we have two properties that control how the Text of each ListItem will be extracted and shown from the data source, with a bit more of effort we can handle the new functionality. I have come up with a very intuitive solution:

• DataTextField allows one or more properties (that can be nested if needed) separated by commas.
• DataTextFormatString can be used to control the formatting of the items, using {0}, {1}, {2}, {3}… as the value of the properties specified with DataTextField.

The code that performs the data binding is:

As we need to call this data binding code instead of the one from ListControl I needed to subclass all controls that inherit from ListControl in order to fix them. So now I have BulletedListEx, CheckBoxListEx, DropDownListEx, ListBoxEx and RadioButtonListEx.

I have created a simple example showing how to use them. Imagine that an Item can be supplied by a set of suppliers, and each supplier sells the item with a different price. The following image shows the properties of the 3 entities of the sample:

I have created a web page with a DropDownList with some items, and when you select an item in the DropDownList you will see a ListBox with the all suppliers of that item with the associated price for the item. Notice how I show the Id and the Name in the DropDownList and how I show the price, supplier Id and supplier name in the ListBox:

The ASPX of the page follows:

As you can see, the solution is really clean and elegant, and backward compatible with the current controls that inherit from ListControl.

That’s all for now,
Merry christmas!

ListControlsEx_bin.zip (6.53 KB)

Probably most of you have heard about AdSense, the ad serving program run by Google. It is very simple to have a website to display ads using AdSense, as you only need to include an external script with a few parameters (that mainly configure the size and colors of the ads). For each click in a banner of the website, the associated AdSense account gets a bit of money.

However, this is not the only way to get AdSense in your pages. Google has the "AdSense API", which consists in several web services that allow sharing the revenue of the banners between the website owner and the content publisher. That way, website with user generated content can pay their users if their content is useful.

A customer asked me to modify their website so they can share some of the benefits of their banners with the creators of the articles in their site using the AdSense API. I’ll detail some of my experiences with the integration of the AdSense API in the webpage.

As I had to deal with web services, and I was comfortable with the web service support present in the .NET Framework 2.0, the easiest thing to do for me was to follow that route. However, I always try to learn new stuff when I can, so it was an excellent chance for me to learn WCF. Creating a proxy for a web service in WCF is as easy as in .NET Framework 2.0. However, the problems started really soon. For some unknown reason, the Google guys require some SOAP headers in the web service calls that are not exposed in the AdSense’s WSDL. Reading some of the comments in the AdSense Group revealed that the SOAP headers were exposed previously but they were removed without giving any reason. I found this to be really annoying and. It can be fixed adding the headers to the OutgoingMessageHeaders in the current OperationScope. Only for this detail the Adsense API can be labeled as unfriendly for the developers.

Once you set the headers manually, it is easy to call to the different methods to create AdSense accounts, associate them with a publisher and generate the code to display ads. However, I found very confusing some parts of the documentation, specially all the ids available (you end up using clientId, publisherId,  synServiceId, developerId, and in some methods having to pass the same id twice).

Inspecting the generated code for the WCF proxy, I found one thing I didn’t like about WCF. The DataContractSerializer does not handle bare arrays, so the generated code was using the XmlSerializer for a lot of things that need directly or indirectly to handle bare arrays in the AdSense API. The official statement of Microsoft about this seems to be that bare arrays do not support the distinction between null arrays and empty arrays, so the best is to use a wrapped collection.

The AdSense API has a technical requirements page that informs you how to proceed in some interactions with the users and how to react to error conditions. The AdSense API methods expose the fault details as specified in the AdSenseApiException. When an error happens when your WCF proxy is calling to the AdSense API, a FaultException is generated. I was expecting the proxy to receive a FaultException< AdSenseApiException> instead so I could easily access to the exception details but that wasn’t the case (maybe that only works in WCF to WCF scenarios?). I tried to create a generic error handler in order to convert the SOAP faults to the FaultException<AdSenseApiException> type, but the information I found about WCF error handling in the web was only talking about server side error handling using the IErrorHandler interface. It was clear that I needed more in depth knowledge of WCF in order to do that. After some research I bought Inside Windows Communication Foundation:

The book was exactly what I was looking for. Good and detailed internal information about how WCF worked.

Thanks to the book I was able to make my proxy generate the FaultException<AdSenseApiException> transparently, but it wasn’t easy. Well, it is not difficult but you have to create a lot of classes to do something simple.

A channel exposes a method called GetProperty<T>. When an exception is generated in the channel, the GetProperty<T> method is called with a parameter of type FaultConverter. The method should return an an instance of a FaultConverter, which creates the appropriate exception/fault type. To solve my problem I created an AdSenseFaultConverter:

However, in order to expose the AdSenseFaultConverter I had to create a ChannelFactory (AdSenseChannelFactory), a channel (AdSenseChannelBase and AdSenseRequestChannel), a Binding (AdSenseHttpBinding), and a BindingElement (AdSenseBindingElement). A lot of classes with little added value just to override the GetProperty<T> method in the channel. Also, to be able to use the configuration instead of setting up the communication programmatically, I had to create a configuration BindingElement(AdSenseHttpBindingElement) and a CollectionBindingElement (AdSenseHttpBindingCollectionElement). As I said before, it is a lot of code.

One thing that I haven’t commented yet is that your AdSense API implementation needs to be reviewed in order to be able to serve live ads, and you need to follow some policy requirements that you need to carefully study as they want to control even the help pages you put in your web site. One of the requirements to be eligible to participate in the AdSense API program is to have more than 100000 daily page views. This is a showstopper for most of the web sites making the AdSense API useless for 99.9% of the people.

Another negative point of the AdSense API is that the WSDL has been broken for more than a month. I planned to write this article a couple of months ago so I don’t know if the WSDL of the AdSense API is still invalid or not, but having it broken for more than a month can give you an idea of the overall satisfaction I have with the AdSense API.

To finish talking about the AdSense API, I have to mention the Sandbox. The development web services don’t behave exactly as the live web services (something that I hate), so some methods need to have additional headers when testing the API to work properly. Why? Well, basically because they do not have a decent sandbox. They just have the development version of the web services and you have to live with that. No GUI for managing the developers accounts, payments or something similar to the AdSense account webpage. If you compare this to other sandboxes like the paypal one where you can even receive the IPN confirmation messages you can clearly see the difference.

And the last rant of the post is about Google policies regarding to AdWords and AdSense. If you are an European citizen and you want to advertise using AdWords, Google forces you to pay in Euros. However, when you have AdSense in your page, Google will pay you in dollars. Judging by the prices of the ads in AdWords and the money you receive with AdSense, Google is taking advantage of the European citizens by charging a 60% more for an ad in AdWords and paying a 60% less for it (the exchange rate is approximately 1 euro = 1.60 dollars). Is the European commission too busy fining Microsoft for including free products with Windows to investigate things like this?

To sum up, I’m completely disappointed with Google and their AdSense API. If some people ever had the illusion that Google was a non evil company that did the things the right way, they need to wake up.

One of the tools I’m using is NDepend (http://www.ndepend.com/). It is a cool application that can be used for a lot of things, as will be detailed later.

To use NDepend you have to feed it with a set of assemblies. After that, it will analyze those assemblies and their reference assemblies. If the PDBs of the assemblies are available it will use them so more data is analyzed, although the PDBs are not needed for NDepend to work.

After the analysis has been completed, NDepend looks like this:

I have used version 1.1 of Spring.NET Framework (http://www.springframework.net/) for the sample. As you can see the application has a beautiful GUI. I’ll talk a bit about the main windows of the tool:

• Metrics: this window gives a graphical high level view of the project. You can see a lot of ellipses and some of them that are made of smaller ellipses. The big ellipses are types of the project. If a big ellipse is made of several smaller ellipses, those smaller ellipses are the methods of the type. The types of the same namespace are distributed in the same rectangular region, separated by a small yellow line. The assemblies are separated by a thicker yellow line.

By watching this window you can get an idea of which assemblies contain more code, which types are big, which types have a lot of methods, etc. The window was in “Method Level”. The available levels are Assembly, Namespace, Type, Method and Field.

• Class Browser: this is where the assemblies of the project and its referenced assemblies are shown. You can expand the assembly up to the field level. While you hover over an item in most windows, the Metrics window gets updated highlighting the hovered item.

What it’s really cool about the Class Browser is that for the referenced assemblies, only the namespaces, types, methods and other items that are used by the project are shown.

• Info: When you hover over an item the information about it is displayed here, with some metrics if applicable. There are a lot of metrics so it can take time to interiorize them and judge their value appropriately.

• Dependencies: this window is a bit intimidating at first but like a lot of things, with a bit of effort you can master it. The window displays very important information of the dependencies in a matrix form. As in other windows, the detail level is selectable, from the assembly level to the field level.

A cell can have 3 colors. Blue means that the associated item in the top header uses the associated item in the left header. Green means that the associated item in the left header is using the associated item in the top header. Black means a cyclic dependency. The blue and green cells express the same dependency using opposite points of view.

You can see a diagram of the dependencies clicking on a cell. For example, if you click in the cell corresponding to System.Web.Extensions and System.Web you get this:

To read more about the theory of dependencies take a look here:

ttp://www.theserverside.net/tt/articles/showarticle.tss?id=ControllingDependencies

• CQL Queries: CQL stands for Code Query Language and it is a central piece of NDepend. Being able to ask questions about the code is great. If you like using the Analyzer window in Reflector you’ll love this. You can create new queries and modify the existing ones.

I have presented the main NDepend windows, but the main question is: what you can do with NDepend?

There are several uses that I can think of:

• It is an excellent tool to check differences between different builds of an assembly. You can use it as a diff tool, as it will show in the Class Browser which types, methods, etc have been added, removed or changed. This information is also available for CQL Queries, with the power that it implies.

• It can be used to understand code. When you are thrown with code that you have no clue about how it works you have several options to accomplish that task:

1. Look at the source code directly.
2. Make some diagrams from the source code (for example, using View Class Diagram in Visual Studio).
3. Use NDepend.

These methods are not exclusive, and probably you’ll use a combination of them to understand how the code works. However, the abstraction level is higher when you use NDepend is higher than when you use a Class Diagram (and a Class Diagram is a high level view compared to raw source code).  I’d use NDepend to have a global vision of the main assemblies, type most used, dependencies between namespaces, more complex methods, etc. and then focus my attention in the most important parts of the application, going to a lower level of detail.

• It can be used in the refactoring process, to see the impact of a change, and to keep dependencies to a minimum, in order to have a project that is easier to understand and maintain.

• Integrating NDepend in the software development process. IMHO this is a key point, as using NDepend regulary will certainly improve the quality of the software that is being developed.  NDepend can generate a report of a project. A sample report is available here:

http://www.ndepend.com/SampleReports/OnNUnit/NDependReport.html

There are a lot of sections in the report, but I’ll concentrate on the section called “CQL Queries and Constraints”. That section is really important as it shows a list of items that can be problematic or don’t follow the standards of your company (methods with a lot of lines of code, very complex methods, poorly commented methods, types or methods or properties or fields with incorrect naming conventions, etc). This way the project leader can make periodic checks of those problems and report them to the people that are responsible for them.
This also allows you to know more the people you’re working with. Some developers don’t comment code, others like to make very long methods, and others tend to write complex logic that is difficult to understand by others. With the review of the report, they can be instructed so the resulting code and comments is better and more uniform.

However, there are times where a method with more than 30 lines of code is necessary. Or a method with a cyclomatic complexity of 25 is needed. So it is up to the project leader to review the suspicious method and inform the developers of the problem or simply acknowledge that it is necessary.

Unfortunately, NDepend does not provide built in support for this level of detail of project management. I have been told that in a future version it will provide hooks so custom scenarios like the exposed above are fully supported.

So, after what I have said about NDepend, you can be thinking, is it for me? Well, that depends on a lot of factors. I know a lot of companies that have no interest in achieving high quality products and that they only care about having a project done as soon as possible. If the maintenance of that project is a nightmare, well, that is another problem for another moment. However, there are some companies that actually spend time and effort to produce excellent products, so they may be interested in what NDepend can add for them.

I have used NDepend in several projects that are widely used when learning about it, and I found something curious:

The image shows that almost all of the NHibernate code has a big dependency cycle. I wish them good luck making the transition to ASTs.

To finish this post, I have to say that I have received a free professional version of NDepend. This has not conditioned my post about it. If I think it was a useless product I will not have posted about it.

Some time ago I created a DebuggerVisualizer for LINQ expressions. I have finally updated the DebuggerVisualizer for Visual Studio 2008 RTM version. You can download it here.

The main changes are that Funclets and LiftExpressions were removed after Beta 1. For more information check this post.

I have tested it only in a couple of expressions so let me know if you find any problem.

If you have worked with the design time infrastructure you will know that there are parts of it that are poorly documented.

To complement MSDN help and internet searches you can try another source of information: patents.

Most big companies like to patent a lot of stuff so I stumbled upon this patent that has some good information about design time. The inventors of the patent are:

• Andrew Cheng-Min Lin
• Bulusu Krishna Mohan
• Nikhil Kothari
• Simon Calvert

Does any one sound familiar to you?

Anyway, nothing beats using reflector to figure out some of the more obscure interfaces. Unfortunately, it is also the slowest method.

One of the coolest tools I have been using for the lasts months is DocProject:

http://www.codeplex.com/DocProject/

DocProject generates documentation from an xml comment file using sandcastle. The generated documentation can be a precompiled help file (help 1.x or help 2.x) or a web site.

There are other tools that serve as a friendly GUI for sandcastle but DocProject integrates great with Visual Studio 2005 (and 2008) and lets you create and maintain the documentation without effort.

When DocProject is installed, two new project types will show in the “New Project Dialog”:

• Project Documentation: that lets you generate a precompiled help file.
• DocSite Web Application: that creates a web site with the documentation.

When you select one of those project types, it will ask you for which projects you want to generate the documentation, and basically that is all that you need to get DocProject working (of course you need to generate the xml comment file for the projects you want to document, so be sure to have the checkbox at “Build -> Output -> XML documentation file” enabled in the project properties.

You can configure a lot of options when generating the documentation. To do that you have to click in Tool->Options->DocProject. I won’t detail the options, but it is really straightforward to use. The only thing I miss is that this can’t be done in the project properties for the projects created by DocProject.

There are several styles available for the generated documentation. Here is a pic of a precompiled help file of the one I’m using:

You can take a look at a website that uses DocProject here:

http://docs.mbunit.com/

If you compile often (as I do) you will see that generating the documentation takes a long time so probably you’ll want to exclude the generation of the documentation when you compile (Build->Configuration Manager->uncheck the build from the documentation project) and build it when you really need it or if you have use a continuous integration server, let it do the work for you.

So the only excuse now to not have good documentation for your projects is being lazy.

From time to time I like to evaluate the way I’m working, investigate on new alternatives, try other approaches, etc. in order to improve the way I do things. Other times, I have to change the way I do some things by necessity. This time was a bit of both. The number of components I’m selling on my web page www.manuelabadia.com is growing slowly and some things I was doing by hand were a pain to maintain and started to take too much time, so a change was really needed. Also, in the development area, things were also getting more complex.

Distribution of the components by mail is a lot less reliable than I thought as some companies have really strict firewall/attachment policies. Implementing a decent LicenseProvider based on digital signature was more complex than I thought, has some hurdles with ASP.NET and works a bit different between web site projects and web application projects. However, having the LicenseProvider implemented has freed me of all component distribution and signing, saving a lot of time and headaches.

Maintaining and developing new components was also getting more complicated, because components were starting to share code, and having to maintain two versions of the same code is problematic and error prone. However, splitting the components in several projects and then using ILMerge seems to work great. Improving my subversion merge/branch skills was also necessary as I end with a lot of releases with a very similar codebase but with little differences between them. Better tool integration for ordinary tasks with Visual Studio has also helped a lot. I’ll comment about a few of these tools in a future.

There seem to be a lot of good tools emerging with a goal to improve the quality and reliability of our code. Microsoft research is also trying some things with Spec# and Pex. Maybe I’ll blog about those when Pex becomes available to the masses.

So what about the future? Well, I have a lot of ideas for new components and new features for the existing ones. As always, the biggest problem is time. There’s no time to do all the stuff I have in mind or the cool things I want to try (has anybody tried SharpMap?) I have a component in the works that has been nearly finished for a couple of months. I hope to finish it at the end of the year or in January 2008. After that probably I’ll concentrate on further improving the existing components and checking if something new can be added if the version of ASP.NET shipped Visual Studio 2008 has a couple of significant changes. After that probably I won’t release more components because I can’t be sure to be able to provide a good service if I release more components.
 
As I said before, the sales are growing slowly, and the components are being used in most of the “important” countries:

This shows me that I’m doing things well and encourages me to work harder and keep improving my skills.

Due to lack of time I haven’t been able to make a sample website showing a complete usage scenario for my NHibernate custom Membership and Role providers. However, as there are nearly 100 downloads per month and there is some people using it I’m uploading a new version. The changes from the latest release are:

• Fixed an important bug that didn’t prevent locked out users from login in.
• Added a CreateUserWizardEx control that allows creating a user with extra information in an easy way.
• Added auto unlock support.
• Fixed some minor bugs/typos.

I’ll detail a bit the new features. The standard CreateUserWizard control generates an event before creating the user (CreatingUser) and one after the user has been created (CreatedUser). Somewhere between these two events the control calls the CreateUser method of the membership provider. The CreateUserWizardEx control extends the CreateUserWizard that ship with ASP.NET 2.0. It is supposed to be used when you want to insert a user in the system without using the CreateUser method of the membership provider (for example, because you want to store more data in the database).
 
The CreateUserWizardEx control adds a CreateUserEx event. You should handle this event, retrieve all the information for the new user and make the insert in your users table manually (as the provider interface can’t handle extra data). This event will be generated after all the password checking and generation have been made, so you only have to insert the user in the database. When you handle this event, you’ll be passed an instance of the CreateUserExEventArgs class. This class has 3 properties: EncodedPassword, EncodedPasswordSalt and EncodedPasswordAnswer that are useful if you’re not storing the sensitive data in plain text. After the CreateUserEx event is generated, the CreatedUser event is also generated.

There is one little problem with the CreateUserWizardEx control. Because quite a bit ASP.NET classes (especially controls) haven’t been designed with extensibility in mind, it is impossible to override some of the behavior of the CreateUserWizard control to achieve the desired result. In this particular case I can’t make the control to skip the CreateUser call to the membership provider in any way without rewriting the full control (the CreateUserWizard is one of the lengthiest controls in ASP.NET this is not an option). So the only solution without compromising medium trust scenarios is to ignore the CreateUser method calls in case you plan to insert the users manually. To do this you have to set the provider attribute ignoreCreateUserMethod to true. If the CreateUserWizard control is ever rewritten with extensibility in mind this hack could be removed, but I don’t think that will happen anytime. I posted about similar lack of extensibility problems here.

The auto unlock feature is useful if you use locking. If a user fails entering the password several times in a predefined period of time (specified by the maxInvalidPasswordAttempts and passwordAttemptWindow properties) his account gets locked out. However, you need human interaction in order to unlock the user.

If the auto unlock feature is enabled, when a locked user tries to log on the system, if the data supplied is correct and a certain period of time has passed since the account was locked out, the provider automatically unlocks the account. To enable auto unlocking you have to specify enableAutoUnlock="true" in the web.config. The provider property autoUnlockMinutes controls the minium period of time to wait since the last failed login to be able to auto unlock an account. The default is 30 minutes. Note that setting this value too low can make the security benefits of having a lockout feature disappear.

I’m thinking in adding some code to enforce single user log on for the next version as time permits (don’t expect this to be implemented in 2007).

As always, feedback is welcome. As the previous version, it has been tested only in SQL Server 2005. Let me know if you try it in other databases.

You can download the providers here.

Something that is really annoying is the list of little bugs or problems that pollute the .NET Framework and doesn’t seem to get fixed (mostly because of remotely backward compatibility reasons). Others problems are simple solved as “won’t fix“, without giving any explanation.

For example, take a look at this one:

https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=94108

If you use a TreeView, invalid XHTML gets generated because a script tag is added without specifying the type. They only need to add type="text/javascript" to the generated script. It is clearly a bug and it won’t break anything. However, it is marked as “won’t fix”. WTF?

Sometimes one can be lucky and they actually fix the bug (or at least they told you that they will fix it) but this seems to be the exception rather than the rule. For example, I submitted this bug some time ago and it seems it will be fixed:

https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=265321

However, the list of unresolved issues that clearly need to be fixed is big and growing, but a lot of them aren't addressed. IMHO this should change. I prefer to have a remote possibility of having something that works to stop working because of bug fixes than keep going with thousand of bugs and work arounds for not solving them. Also, I'd like to have updates more often. To wait a year for a service pack for the framework is too much.

Usually, the built-in providers for ASP.NET do not integrate very well with existing applications. The membership provider has a very complex structure that in most cases is not needed at all. The following diagram shows most of the tables and relations for the built-in ASP.NET providers:

The membership related tables are selected in red and the role related tables are selected in blue. The default membership and roles provider have support to store data for multiple applications using the aspnet_Applications table. I think this table was introduced because some portal frameworks like dotnetnuke needed the ability to store membership and role data for multiple users in a single store because they support multiple portals. If you don't need this scenario (and most applications do not need it), this only complicate things.

Also, the built-in providers use tables that are not part of your application tables, and some of those tables have quite a few columns, and probably some are not needed by your application.

If you do not use SQL Server, you can not use the default providers, so that is a big showstopper.

To overcome all of this problems I have created a custom membership and role provider that use NHibernate and can be easily integrated with existing or future applications using your own tables.

The reference book about providers is:

Professional ASP.NET 2.0 Security, Membership, and Role Management.

I suggest you to read it if you really want to get deep into providers as it is an excellent book.

The NHCustomMembershipProvider uses NHibernate to do all membership related database operations so it should work on all databases supported by NHibernate. The membership data will be stored and retrieved from a table configured in the provider section in the web.config so you can map it to an existing table like users. The column names for your user table can also be configured in the provider section. The cool thing about the provider is that it adapts its behaviour to the number of columns supported by your data store. For example, if you use the provider and map it to your users table where you only have the membership data for userId, userName, password and email, it will work without any problem. If your data store supports more advanced options of the membership provider like account lockout, password question and answer, password salt or last activity date, that advanced functionality will be automatically available, but it is not required.

The provider knows what functionality to offer based on the number of columns that are mapped to your data store. To support account lockout when the user do not supply a valid password after some tries in a predefined period of time you need to map the isLockedOutColumn, failedPasswordAttemptCountColumn and failedPasswordAttemptWindowStartColumn to columns in your data store.

The default configuration for the provider is to use integer identities or sequences for the user identifier, but that can be changed using the userIdType and idGeneratorClass properties in the provider configuration section.

If you use the provider with an existing user table, probably you will create the users manually without using the CreateUser method from the membership provider. The provider exposes the methods CheckPasswordPolicy, ValidatePassword, EncodePassword and GenerateSalt to allow manual creation of the user in case you need it, without sacrifying password strength.

To finish with the custom membership provider lets see an example. If you want to use the membership provider with a table like this (that has columns for all supported functionality):

The configuration section for the membership provider will be something like this:

   
The column sizes are optional but it is a good practice to fill them.

The built-in provider raises events after a succesfully or a failed login. However, ASP.NET 2.0 does not allow to create or raise built-in health monitoring events, so I had to use reflection in a private method in order to raise the same events. That means that the provider will not run in medium trust if it is not installed in the GAC if you want to raise the system events. As this can be a showstopper to some people I have added an option to disable system events generation (raiseSystemEvents).

The provider reads the config section and programatically generates a mapping for the specified columns in the ConfigureNHibernate method. The provider expects that you have configured NHibernate in the web.config to work. If you need extra configuration for the provider you can inherit from the provider and use override the ConfigureNHibernate method, modifying the Configuration object that the base class returns.

The NHCustomRoleProvider is also configurable in order adapt to your own tables. It only needs a Users table with id and user name columns, a roles table with id and role name columns, and a join table used to store the information for the many-to-many relationship between the users and roles.

For example, if we the previous user table with the following tables (FKs omitted for brevity):

The configuration section for the role provider will be something like this:

Note that the id columns in the join table need be named as in their respective tables in order to work.
   
Programatically creating the mapping using the CreateMappings for something more complex than a single table seems to be something very tied to the NHibernate implementation so I opted to create a XmlDocument with the mapping read from the configuration settings as it seems to be the best thing to do in complex cases.

One of the problems I faced with the NHCustomRoleProvider is that some times I wanted to save or delete data from one side of the many-to-many relationship but in some cases I wanted to use the other side. If you know a bit of NHibernate you know that the inverse attribute controls what side of the relationship has to be used in order to have changes persisted. As I was generating the mapping dynamically I created two session factories, one with the inverse set to true on one side and the other with the inverse in the other side. That way I open a connection in the session factory that has the mapping with the inverse in the side of the relationship I want.

I'd appreciate usage feedback, suggestions, bug reports, etc. I have only tried these providers with NHibernate 1.2.0 GA and the MsSql2005Dialect so let me know if it works in other configurations.

Please do not store the providers in your server. Link to this page instead. Updates to the providers will be posted in this page.

History

v1.0 - 08/04/2007

v1.1 - 21/06/2007

• Now using NHibernate 1.2.0 GA
• Some minor fixes

v1.2 - 12/10/2007:

• Fixed an important bug that didn’t prevent locked out users from login in.
• Added a CreateUserWizardEx control that allows creating a user with extra information in an easy way.
• Added auto unlock support.
• Fixed some minor bugs/typos.

For more details about the features introduced in version 1.2 take a look here.

v2.0 - 01/08/2008:

For more details about the features introduced in version 2.0 and a sample of its usage take a look here.

NHCustomProviders_bin.zip (67 KB)
NHCustomProviders_source.zip (122 KB)

In a previous post I said that the ObjectDataSource can handle generic types using the backtick (`) notation, even if the ObjectDataSource doesn't have any design time support for this.

This works because the ObjectDataSource internally calls to Type.GetType to resolve type names to Type objects. As the Type.GetType method handle generics transparently, the ObjectDataSource does not have to do anything special to support it at runtime. Supporting generic types at design time is more complex so they fiter out generic types in the "Select Type" wizard step.

For generic methods things change. There is some inconsistency in generics handling in the .NET Framework. If you get the type of a class like List<Customer>, its Name is List`1[Customer]. However, if you have a method like Customer Get<Customer>(int id), the method name is Get. IMHO, to have an orthogonal implementation, the method name should be Get`1[Customer]. Also, the GetMethod method should understand the backtick notation for methods and retrieve the specified method properly but it isn't supported for now. I have requested an improvement for the next version of the .NET framework here. Probably they won't fix it but it would be nice if you vote for it.

Returning to the ObjectDataSource... When the ObjectDataSource tries to find a method, it call the GetMethods method and compares the method name with the expected name, filtering out generic methods.

So, I think Microsoft didn't bother to add support for generic types or generic methods, and if generic types work at runtime is by luck (because of the GetType behaviour).

As more developers are using generics, there is a need to properly support generic methods and generic types. The next version of the ExtendedObjectDataSource (that will be released next week), will have full generics support, at design time and at runtime.

It wasn't easy to add design time support for generics as several problems appeared:

* Generic type expressions and generic method expressions can be arbitrarily complex (like MyGraph`2[[Utilities.Set, Utilities, Version=1.0.0.0, Culture=neutral, PublicKeyToken=a7a65c534534e209],[Utilities.Graph, Utilities, Version=1.0.0.0, Culture=neutral, PublicKeyToken=a7a65c534534e209]], MyGraphLibrary, Version=1.0.0.0, Culture=neutral, PublicKeyToken=2134afb4534e209), so I had to write a parser for both kinds of expressions.
 
* At runtime, specifying "__code" for the assembly name gets traslated to the dynamically generated assembly (as explained in the previous post). However, at design time this translation has to be done manually.
 
* I had to rewrite the select type wizard step and select method wizard step as the code was getting harder to maintain with generics.

* Showing and choosing generic methods and types is half of the problem, after choosing them you have to create the generic type or the generic method based on the generic type definition and the generic method definition.
 
Lets see an example of what can be accomplished now with the CompatObjectDataSource.

Imagine that we want to be able to register some objects at the start of the application in order to used later using a registry.

The registry can be implemented like this:

   
Some of the data stored in the registry when the application starts is a list of all available countries (for example, retrieved from the database):

The Country class is very simple:

If we want to show all the countries in any webform we can use an CompatObjectDataSource. After dropping one to the toolbox we configure the type:

And then we can see the list of methods compatible with the Select operation (GetObject and GetObjectList):

As both methods are generic method definitions, after selecting one of them, we have to specify the generic parameters. In our case, we stored the countries using a string key, and we stored a List of Country objects so the generic parameters for the method are System.String, Manu.GenericTests.Country. After clicking in "Make Generic Method" we have the data source configured properly:

If we take a look to the ASPX we can see the following code:

The SelectMethod is saved as "GetObjectList`2[System.String, [Manu.GenericTests.Country, __code]]" and the TypeName is saved as "Manu.GenericTests.Registry, __code".

The only thing to do now is to set the DefaultValue property for the key parameter to countries. Now the CompatObjectDataSource has enough information to call to GetObjectList<string, Country>("countries") from the Registry class and retrieve the countries.

The only thing I'm missing in the CompatObjectDataSource/ExtendedObjectDataSource is support for Bind expressions for nested properties (<%# Bind(Address.PostalCode) %>), but unfortunately, the framework can not understand those expressions so nothing can be done ast the moment. I wrote an email to ScottGu about that and got this reply from Shanku Niyogi:

"This is definitely a doable thing - we'll add it to our list of items we're considering adding for an upcoming release."

So keep your fingers crossed.

I'm open to suggestions in what to functionality you would like to see in the ObjectDataSource/CompatObjectDataSource/ExtendedObjectDataSource.

Recently I was pointed out by a friend that a new version of the Hibernate Reference book was released. I have already read the old version of this book, called "hibernate in action" but as it was half the size of the new book, I decided to read this one too. Even if the content is heavily based on the previous version, a lot of parts have been rewritten and it is a lot more clear now (or maybe it is just that I'm a better hibernate user...). Also, I found out that someone is finally doing a book about NHibernate:

I didn't want to talk about the book in this post. What really caught my attention was the improvements in the Java 1.5 language. I haven't coded anything in Java for quite a few years (the last version I used was 1.3). In that time the Java language didn't have any important changes but the version 1.5 has added a lot of useful things to the language: Generics, Metadata, Boxing/Unboxing, Enumerations and variable number of arguments.

Does some of this sound familiar to you? Java implemented Generics a bit before .NET Framework 2.0 but all the other stuff was already present in the first version of the .NET Framework. Also, something that I can't live without now is Properties and Java doesn't has them. Simulating properties using a get_/set_ methods is nothing more than a dirty hack (you can also simulate OOP in assembler...)

I have been an early .NET adopter since the beginning of the .NET Framework and a lot of my friends were jumping on me about that, because in their opinion Java was a lot better, the .NET Framework was crap, etc. Now they have to shut up ;-)

The local companies here are also using more .NET than Java and I have read some studies that said that both technologies are similar but usually is less expensive to develop with .NET (unfortunately I haven't found the links). Also, if you consider the new features added since the release of .NET Framework v2.0 (WPF, WF, WCF, ASP.NET AJAX Extensions, LINQ, DLINQ...) clearly Java is lagging behind .NET.

The ObjectDataSource supports using generic classes in the TypeName property although it doesn't provide any support for it. Usually you'll see that the TypeName property stores the name of a type (i.e. CustomProject.MyDAL), but it can store the type's full assembly qualified name (i.e. CustomProject.MyDAL, CustomProject, Version=1.0.0.0, Culture=neutral, PublicKeyToken=a7a65c534534e209).

The type's full assembly qualified name can be quite complicated and it is described by the following grammar:

TypeName := NamespaceTypeName | NamespaceTypeName ',' AssemblyNameSpec
 
NamespaceTypeName := NestedTypeName | NamespaceSpec '.' NestedTypeName
 
NestedTypeName := GenericTypeName | NestedTypeName '+' GenericTypeName

GenericTypeName := IDENTIFIER | IDENTIFIER '`' Number '[' GenericTypeArgList ']'

GenericTypeArgList := GenericTypeArg | GenericTypeArg ',' GenericTypeArgList

GenericTypeArg := NamespaceTypeName | '[' TypeSpec ']'
 
NamespaceSpec := IDENTIFIER | NamespaceSpec '.' IDENTIFIER
 
AssemblyNameSpec := IDENTIFIER | IDENTIFIER ',' AssemblyProperties
 
AssemblyProperties := AssemblyProperty | AssemblyProperties ',' AssemblyProperty
 
AssemblyProperty := AssemblyPropertyName '=' AssemblyPropertyValue

For more details about the grammar for type's full assembly qualified name in .NET Framework v1 take a look here

If you look in detail the grammar, it accept generic type names, that have a '`', then the number of generic argument and then the generic arguments inside square brackets.

So the following are valid type names: 

System.Colllections.Generic.List`1[System.Int32], mscorlib

MyGraph`2[[Utilities.Set, Utilities],[Utilities.Graph, Utilities]], MyGraphLibrary

MyGraph`2[[Utilities.Set, Utilities, Version=1.0.0.0, Culture=neutral, PublicKeyToken=a7a65c534534e209],[Utilities.Graph, Utilities, Version=1.0.0.0, Culture=neutral, PublicKeyToken=a7a65c534534e209]], MyGraphLibrary, Version=1.0.0.0, Culture=neutral, PublicKeyToken=2134afb4534e209

So if you want to use a generic type in the ObjectDataSource you just have to specify it using the syntax explained above. Be sure to add the assembly names if you use generic types.

What happens if the generic type you want to use is in the App_Code folder? The code in the App_Code folder gets compiled into a dynamically generated assembly with a random name so it is impossible to know it. Fortunately, you can use __code as the assembly name and it will work. Why does this work? When the framework can't resolve an assembly, the AppDomain AssemblyResolve event is fired. The BuildManager class (that is a helper class for ASP.NET compilation), captures this event and checks the specified assembly name. If it is __code, it returns the dynamically generated assembly for the App_Code folder. Returning App_Code also works (and you save executing a few instructions).

As an special case, you can use VB, C# or other .NET supported language in the same web application if you place the code for each language in a different folder. For example, if you want to use VB and C# you have to create these folders:
App_Code/VB
App_Code/CS

and add this to the web.config:

<codeSubDirectories>
    <add directoryName="VB"/>
    <add directoryName="CS"/>
</codeSubDirectories>

If you use this configuration two assemblies will be dynamically generated for your code behind classes, one for each folder. If you want to reference it in the TypeName of the ObjectDataSource or where the framework expects a fully qualified name you have to use App_SubCode_VB or App_SubCode_CS.

Of course, the ObjectDataSource doesn't have design time support for this. In fact, the ObjectDataSource choose type dialog filters out the generic types, the interfaces and the types registered in the GAC.

The ExtendedObjectDataSource will have full design time support for generic types and more control about the types that are filtered in the next version:

When you select a generic type, a textbox and a button appear. You can use the textbox to specify the type of the generic parameters and click in the button to select the generic type. Also, there are some checkbox to have control over what types get filtered.

The above dialog has some little positioning issues that I think are caused by Windows Vista, because the dialog was defined as:

In a future post I'll talk about generic methods. 

Click here for Part 1 of this overview
Click here for Part 3 of this overview

IMHO the more awkward thing in ANTLR is the tree implementation. A tree is implemented as a tree node and all the operations related to trees are moved to another class. Also, there can be “nil” nodes, that are fake nodes used as parent for childs (this is used to create flat lists).

In ANTLR a tree has to implement the ITree interface:

In the ITree interface there are some properties that are also present in the IToken interface (Line, CharPositionInLine, Type and Text) in order to associate a position with the current tree for errors and because it is also needed for parsing.

The Nil property is used to check if this node is a “nil” node.

The property ChildCount and the method GetChild are used to navigate the tree.

The DupNode methods clones the current node and the DupTree method clones the whole tree.

In a normal implmentation of a tree node, the method AddChild doesn’t need more comments but in ANTLR the methods that operate on trees are not orthogonal and usually differ if one of the operands is a “nil” node. In this case, if the tree node passed to the AddChild is a “nil” node, all the children of that “nil” node are added as children of the tree. Otherwise works as expected.

The ITree interface is implemented by the abstract class BaseTree that implements the tree using a list of child nodes, and the node doesn’t have any payload.

The concrete classes that implement BaseTree in ANTLR are:
• CommonTree: that adds a token as the payload.
• ParseTree: that stores strings and tokens as the payload.

To operate on the tree you have to use a class that implements the ITreeAdaptor interface:

The AddChild and Dup methods do the same as the ITree counterparts.

The Create methods are used to create a tree node from a token.

The method GetNilNode is used to obtain a “nil” node.

The GetChildCound and GetChild methods are used to traverse the tree.

The BecomeRoot method returns a tree where the newRoot is the root of the resulting tree and the oldRoot is a child of the resulting tree. There is special handling for “nil” nodes (see the ANTLR code for details).

The RulePostProcessing method is called after a rule has been processed in order to optimize the resulting tree. After this method, the SetTokenBoundaries is called in order to set the bounds in the input token stream of this tree (including all the children).

The other methods are for getting/setting the node type and text when doing tree parsing.

The abstract class BaseTreeAdaptor implements the ITreeAdaptor interface and adds an abstract method called createToken. The concrete class that is used in ANTLR is the CommonTreeAdaptor class, that inherits from BaseTreeAdaptor. The CommonTreeAdaptor class creates CommonTree and CommonToken objects.

When ANTLR generates a Parser that builds the AST, the generated class has a protected field called adaptor (and the associated property Adaptor) which is used in the rules to create the AST.  Also a return type of each method created for each rule changes to a new generated class (with a name rulename_return) that inherits from the ParserRuleReturnScope:

The RuleReturnScope class is used to provide start and stop information (token or tree) as well as the AST or the template when output=AST or when output=template is used.

The generated class that inherits from ParserRuleReturnScope stores the resulting AST and any return value defined in the rule.

You may be wondering why to generate an AST and then create a Tree Parser for the generated AST to do more work if you can add attributes and logic to the rules in the parser to do what you want to do. The reason is that there are a lot of rules in the grammar that are just for precedence or to make the grammar cleaner and also there are some tokens (for example the brackets) that are not needed after parsing. The resulting AST has only the data need to perform our task without garbage. For example, consider a grammar for evaluating expressions with support for relational expressions and bit operators. The expression ((i >= 0) AND (i <= n)) OR ((index & 2) = 1) produces this parse tree:

The generated AST from that expression is:

So the AST is a lot easier (and clearer) to use.

I have used my StructsViz DebuggerVisualizer pack to generate those tree diagrams in the Visual Studio.NET 2005 debugger.

After studying the tree related classes and the Parser class it is time to start with Tree Parsers. The TreeParser class inherits from BaseRecognizer and will consume tree nodes from an ITree. The input of a tree parser will be a class implementing the ITreeNodeStream interface. This interface is an specialization of the IIntStream interface to operate on tree nodes:

As you can see, the interface can return any tree node with the LT method, and with the TreeSource and TreeAdaptor properties you can access to the tree providing the data and to the associated adaptor.

The ToString method prints the text of the specified nodes.

The HasUniqueNavigationNodes is used in debugging mode and specifies if the UP and DOWN nodes are reused or created each time they’re needed. The meaning of the UP and DOWN nodes will be explained later.

The CommonTreeNodeStream implements the ITreeNodeStream interface, and also the IEnumerable interface (so the nodes can be accessed sequentially using the IEnumerable related methods and properties, that is, Reset, MoveNext, Current). The class has 3 static tree nodes defined: the UP node, the DOWN node and the EOF node. To traverse the tree sequentially the class uses two stacks, the node stack and the index stack. The node stack is used to track the parent node when we’re walking down the tree. The index stack is used to know which child node we’re currently visiting. The tree is visited using a depth first traversal. When the first child of a node is visited, a ficticious DOWN node is added to the lookahead buffer and when the last child of a node is visited, a ficticious UP node is added to the lookahead buffer. After all nodes have been visited, the ficticious EOF node is always returned. When the nodes are accessed sequentially using the IEnumerable interface, the ficticious nodes are not returned. The ficticious nodes are returned only when the LT method of the ItreeNodeStream interface is called.

For completeness here are all the methods and properties of the CommonTreeNodeStream class:

After the constructors, you can see the method and properties of the IEnumerable interface, followed by the methods and properties of the IIntStream interface, and then the methods of the ITreeNodeStream interface.

The other methods and properties are used to maintain a circular lookahead buffer of tree nodes and to handle the navigation taking the ficticious nodes into account.

Nothing better than a couple of images here. For the following AST:

The lookahead buffer with the ficticious tokens included looks like:

The TreeParser class inherits from BaseRecognizer and gets tree nodes from a class implementing the ITreeNodeStream interface:

As you can see, the TreeParser is a very simple class that has properties for the input stream and overrides 2 methods of the BaseRecognizer class.

When ANTLR creates a TreeParser, it generates a class that inherits from the TreeParser class. The generated class has a method for each rule in a similar way as the parsers.

Thanks to the smart idea behind ITreeNodeStream, a Parser and a TreeParser are very similar if you take a look at the generated code. The Parser consumes tokens from an ITokenStream in a linear way (one dimension) and the TreeParser consumes tokens in a hierarchical way (two dimensions), but serialized to one dimension thanks to the ficticious UP and DOWN tree nodes.

To complete this ANTLR overview, I’ll post an example of a lexer, a parser and a tree parser and the generated code in a few days.

Another year ends and a new one comes. Life goes as fast as usual and we’re keep on our way. It’s time to think about it… our actions, hopes, wishes, mistakes, etc.

I usually listen to a special song a few minutes after the start of the year. In the last few years I have been choosing "Aerosmith – Full Circle" because it has special connotations for me.

I wish you the best in the coming year.

Click here for Part 2 of this overview
Click here for Part 3 of this overview

There are some articles out there about ANTLR v3 that cover how to define grammars and tree parses but I haven’t seen any that gives a global vision about all the classes and interfaces involved, so I’ll try to fill that gap. I’m using the C# version of ANTLR 3.0b5 as it is what I use and it has better name conventions for interfaces and classes.

There’s an upcoming book about ANTLR by its autor that will be a very valuable material if you work with ANTLR:

ANTLR lets us create a lexer, a parser and a tree parser. Depending on what you are creating, the classes generated by ANTLR will inherit Lexer, Parser or the TreeParser class. Those 3 classes inherit from a common abstract class called BaseRecognizer where all the code for parsing is present.

The Lexer class is also an abstract class that extends BaseRecognizer and implements the ITokenSource interface.

The ITokenSource interface has only one method, NextToken, that is used to get all tokens from the input. For ANTLR a token is a class that implements the IToken interface:

The basic properties of a token are its type and its text. The Line and CharPositionInLine are used to know where the token was read from the input.

A parser will read tokens from a channel, so the Channel property is usually used to disntinguish from a “normal” token from a white space or other token used as separator that does not alter the input to be parsed.

All tokens read from the input are assigned an unique TokenIndex (starting from 0 and incremented each time a token is read).

The CommonToken class implements IToken and it is the class that is usually used to represent a token in ANTLR. It has two properties not present in the IToken interface, StartIndex and StopIndex, that are used to indicate where the token starts and ends in the input stream. So the Text property returns the data directly from the input stream unless the Text property has been explicitly set. The ToString method prints a token like this:

[@15,23:26=’else’,<10>,1:22]

Where 15 is the TokenIndex, 23:26 are the StartIndex and StopIndex respectively, else is the Text, 10 is the Type, and 1,22 are the Line and CharPositionInLine respectively.

Returning to the Lexer class… it reads the characters from a ICharStream, that is an interface that extends the IIntStream interface for reading characters and adding line and column information to it:

The IIntStream interface is a contract for a stream of integers with variable lookahead and marker points so the stream can save a return point with the Mark method and return to it using the Rewind method.

There are several classes implementing the ICharStream interface so the Lexer can get data from several sources:

• ANTLRStringStream: to provide data from a string.
• ANTLRFileStream: to provide data from a file.
• ANTLRReaderStream: to provide data from a TextReader.
• ANTLRInputStream: to provide data from a Stream.
 

So finally we’re ready to understand the Lexer class:

The Lexer works using a ICharStream as its input and offers a bunch of properties to get information about the current character position of the underlying ICharStream.

The main operation accomplished by the Lexer class is to obtain tokens from the input stream when the NextToken method (of the implemented interface ITokenSource) is called.

When a Lexer is generated using ANTLR, a class that inherits Lexer is generated. That class will have a method called mXXX for each lexer rule where XXX is the rule name. Also a method called mTYYY (where YYY is an integer) will be generated for each token implicitly defined in a lexer rule like ‘>=’.

By default, each generated method will check if the expected characters are read from the input stream (using the Math, MathAny and MathRange methods) and then call the Emit method to generate the associated token, that is stored in a field called token.

When the NextToken method is called, the Lexer class the abstract method mTokens that is implemented in the class generated by ANTLR and  what it does is to select one of the m* methods explained before to call based on the input (using a DFA for this). That method will set the token field, and if the token should not be skipped (to skip a token you have to call to the Skip method in the Lexer class) it returns it. If not it repeats the process until a token is returned or the end of the input is found (EOF).

The Parser class inherits from BaseRecognizer and reads tokens from an object that implements ITokenStream. ITokenStream is just a specialization of the IIntStream to work with ITokens instead of ints:

ITokenStream uses an object that implements ITokenSource (remember that the Lexer class implements ITokenSource) to obtain tokens. There are two classes implementing ITokenStream, CommonTokenStream and TokenRewriteString. The CommonTokenStream class buffers all the tokens and will return tokens in a specific channel. The TokenRewriteString class has additional functionality to insert, update and delete tokens.

The Parser class is very simple:

As only adds an additional property to return the ITokenStream that feeds the parser.

When a Parser is generated using ANTLR, a class that inherits from Parser is generated. For each parser rule, that class will have a method with the same name as the rule. The method signature will depend on the attributes used (if any) in the rule and if abstract syntax tree (AST) construction is enabled or not.

Tree parsers will be explained in another post as there is a lot of tree related theory to introduce.

I couldn’t resist to improve StructsViz in order to add support for trees. Even if everybody has his own tree implementation, we can come up with a common agreement:

A tree is a collection of nodes, where each node has zero or more child nodes. Each node has zero or one parent. The only node that doesn’t have a parent is the root node.

So based on the previous definition, I have defined this interface that is mandatory to implement in order to visualize a tree node in StructsViz:

I think that any tree node implementation can easily implement this interface.

With that interface implemented, StructsViz will be able to draw your tree node.

I have been playing a bit with automatic layout positioning for trees, and with different kind of connections:

The pictures show debug information of the subtree areas needed to position the elements properly using a recursive algorithm. For example, the rectangle labeled as ST 2, is the subtree area for node 2, and it is calculated based on the ST 6 and ST 7 (subtree areas for node 6 and 7 respectively). With ST 1, ST 2 and ST 3, we can properly position node 0.

I’m giving the final touches to this and it will be included in the next StructsViz version.

Visual Studio.NET 2005 has a really cool feature called debugger visualizers. When you are inspecting an object while debugging, there are times that the standard inspection of fields and properties of the debugger is not the best way to check if the internal state of the object is correct. For example, a DataSet is very difficult to debug, because the data is stored in a long hierarchy of objects. A debugger visualizer is a viewer of objects of a predetermined type that presents the object to the developer using a custom UI that helps debugging it.

The debugger visualizer executes on a different application domain than the application we’re debugging so to access the object being inspected the data needs to be sent from one application domain to the other application domain. This implies that the object we’re inspecting needs to be serializable or we need to create a serializable proxy in order to visualize it.

There are quite a few articles on the web about debugger visualizers so I will not enter in detail in the process of writing a visualizer, but I’ll explain the basics. As I explained before, the visualizer runs in a different application domain than the process we’re debugging. To communicate those application domains, an object that inherits from VisualizerObjectSource is created. The visualizer inherits from the DialogDebuggerVisualizer class and has to implement the Show method, showing a custom dialog. Before showing the dialog, the visualizer usually retrieves data requesting it to the VisualizerObjectSource using an intermediate object that implements the IVisualizerObjectProvider.

The main method of the VisualizerObjectSource is GetData, where the object being debugged should be sent through a stream. If the object being debugged is big, this could be a long operation so a thing that can be done is to send a simplified object in the GetData method, and when the user wants to see more detail of the simplified object, transfer a more data using the TransferData method.

A data structure is something that is better represented graphically so I spent some days writing a bunch of visualizers for all the .NET System.Collections.* classes. Here you have some pics of it:

I had to write a basic diagramming engine in order to get some nice results. Writing the diagramming engine was the funniest part. I wanted a lot of flexibility so I end up implementing absolute and relative element positioning, cascading styles, shape composition, expandable elements, automatic layout adjustment, zooming, etc. I think that the resulting object model was pretty good even if it isn’t a complete diagramming engine.

After some work with the debugger visualizers, a big limitation surfaces. You can only attach a visualizer to classes and not to interfaces. I think Microsoft should change this is the next version as this is severely limiting the power of the visualizers.

Also, it is a shame that the .NET framework doesn’t include more complex data structures like a Tree or a Graph. A visualizer for a Dictionary, a List or a Queue can be useful, but a visualizer for a Tree and a Graph is very very useful. Every developer has to code his own Tree and Graph, so creating a viewer for them will make the developers to use my implementation if they want to visualize it because of the limitation of the debugger visualizers, and that can’t be done in a lot of cases.

You can read more about the visualizer pack here:

http://www.manuelabadia.com/products/StructsViz_features.aspx

You can take a look to the evaluation version here:

http://www.manuelabadia.com/products/Purchase.aspx

I have been playing lately with compiler generators and related stuff for something I’m working on. I had to re-read some parts of the dragon book (my memory is pretty bad) and I also found out that a new edition of the dragon book was released this year:

The best tool I have found is ANTLR. It is completely free, it can generate a lexer, a parser and a tree parser, so it can be used to generate all the needed stuff without having to use different tools. It supports C#, Java, C++ and Python. Also, it generates descendant recursive parsers so the code generated can be understood by a human (no more transition tables like yacc/bison). Even if predictive top down parsers can not handle all the context free grammars, this one supports pred-LL(k) grammars (LL(k) means Left to right Left most derivation with k tokens of lookahead, and the pred prefix means that it supports predicates to see if a rule has to be applied or not) with backtracking so it is really powerful.

The latest version (v3) is still in beta but it has some nice improvements over v2 so that’s the one I’m using. For v3 there isn’t much documentation so the learning curve is a bit high but it seems worth the effort. IMHO one of the bests things of ANTLR is that it has a great development IDE called ANTLRWorks that is amazing. Just look at this screenshots I took of some C# 2.0 grammar tests I did:

You have real time error checking, you can see a syntax diagram of the current rule, you can view the generated parse tree, debug the grammar, see rules dependencies, the DFA for the tokens, automatically remove left recursion from rules… awesome!

If you are wondering if ANTLR is good enough for you, think that JetBrains (the company that created ReSharper) uses it.

Enough talking, back to ANTLR…

The NDoc creators where spending a lot of time with the project with very little (if any) economic reward as it is happening in most open source projects (my personal experience with M.A.M.E. was even worse, because us (the developers) didn’t get a buck but a lot of people outside the team were getting tons of money, illegally selling it in cabinets or creating commercial console (or computer) arcade packs using the MAME source code without permission…) So, they have discontinued NDoc.

As Visual Studio 2005 didn’t come with a documentation generator (as Visual Studio 2003 had) this was a big problem for the dot net community, but Microsoft reacted quickly and created SandCastle, a cool documentation project. The problem is that SandCastle is in beta stage and it doesn’t have any GUI yet so you have to run several executables one after another to get the documentation. You can download the current version here:

http://www.microsoft.com/downloads/details.aspx?FamilyId=E82EA71D-DA89-42EE-A715-696E3A4873B2&displaylang=en

And there is a nice blog about information in how to use the executables to get the documentation done here:

http://blogs.msdn.com/sandcastle

As most people (myself included) don’t want to spent a lot of time and neurons in order to build a CHM documentation file, Eric Woodruff has provided a nice GUI that makes using SandCastle straightforward (and it’s very similar to the one provided by NDoc):

http://www.codeproject.com/useritems/SandcastleBuilder.asp

If you are curious, here you have a screenshot of the generated CHM file using the most common settings:

And talking about sandcastles… here are two from my visit to Torrevieja (Spain) this summer:

Being able to dynamically call an arbitrary method on an object or the ability to get/set any arbitrary property by name at run time is very important when it comes to do some generic stuff. In .NET Framework 1.x the only way to do that was using reflection and that means to do it slow. For simple things or if we don’t have a lot of concurrent users we can live with it, but sometimes the performance is important and reflection isn’t an acceptable solution.

Fortunately, the .NET Framework 2.0 introduces a new class that let us to quickly call an arbitrary method without the overhead of a reflection call. How? Well, the idea is to generate the same MSIL as the compiler generates when calling a function but on the fly, and then we obtain a delegate that when called will execute the MSIL we have just created. This is really useful in a lot of scenarios!

I read something about this in Marc’s blog (http://musingmarc.blogspot.com/) but I didn’t have time to look at it in more detail in that moment. Today I read an article at codeproject that talked about this (http://www.codeproject.com/useritems/FastMethodInvoker.asp) so I finally forced myself to spend some time playing with dynamic MSIL code generation and I’m quite impressed with the possibilities.

If you want to play with this you should download an excellent debugger visualizer that will let you show the MSIL generated by a DynamicMethod:

http://blogs.msdn.com/haibo_luo/archive/2005/10/25/484861.aspx

I have made a little helper class (based on the codeproject article) that can be used to replace MethodInfo.Invoke, PropertyInfo.GetValue, PropertyInfo.SetValue and Activator.CreateInstance that are the reflection methods that I use most often. In order to use it in your own projects, remember to cache the delegates obtained because every time you call the CreateDelegate the MSIL generation takes place and that takes some time.

I have also created a simple test to show a comparison between direct calls, reflection calls and dynamic method calls. The results are here:

------ Object Creation ------
direct - object creation: 63ms
dynamic method - object creation: 86ms
reflection - object creation: 391ms
------ Property Get ------
direct - property get: 21ms
dynamic method - property get: 81ms
reflection - property get: 1849ms
------ Property Set ------
direct - property set: 21ms
dynamic method - property set: 67ms
reflection - property set: 2476ms
------ Instance Method Call ------
direct - instance method call: 24ms
dynamic method - instance method call: 154ms
reflection - instance method call: 3266ms
------ Static Method Call ------
direct - static method call: 36ms
dynamic method - static method call: 157ms
reflection - static method call: 3139ms

To summarize:
• For object creation, using reflection is 6.2 times slower than direct object creation. With DynamicMethods is only 1.3 times slower than direct object creation. Using DynamicMethods is about 4.5 times faster than using reflection.
• To get a property using reflection is 90.1 times slower than direct property access. With DynamicMethods is only 3.8 times slower than direct property access. Using DynamicMethods is nearly 23 times faster than using reflection.
• To set a property using reflection is 117.9 times slower than direct property set. With DynamicMethods is only 3.2 times slower than direct property set. Using DynamicMethods is about 37 times faster than using reflection.
• For instance method invocation, using reflection is 136.1 times slower than direct method call. With DynamicMethods is only 6.4 times slower than direct method call. Using DynamicMethods is about 21 times faster than using reflection.
• For static method invocation, using reflection is 87.2 times slower than direct method call. With DynamicMethods is only 4.3 times slower than direct method call. Using DynamicMethods is about 20 times faster than using reflection.

I’m going to modify my ExtendedObjectDataSource to use DynamicMethods in order to gain a 20x speed increase compared to reflection calls.

The code used for the tests and the helper class for the DynamicMethods are shown here and can be downloaded at the end of this post:

DynamicMethods.zip (5.01 KB)

A friend recommended me to read the book “POJOs in action: Developing Enterprise Applications with Lightweight Frameworks”. In case you haven’t heard what a POJO is, it’s an acronym that stands for Plain Old Java Objects. Of course, my first reaction was that it was a Java book not a .net book but he told me that the book is about generic concepts that could be used in .net without much trouble so I purchased the book:

This book complements Fowler’s Enterprise Architectural Patterns[1] as it is more practical and it has a lot of samples. The author (Chris Richardson) explains his methodology to build enterprise applications using test driven development.

I haven’t used Java since 2001 and I didn’t have any problem following the book. If you are not a Java expert you can still read the book ignoring the comments about EJB. A little of background of JSP and Servlets is usefull when talking about the presentation layer but you can understand most of the content without it. The author uses a lot of Java tools/frameworks that have an equivalent counterpart in the .net world (NUnit, NMock, NHibernate, Spring Framework .net, etc) so even if there are some differences between the Java and the .net versions the information provided is useful.

The first pages of the book are about the decisions to make when building an application (organizing the business logic, encapsulating the business logic, accessing the database, handling concurrency in database transactions and handling concurrency in long transactions), and the rest of the book elaborates more on the usual strategies chosen in the design (OO domain model with O/R mapping) and some variations (transaction scripts, exposed domain model). When the author explains a topic it provides useful samples and the pros and cons of using one approach versus other approaches.

The O/R mapping material is very good and details the inner workings of JDO (AFAIK a Java only O/R mapper so I skipped most sections about it) and Hibernate. Also there is a chapter about complex search forms and how to make dynamic paged queries in order to retrieve data as fast as possible and integrate those queries with Hibernate.

The last part of the book is about concurrency in short and long transactions and I found it quite useful as in most applications I’ve seen don’t do anything about concurrency conflicts in long running transactions and the book has code explaining how to implement it where other books only give guidelines.

After reading the book I have added to my long TODO list to take a look to the spring framework so if somebody that reads this wants to share his experiences with it feel free to send me an email or post a comment.

First I have to apologize for not posting anything in nearly two weeks, but a lot of things have happened lately: I did a quick work (a week) for Namco Mobile; I have been two days in bed with temperature; I did a Reiki first degree course; too much work in the last days and a lot more to come until august…

I read somewhere that Microsoft .NET Framework 3.0 will be released at the end of the year. Even if it is not a complete change of the .NET Framework 2.0 and can be seen as .NET Framework 2.0 + Windows Presentation Foundation (WPF) + Windows Communication Foundation (WCF) + Windows Workflow Foundation (WF) + Windows CardSpace (WCS) it is kind of a pain to be learning version 2.0 and have a new release before mastering .NET Framework 2.0. If you add Atlas to the mix the conclusion is: “Too many things to learn and no time to do it!”. I hope we can live with .NET Framework 3.0 for a few years.

I don’t know what happens in other cities/countries but most of the companies here are still using .NET 1.x and probably a lot of them will still be using it when .NET 3.0 is released…

Talking about other things, I have started to add full design time support to the ExtendedObjectDataSource package and this is what I have at the moment:

As you can see I have added more options that are not present in the standard ObjectDataSource like being able to configure paging in the wizard and also being able to choose also the SelectCountMethod (for the CompatObjectDataSource. The ExtendedObjectDataSource can extract the total row count in the SelectMethod).

If somebody misses something in the wizard let me know ASAP so I can think of including it.

Every time I have to use Winforms I really hate the poor control set that is available (at least in version 2.0 you have a menu that does not look like it was made for windows 95 :-P). I can’t believe that the framework didn’t ship with a wizard control so I had to waste my time coding one.

To finish this post, I’ll tell you a cool tip to debug design time stuff. Forget everything that you did to set up the project to debug design time classes. You’ll have a debugger attached to the current Visual Studio instance and awaiting your orders if you add this line of code where you want the debugger to show up:

System.Diagnostics.Debugger.Launch();

When I was making the layout for the book last month I had to do a lot of repetitive tasks. I was on a hurry to finish the book so I didn’t have a chance to look at InDesign scripting capabilities. However, a few days after helping my friend I told him that a process like making the layout of a book could be automated a lot with scripting in order to avoid repetitive tasks.

When I started reading about InDesign scripting I thought that I had to do it using JavaScript. I didn’t have to learn another language so that was good. However, working with JavaScript is always slower than working with other languages (maybe because the lack of good tools?).

However I was surprised when I read that all InDesign features were exposed as a COM object. I could leverage the power of Visual Studio 2005 to do everything without having to do it in JavaScript!

To use InDesign from C# I had to import the InDesign Type Library and Visual Studio generated a COM wrapper for it. That was it! Of course, not all was perfect or straightforward. There were some things to consider:

• All InDesign collections start at index 1 instead of 0
• A lot of methods return object instead of a strongly typed object so a lot of casting is needed.
• There are times when you don’t know exactly what kind of object you get when calling a method and inspecting it in the debugger doesn’t help. Fortunately this guy used the compiler services to load all InDesign types and try to convert the object to all types until a match was found. This is very helpful when working with InDesign.
• There were times that setting the content of a paragraph or some text didn’t work as expected but setting it indirectly (calling a find/replace method for example) worked fine.
• The documentation is minimal. Sometimes I wonder why the MSDN documentation is obscure and reduced in some classes but you have to see InDesign scripting reference:

If you find a description with more than four words it’s your lucky day. It seems that the documentation is completely generated by a tool similar to GhostDoc and then a few properties and methods have been comented.

And the description of the classes is amazing:
Story: A story.
TrapPreset: A trapping preset.

If someone wants to start using InDesign with C# an example could be useful:

This sample removes the carriage return at the end of the footnotes that word usually lefts.