Thursday, 28 October 2010

Saturday, 16 October 2010

Validation with ASP.Net MVC2

I've blogged previously about the validation methods I've used on ASP.Net MVC projects and have recently been adapting this for use with version 2 of the framework.

One of the new features I wanted to make available was the use of data annotations for validation, where domain or view model classes can be decorated with attributes for validation of required fields, field lengths and range values. These can then be used for both server side and client side validation.

I ran into a problem with this though when validating entities that had related items - for example in my case I was working on a project management tool, where an entity such as a project is related to others such as a client and an account manager. The client entity has required fields such as a name, but when validating before saving the project entity I didn't really care about that as I was only interested in the client ID to perform the update operation.

The validation by default is set up to perform whole model validation, but a useful post from Steve Sanderson details how to get around this issue and perform partial validation based on just the values that are passed in the form post.

Data annotiation based validation isn't the the full story though - in a robust application it's considered necessary for the domain model to handle it's own validation too. This is partly a result of using techniques such as the above, which would allow a determined hacker to modify the form input to by-pass data annotation based checks on required fields.

In previous projects I've used straightforward C# code for the validation of model fields, but as I'd already used data annotions to define these rules it was a obvious violation of the DRY principle to have to do this again. .Net 4 provides a means of doing this using the Validator.TryValidateObject method.

For .Net 3.5 though I used a technique illustrated by JohnyO in this post on stackoverflow. It validates based on rules specified both directly on the class or in a meta data "buddy" class and thus allows the domain model to handle validation without repetition of all the rules.

Saturday, 2 October 2010

iPhone Custom UIView with Controls

Armed with a copy of the Pragmatic Programmer's book and a lot of Googling, have made a start recently on developing apps for the iPhone. Took a while to find my way around XCode, Interface Builder and to get to thinking again about things like memory management, but made some progress and have released an app for bass guitar players learning scales or modes.

It's fairly straightforward - is based on a navigation controller app, and has a single table view for selecting the scale and then a custom view for displaying the scale itself. You can choose the mode, key and speed and then view the tablature for playing the scale as well as listening to it played and viewing the fingering positions.

The only tricky bit really was the display of the scale and for this I needed to develop a custom UIView. By doing this you can draw directly to the canvas - which I needed to do for the tab lines (or strings) and for plotting the finger positions of the scale. Normally with a view you use Interface Builder or code to generate controls and associate them with IBOutlets and IBActions in XCode. The problem I found was that I wanted to do both - i.e. I needed a custom view in order to display the information that couldn't be rendered using controls, but also wanted to make use of controls for functions like the selection of mode and key.

I also had the issue that I wanted to pass information to the custom UIView, by setting a property for the scale (a custom class I had created which represented the selected scale - with properties like the name, description and intervals).

I solved this using the following code which involves casting the view object to the type of my custom UIView in the UIViewControllers viewWillAppear method, which then allows me to set it's custom property.

- (void)viewWillAppear:(BOOL)animated {
[super viewWillAppear:animated];

//Pass details of scale in custom property
[(ScaleDisplayView *)[self view] setScale:scale];

//Display scale details
self.nameLabel.text = [NSString stringWithFormat:@"The %@ scale",];
self.descriptionLabel.text = self.scale.description;

//Initialise view
self.title =;
self.view.backgroundColor = [UIColor whiteColor];
[self.view setNeedsDisplay];

Thursday, 22 July 2010

jquery - the trigger method

Having recently had to work on a client website where jquery wasn't available, it was really apparent just how useful this library is and how pleasureable it makes working with JavaScript. Having to resort back to getElementById and getElementsByClassName was painful in comparison.

Today I discovered and made use of yet another useful feature - the trigger method which enables an action on one element to fire the functions associated with events on another.

I had a simple accordion style user interface that looked like this:

On clicking the cross on the right a panel below would expand and display the content.

The mark-up looked like this:

    1 <ul id="list">

    2     <li>

    3         <div class="head">Title</div>

    4         <a href="">+</a>

    5         <ul>

    6             <li>Content one</li>

    7         </ul>

    8     </li>

    9 </ul>

And I had a function that carried out the toggling of the display like this:

    1 $("#content ul.list a.toggle").click(function() {

    2     if ($(this).next().is(":visible")) {

    3         $(this).next().hide("fast");

    4         $(this).text("+");

    5         $(this).parent().removeClass("on");

    6     } else {

    7         $(this).next().show("fast");

    8         $(this).text("-");

    9         $(this).parent().addClass("on");

   10     }

   11     return false;

   12 });

As you can see the code relies on the position of the link element (it's looking for the next element to expand) and makes some styling changes to the link itself.

So when the designer asked to make the whole bar clickable this was a bit annoying, as it looked like I would to replicate the function and adapt it to cope with a click on the bar instead of the cross.

But with jquery trigger you can fire an event on another element, so adding this small addition to the function provides the means for a click on the div element firing the function associated with the click of the link.

    1 $("#content ul.list div.head").click(function() {

    2     $(this).next().trigger("click");

    3 });

And the last line can be simplified further to:

  1 $(this).next().click();

Sunday, 18 April 2010

Progressive Enhancement with ASP.Net MVC

With its clean model for separation of concerns, one of the things I’ve been impressed by in use of MVC with ASP.Net is the support for building AJAX enabled applications that support progressive enhancement.

This technique takes the traditional strategy of graceful degradation in web design and turns it on its head. Rather than focussing on having the best possible user interface degrade to a less rich but still workable and presentable one, the focus is on creating a functional base UI that is enhanced by the addition of CSS and JavaScript to produce an improved user experience for those that can make use of it.

Of course either method can lead to the same result, but – in addition to the use of some specific techniques such as external referencing of assets to avoid swamping the more basic client with code or data that can’t be used – the change of approach leads to a different design mindset and result that better supports the full user base.

In this post I’ll illustrate a small part of an application I’ve recently worked on, where we developed an RSS reader for a site. One function was that users should be able to view a list of feeds, and add and remove RSS sources to it – and that we needed to support both script enabled clients and those that don’t have access to such technologies.

Firstly, the basic application before enhancement:

In the domain model I have a feed class that represents an RSS Feed object. Not shown in code but also developed for the application are repository and service methods for validation and the persistence of the list of feeds to the database.

    1 public class Feed

    2 {

    3     public int Id { get; set; }

    4     public string Name { get; set; }

    5     public string Url { get; set; }

    7     public List<FeedItem> Items { get; internal set; }

    8 }

In the controller code I have a few methods, for the retrieval of the list of feeds, and to support functions for the addition to and removal of them from the list. The key points to note here is that the addition of a new feed requires a straightforward form post, and the removal is via a link to a confirmation page, that in turn uses an HTTP post to proces the deletion. No client-side script required for a functional, if basic, UI.

    1 /// <summary>

    2 /// Render feeds page default view

    3 /// </summary>

    4 /// <returns></returns>       

    5 public ActionResult List()

    6 {

    7     IPrincipal user = GetUser();


    9     return View(new IndexViewModel

   10     {

   11         Feeds = feedsService.GetFeedList(user),

   12         IsLoggedIn = (user != null && user.Identity.IsAuthenticated),

   15     });

   16 }


   18 /// <summary>

   19 /// Receives the URL to an RSS feed, verifies it, and saves to the user's account

   20 /// </summary>

   21 /// <param name="url">URL of RSS feed</param>

   22 /// <returns></returns>

   23 [AcceptVerbs(HttpVerbs.Post)]

   24 public ActionResult Add(string url)

   25 {

   26     if (url.Length >= 4 && url.Substring(0, 4) != "http")

   27         url = "http://" + url;


   29     bool success = false;

   30     int feedId = 0;

   31     string feedName = "";

   32     string message = "Feed addition was unsuccesful. Please correct the errors and try again.";

   33     if (feedsService.ValidateRSSFeed(url))

   34     {

   35         Feed feed = new Feed();

   36         feedName = feedsService.GetRSSFeedNameFromUrl(url);

   37         feed.Name = feedName;

   38         feed.Url = url;

   40         try

   41         {

   42             feedId = feedsService.AddFeed(feed, ControllerContext.HttpContext.User);

   43             success = true;

   44             message = "The RSS feed has been added to your list.";

   45         }

   46         catch (RuleException ex)

   47         {

   48             ex.CopyToModelState(ModelState);

   49         }

   50     }

   51     else

   52     {

   53         ModelState.AddModelError("Url", "The URL does not point to a valid RSS feed.");

   54     }


   56     TempData["Message"] = message;

   57     return RedirectToAction("Index");

   58 }


   60 /// <summary>

   61 /// Renders view for remove confirmation

   62 /// </summary>

   63 /// <param name="id">Id of feed for removal</param>

   64 /// <returns></returns>       

   65 public ActionResult Remove(int id)

   66 {

   67     Feed feed = feedsService.GetFeed(id, GetUser());

   68     return View("Remove", feed);

   69 }


   71 /// <summary>

   72 /// Processes feed removal

   73 /// </summary>

   74 /// <param name="contact">Feed object (created by model binding)</param>

   75 /// <returns></returns>       

   76 [AcceptVerbs(HttpVerbs.Post)]

   77 public ActionResult Remove(Feed feed)

   78 {

   79     bool success = false;

   80     string message = "Feed removal was unsuccesful. Please correct the errors and try again.";

   81     try

   82     {

   83         feedsService.RemoveFeed(feed.Id, GetUser());

   84         success = true;

   85         message = "The RSS feed has been removed from your list.";

   86     }

   87     catch (RuleException ex)

   88     {

   89         ex.CopyToModelState(ModelState);

   90     }


   92     TempData["Message"] = message;

   93     return RedirectToAction("Index");

   94 }

My view code marks-up the HTML for the display of the list, the links for removal and the form for adding a new RSS feed to the list.

    1 <ul id="feed-list">


    3 <% foreach (var item in Model.Feeds) { %>


    5     <li>

    6         <span><%= Html.Encode(item.Id) %></span>

    7         <%= Html.ActionLink(item.Name, "Details", new { id = item.Id })%>


    9         <%if (Model.IsAdmin || !item.Required) { %>

   10             [<%= Html.ActionLink("Remove", "Remove", new { id = item.Id })%>]

   11         <% } %>

   12     </li>


   14 <% } %>


   16 </ul>


   18 <% if (Model.IsLoggedIn) { %>


   20     <% using (Html.BeginForm("Add", "Feeds", FormMethod.Post, new { id = "form-add-feed"}))

   21       {%>


   23         <fieldset>

   24             <legend>Add new feed to your list</legend>

   25             <p>

   26                 <label for="form-add-feed-url">Feed URL:</label>

   27                 <span>

   28                     <%= Html.TextBox("Url", "", new { id = "form-add-feed-url" })%>

   29                 </span>

   30             </p>

   31             <p>       

   32                 <input type="submit" value="Add Feed" />                   

   33             </p>

   34         </fieldset>     


   36     <% } %>


   38 <% } %>

So far very straightforward, but the interface requires a number of full page post backs to function, and hence for a better user experience we can progressively enhance the application with AJAX.

Firstly, for the addition of the feeds to the list we can hijack the form post using some jquery. The key point to note about this and the other progressive enhancement techniques described is that if the client doesn’t support them, they are simply ignored and the base functionality remains untouched.

Within this function we can read the URL from the form, and construct an HTTP post response from client side code. When the response comes back from the post, we can process it and display either a success message (and carry out additional processing such as adding the new feed to the HTML list) or error response as appropriate. Finally, and importantly, we return a false response such that the default form submission process is cancelled.

    1 $(document).ready(function() {

    2     hookUpAddFeedFormAjaxPost();

    3 });


    5 function hookUpAddFeedFormAjaxPost() {

    6     $("#form-add-feed").submit(function() {


    8         //Get feed details

    9         var url;

   10         url = $("#form-add-feed-url").val();


   12         //Post to controller action for adding feed

   13         $.post("/Feeds/Add/", { url: url },

   14             function(data) {


   16                 if (data.Success) {


   18                     //Success... show message

   19                     $("#ajax-message p.message").text(data.Message);

   20                     $("#ajax-message").show();


   22                     //Add feed to list (if not already there)

   23                     var id, name;

   24                     name = data.Data.split("|")[0];

   25                     id = data.Data.split("|")[1];


   27                     if ($("#feed-list li span:contains(\"" + name + "\")").length == 0) {

   28                         $("#feed-list").append("<li><a href=\"/Feeds/View/" + id +

   29                             "\" class=\"feed\">" + name +

   30                             "</a> [<a href=\"/Feeds/Remove/" + id +

   31                             "\">Remove</a>]</li>");

   34                     }


   36                     $("#form-add-feed-url").val("");


   38                 } else {


   40                     //Failed... show errors

   41                     $("#ajax-error span.validation-summary-errors").text(data.Message);

   42                     for (var i = 0; i < data.Errors.length; i++)

   43                         $("#ajax-error ul").append("<li>" + data.Errors[i] + "</li>");

   44                     $("#ajax-error").show();

   45                 }


   47             }, "json");


   49         return false;

   50     });

   51 }

In order to hook all this up we need to make some changes to our controller code, but actually very little - which is where the support of this approach from the ASP.Net MVC framework comes in. In the controller we can examine a property of the Request object to determine if the action method is being called from an AJAX request, and take appropriate steps. Here we want to by-pass the usual RedirectToAction at the end of the method, and instead format and return a JSON response.

   56     if (Request.IsAjaxRequest())

   57         return CreateAjaxResponse(success, message, ModelState, feedName + "|" + feedId);

   58     else

   59     {

   60         TempData["Message"] = message;

   61         return RedirectToAction("Index");

   62     }

The JSON response itself is contructed using this helper function:

    1 /// <summary>

    2 /// Private helper method to format a JSON respone for AJAX requests

    3 /// </summary>

    4 /// <param name="success">Flag indicating success or failure of the method</param>

    5 /// <param name="message">Message</param>

    6 /// <param name="modelState">ModelStateDictionary containing error details on failure</param>

    7 /// <param name="data">Any further specific information required by callback</param>

    8 /// <returns></returns>

    9 private JsonResult CreateAjaxResponse(bool success, string message,

   10     ModelStateDictionary modelState, string data)

   11 {

   12     JsonResult json = new JsonResult();

   13     IList<string> errors = new List<string>();

   14     foreach (var item in modelState.Values)

   15         foreach (ModelError me in item.Errors)

   16             errors.Add(me.ErrorMessage);

   17     json.Data = new

   18     {

   19         Success = success,

   20         Message = message,

   21         Errors = errors,

   22         Data = data

   23     };

   24     return json;

   25 }

In a similar manner we can intercept the click of the remove link to disable the normal function and instead provide a client side method of confirming the deletion and constructing the HTTP post to process it. Again, if scripting is not supported, the hyperlink will just function as a normal anchor link to the confirmation form. But if it is, our client-side code will be executed and the rich AJAX UI provided instead.

Thursday, 15 April 2010

View Models and Form Models in ASP.Net MVC - A Second Look

Following my previous post looking at options on the use of view models within ASP.Net MVC, where I came down on the side of using a simpler approach passing domain models to views directly, in this blog post I'll outline the approach I'm taking with more recent project - where I've changed my approach to create view models and form models that wrap the domain model and can include additional fields and options that may be required in the particular view.

The argument for using custom view models is that they can be tailored precisely to each strongly typed view, providing a cleaner approach for display and updating data from views. The downside though is that there is some additional work involved. However, after further reading on the subject - would recommend both Steve Michelotti's post on MVC view model patterns and Jorit Salverda's maintainable MVC series - on balance now I've been using them more and more on projects, it seems the extra steps are worthwhile.

To illustrate this approach let's take again a simple example, where we have a domain model that represents a user. Each user has a user name and a role. The application using them needs to present a form for the update of a user's details, with the option to send them a welcome email.

The following code illustrates the code for the domain model.

    1 public class User

    2 {

    3     public string Username { get; set; }

    4     public Role Role { get; set; }

    5 }

And then I've created two further models in the UI layer - a view model that contains all the data for presentation on the form (the user object itself, and a SelectList for the various role options the administrator of the application can choose from. The second form model is created via model binding following the form post, where the user object is extracted and updated, and additional information also passed in a strongly typed manner is processed.

    1 public class UserViewModel

    2 {

    3     public User User { get; set; }

    4     public SelectList Roles { get; set; }

    5 }


    7 public class UserFormModel

    8 {

    9     public User User { get; set; }

   10     public bool SendEmail { get; set; }

   11 }

The controller code illustrates how the view models are created and information from the form model extracted.

    1 public ActionResult Add()

    2 {

    3     return View(new UserViewModel

    4     {

    5         Roles = new SelectList(usersService.GetRoleList(), "Id", "Name"),

    6     });

    7 }


    9 public ActionResult Edit(int id)

   10 {

   11     User user = usersService.GetUserById(id);

   12     return View(new UserViewModel

   13     {

   14         User = user,

   15         Roles = new SelectList(usersService.GetRoleList(), "Id", "Name", user.Role.Id),

   16     });

   17 }


   19 [AcceptVerbs(HttpVerbs.Post)]

   20 public ActionResult Edit(UserFormModel userFormModel)

   21 {

   22     if (ModelState.IsValid)

   23     {

   24         try

   25         {

   26             usersService.SaveUser(userFormModel.User, userFormModel.ResetPassword, GetUser());

   27         }

   28         catch (RuleException ex)

   29         {

   30             ex.CopyToModelState(ModelState);

   31         }

   32     }

   33     if (ModelState.IsValid)

   34     {

   35         TempData["Message"] = "User account " + (userFormModel.User.Id == 0 ? "created" : "updated") + ".";

   36         return RedirectToAction("Index");

   37     }

   38     else

   39     {

   40         return View((userFormModel.User.Id == 0 ? "Add" : "Edit"), new UserViewModel

   41         {

   42             User = userFormModel.User,

   43             Roles = new SelectList(usersService.GetRoleList(), "Id", "Name", userFormModel.User.Role.Id),

   44         });

   45     }

   46 }

And finally this strongly typed view code demonstrates how the form fields are named to support model binding following the post.

    1 <% using (Html.BeginForm()) {%>


    3     <fieldset>

    4         <h2>Account Details</h2>

    5         <p>

    6             <label for="User.Username">User name:</label>

    7             <span class="field">

    8                 <%= Html.TextBox("User.Username", Model.User.Username)%>

    9             </span>

   10         </p>

   11         <p>

   12             <label for="User.Role.Id">Role:</label>

   13             <span class="field">

   14                 <%= Html.DropDownList("User.Role.Id", Model.Roles)%>

   15             </span>

   16         </p>

   17         <p>

   18             <label for="SendEmail">Reset Password:</label>

   19             <span class="field">

   20                 <%= Html.CheckBox("SendEmail", false)%>

   21             </span>

   22         </p> 

   23         <p>

   24             <input type="submit" value="Save" class="indent" />                             

   25         </p>                           

   26         <%= Html.Hidden("User.Id",Model.User.Id) %>

   27     </fieldset>


   29 <% } %>

Without this approach it would be necesssary to pass the additional data to the view - the list of roles - in a loosely typed manner in the ViewData. And the additional information required from the form post - the flag to send the welcome email - would need to be passed as an extra paramenter to the controller action responding to the form post.

With hindsight I think this is clearly a cleaner approach and am using it for the MVC projects I'm working on now. In particular the method of creating a custom class for both the form display and the form post, provides a very elegent approach in my opinion.

Sunday, 14 March 2010

Working on an Umbraco Project with Multiple Developers

Over the past couple of months I've been working with the open source .Net CMS Umbraco, and been very impressed with the breadth of features and opportunities for extension and customisation that it provides. Operating as a one man band, it's very straightfoward to work with the product using the web based UI for most of the development tasks.

However one concern it gave was how to set it up such that in can be used within a development team - with multiple developers working on the same project and how to integrate it with source control systems.

Issues arise from the fact that whilst some content is saved as files on disk (e.g. templates, XSLT macros), others are held in the database (e.g. data types, document types, and the content itself of course).

In order to get around this, we've set up and used the following model - which seems to ameliorate the concerns:
  • Firstly, a single database is set up on a networked server for access by all developers. This means any created document types, data type and content nodes are immediately available to all developers, and they can rely on any IDs used to be consistent.

  • Secondly, each developer installs a clean Umbraco installation onto their local PC.

  • Finally, a companion Visual Studio solution is created. A project can be created within this for the creation of user controls and custom pages as would necessary in any case if such extensions were required. In addition though, copies of all file based information (stylesheets, scripts, templates/master pages, config files, xslt macros) are also created here.

    This being the case, such files can be managed as they would in any other ASP.Net project, and checked into source control as required. The key change to working practice is that they are edited within Visual Studio rather than Umbraco.

    An added advantage here is that you get access to VS.Net's intellisense and colour coding, which I've found to be a particular benefit when editing XSLT macros. There is the downside that you don't get to use Umbraco's own tooling support - in particular the toolbars for inserting fields and macros. However in practice I found that even an fairly novice Umbraco developer soon doesn't require these helpers, and if they are needed, can still be used of course provided the developer remembers to replicate the change into the VS.Net files.

    In order to integrate with Umbraco, a post-build script is set up that runs an xcopy statement to copy the necessary files from the VS.Net project to the Umbraco installation.

    So whenever an edit is made that needs to be reviewed, Ctrl + Shift + B will build the solution, and run the post-build script to copy the files into the Umbraco folder. And loading up the site running on localhost will allow the latest changes to be viewed in the browser.

Monday, 8 March 2010

Switched on Umbraco

Although for reasons I won't go into it looks like there's a good chance the project isn't going to see the light of day, I've been fortunate in the past couple of months to have chance to properly get to grips with the .Net open-source content management system Umbraco.

Having heard good things about it from colleagues and the web community, I was hoping to find in this product the right balance of providing a rich framework for the generic requirements of a content-rich site, whilst retaining the ability to extend as necessary without running into the limits of the product. And I have to say after spending this time working with the system, I haven't been disappointed.

I plan to blog a few specific developments where I've found need to extend the product to fit the needs of the web-site I'm building, but firstly just wanted to highlight the key benefits for me that I've found.

Custom Document Types
Even quite recently I was pitched a commercial CMS product that whilst flexible in it's design, was very limited in the types of content you could represent. Standard article pages were all very well, but a CMS should allow you to add new fields and relationships as necessary to accurately model your content types. Without it, you end up combining data that should really be seperated to support flexibility in presentation, and generally pushing a product into ways it was never really designed to be used.

Umbraco supports this via the concept of document types which you define with the fields you require, and how the type relates to other types within the site structure. Each type can have one or more templates associated, giving flexibility in the presentation for pages with similar content structures but different layout requirements.

Umbraco is built on .Net, so much of the techniques for extension of the system use familiar techniques to ASP.Net developers such as user controls and aspx pages.

The most efficient way to provide custom functionality when all you are looking to do is to aggregate information already held within the Umbraco content is to use XSLT macros. The syntax for these can get a little verbose and convoluted if you aren't familiar with it, but there are good examples to work from for many common requirements (e.g. breadcrumbs, navigation and site maps) and to get up to speed.

If you need to go further than this though, it's necessary to spin-up a companion VS.Net project where you can create user controls. By copying the ascx and dll files to the appropriate folders in the Umbraco installtion and creating a macro to "wrap" the user controls, all usual .Net functionality including in-line code and code-behind, events and post-backs can be brought to bear to the project.

Finally, Umbraco itself is exposed as an API with the set of classes available providing ample opportunity for further extensibility - such as responding to Umbraco events like "page publish" and allowing information from within and outside of Umbraco to be amalgamated for display.

Community Code
One of the advantages of leveraging a product like Umbraco is the ability to make use of not only the core product, but also the many community contributions that have been built to extend the product. Sample websites, search tools and image processing controls are some of the ones I've looked into thus far, each of which has saved many hours from building into a custom CMS solution.

Any Issues?
As is no doubt clear from the discussion above, I've been very impressed with this product and am looking forward to working with it much more in the future. There have been many occasions where I've been pleasantly surprised in finding that what initially seeemed like being a tricky requirement to implement has either already been accounted for in the core product, or is easily solved via a simple extension.

One possible issue is that the content editing UI - whilst intuitive and very straightforward to work with - does seem to be optimised for small to medium websites. And that it's tree view navigation structure could get a bit unwieldy if you had hundreds of "news stories" for example.

I've got around this by structuring content for a news section using dated year and month folders, which makes the UI a little easier to work with. And using an XSLT macro the content can be collapsed on the front-end website, where I may not want to break out the stories in the same way. However it still leaves the issue of only being able to provide the CMS user with a single route to navigate to their content - in custom CMS solutions I've built in the past I've tended to provide a search form allow the user to locate the story for edit via a combination of criteria such as date, author, category and keyword.

Another potential concern is that the product whilst fantastic for a one-man-band development, requires some workarounds to support multiple developers working on the same installation. The reason is the combination of some CMS assets being stored in the database (data types, document types and content) and others on the file system (templates, CSS, XSLT macros).

In the next blog post I'll discuss how we have set up our environment to support this requirement, and also how similarly we plan to handle deployment of changes from development, to test and production.

Sunday, 24 January 2010

3D Secure Implementation With PayPal

In a recent blog post I documented some details for setting up direct payments with PayPal. I've recently had to extend that to implement 3D secure - known also as Verified by Visa or Mastercard SecureCode.

The primary reason I've needed to implement this in a hurry for a client is that as a deadline has been reached that mandates the technology if you wish to accept Maestro cards on the website. Hence, if anyone else like me has been asked to set this up on the site in a hurry, hopefully this post that documents the steps required and techniques will prove useful...

Project References

Paypal have a relationship with a company called Cardinal Commerce who are available to customers using PayPal Direct Payments for support with implementing the 3d secure technology. I found them very helpful in going through the process. The first step is to obtain a dll file called CMPCDotNet.dll (known as the "thin client software"). Note that the version supplied with the initial documentation is for .Net 2.0 - a version for .Net 1.1 is available too and this is what I needed to use.

Lookup Customer's 3d Secure Enrollment

The 3d secure process involves 2 steps - the first one being a look-up to see if the customer's card is enrolled in the scheme or not. To do this you make a web request passing in details of the order and the customer's card details, and receive back a response indicating their enrollment status. If they aren't enrolled, you just continue as normal with the Direct Payments process. Otherwise you need to continue with the steps described below.

Having made a project reference to the CMPCDotNet.dll, you'll be able to call methods provided by it, as illustrated in the code sample at the bottom of this post.

Redirect to 3D secure site

Having checked the response and confirmed the user is enrolled in the scheme, we need them to transfer the customer to the 3d secure site where they will be asked to enter their saved password. Before this happens though, it's necessary to save some information that will be needed again later in the process - and hence what I did here was save this data to the Session.

Once that's done the redirect can happen - but this needs to be via an HTTP Post. Hence a Response.Redirect() can't be used here, and instead the technique I used was to write an HTML form to the response stream, and have that form automatically posted via javascript.


Once the customer completes the password entry process at the 3d secure website, control is passed back to your website to a URL passed in the initial form post. At this stage it's necessary to recreate state such as the user's card details in the form from the information saved to session.

Before continuing with processing the order via PayPal though, one last step is required which is to check whether or not the user was successfully authenticated with 3d secure. A second web request is made, passing the "payload" response received in the form post back from 3d secure (note - not the payload from the original look-up call - this caught me out for a while), and the transaction ID that was received in the first look-up request.

Again the response can be checked, and depending on the values received you can either decline the order, or proceed with charging the customer's card via PayPal Direct Payments. You will need to pass some additional details to PayPal, and also make sure you are using version 59.0 or later of the API.

Code Sample

The following code sample illustrates carries out the 3d secure authentication process described above. In order to demonstrate the key points from this blog post, it's not a complete working sample - you'll integrate your own methods for creating and retrieving orders of course - as well as set up the Direct Payments call (see my previous blog post if you need further details on this).

You'll see it also includes some code to log web requests and responses to a text file, which is useful for debugging and auditing purposes.

 1 private void Page_Load(object sender, System.EventArgs e)

 2 {

 3    //Check for post-back from 3D secure

 4    if (!Page.IsPostBack && Request.QueryString["3DSecPostBack"] == "1")

 5        Do3DSecAuthentication();

 6 }


 8 private void btnMakePayment_Click(object sender, ImageClickEventArgs e)

 9 {

 10    makePayment();

 11 }


 13 private void makePayment()

 14 {           

 15    //Create order record

 16    Order order = CreatePendingOrder();


 18    if (Page.IsValid)

 19    {

 20        //Open log file

 21        TextWriter objLogFile;

 22        objLogFile = File.AppendText(ConfigurationSettings.AppSettings["PaypalLogFilePath"]);

 23        try

 24        {

 25            //Get request details from card details form in ASPX file

 26            string cardType = ddlCardType.SelectedItem.Value;

 27            string cardNumber = txtCardNumber.Text;

 28            string expMonth = ddlExpiryDateMonth.SelectedItem.Value;

 29            string expYear = ddlExpiryDateYear.SelectedItem.Value;

 30            string expDate = expMonth + expYear;

 31            string startDate = "";

 32            if (ddlStartDateMonth.SelectedIndex > 0 && ddlStartDateYear.SelectedIndex > 0)

 33                startDate = ddlStartDateMonth.SelectedItem.Value + ddlStartDateYear.SelectedItem.Value;

 34            int issueNumber = 0;

 35            if (txtIssueNumber.Text != "")

 36            {

 37                try { issueNumber = int.Parse(txtIssueNumber.Text); }

 38                catch { }

 39            }

 40            string securityCode = txtSecurityCode.Text;

 41            string firstName = txtFirstName.Text;

 42            string lastName = txtLastName.Text;

 43            string street = txtStreet.Text;

 44            string city = txtCity.Text;

 45            string state = txtState.Text;

 46            string zip = txtZip.Text;

 47            string countryCode = "GB";

 48            string currencyCode = "GBP";

 49            string numericCurrencyCode = "826";


 51            //Do 3D secure look-up only for Maestro for now

 52            bool do3DSecure = (cardType == "Maestro");


 54            //Make first 3D secure call for look-up

 55            if (do3DSecure)

 56            {

 57                objLogFile.WriteLine("3D secure look-up commenced: " + DateTime.Now.ToString());


 59                //Set up request

 60                CentinelRequest centinelRequest = new CentinelRequest();

 61                centinelRequest.add("Version", ConfigurationSettings.AppSettings["CentinelMessageVersion"]);

 62                centinelRequest.add("MsgType", "cmpi_lookup");

 63                centinelRequest.add("ProcessorId", ConfigurationSettings.AppSettings["CentinelProcessorId"]);

 64                centinelRequest.add("MerchantId", ConfigurationSettings.AppSettings["CentinelMerchantId"]);

 65                centinelRequest.add("TransactionPwd", ConfigurationSettings.AppSettings["CentinelTransactionPwd"]);

 66                centinelRequest.add("TransactionType", "C");

 67                centinelRequest.add("Amount", ((int)(order.TotalAmount * 100)).ToString());

 68                centinelRequest.add("CurrencyCode", numericCurrencyCode);

 69                centinelRequest.add("CardNumber", cardNumber);

 70                centinelRequest.add("CardExpMonth", expMonth);

 71                centinelRequest.add("CardExpYear", expYear);

 72                centinelRequest.add("CardCode", securityCode);

 73                centinelRequest.add("OrderNumber", order.ID.ToString());

 74                centinelRequest.add("IPAddress", Request.ServerVariables["REMOTE_ADDR"]);

 75                objLogFile.WriteLine("... request prepared: " + centinelRequest.getUnparsedRequest().Replace(cardNumber, "XXXXXXXXXXXXXXXX"));


 77                //Set up variable to hold response

 78                string centinelErrorNo, centinelErrorDesc, centinelEnrolled = "U", centinelACSUrl = "",

 79                    centinelTransactionId = "", centinelPayload = "", centinelEciFlag = "", centinelTermUrl = "";


 81                //Make request and get response

 82                CentinelResponse centinelResponse = new CentinelResponse();

 83                try

 84                {                           

 85                    centinelResponse = centinelRequest.sendHTTP(ConfigurationSettings.AppSettings["CentinelTransactionUrl"], int.Parse(ConfigurationSettings.AppSettings["CentinelTimeout"]));

 86                    objLogFile.WriteLine("... response received: " + centinelResponse.getUnparsedResponse());


 88                    centinelErrorNo = centinelResponse.getValue("ErrorNo");

 89                    centinelErrorDesc = centinelResponse.getValue("ErrorDesc");

 90                    centinelEnrolled = centinelResponse.getValue("Enrolled");

 91                    centinelACSUrl = centinelResponse.getValue("ACSUrl");

 92                    centinelTransactionId = centinelResponse.getValue("TransactionId");

 93                    centinelPayload = centinelResponse.getValue("Payload");

 94                    centinelEciFlag = centinelResponse.getValue("EciFlag");

 95                    centinelTermUrl = "http" + (bool.Parse(ConfigurationSettings.AppSettings["UseSSL"]) ? "s" : "") + "://" + Request.ServerVariables["SERVER_NAME"] + Request.ServerVariables["URL"] + "?ID=" + order.ID + "&3DSecPostBack=1";

 96                }

 97                catch

 98                {

 99                    centinelErrorNo = "9040";

 100                    centinelErrorDesc = "Communication error";

 101                }


 103                //Check 3d secure response

 104                if (centinelErrorNo == "0")

 105                {

 106                    //No error, so check enrolled status

 107                    if (centinelEnrolled == "Y")

 108                    {

 109                        //Customer enrolled, so save collected details into session

 110                        Session["PendingOrderCurrencyCode"] = currencyCode;

 111                        Session["PendingOrderCardType"] = cardType;

 112                        Session["PendingOrderCardNumber"] = cardNumber;

 113                        Session["PendingOrderExpDate"] = expDate;

 114                        Session["PendingOrderStartDate"] = startDate;

 115                        Session["PendingOrderIssueNumber"] = issueNumber;

 116                        Session["PendingOrderSecurityCode"] = securityCode;

 117                        Session["PendingOrderFirstName"] = firstName;

 118                        Session["PendingOrderLastName"] = lastName;

 119                        Session["PendingOrderStreet"] = street;

 120                        Session["PendingOrderCity"] = city;

 121                        Session["PendingOrderState"] = state;

 122                        Session["PendingOrderZip"] = zip;

 123                        Session["PendingOrderCountryCode"] = countryCode;

 124                        Session["PendingOrderCentinelTransactionId"] = centinelTransactionId;


 126                        //Redirect to completion URL via form post

 127                        // - do this by writing out a form to the response stream that will submit automatically

 128                        StringBuilder centinelAuthForm = new StringBuilder();

 129                        centinelAuthForm.Append("<html>");

 130                        centinelAuthForm.Append("<body onload=\"document.auth.submit();\">");

 131                        centinelAuthForm.Append("<form name=\"auth\" action=\"").Append(centinelACSUrl).Append("\" method=\"post\">");

 132                        centinelAuthForm.Append("<input type=\"hidden\" name=\"PaReq\" value=\"").Append(centinelPayload).Append("\">");

 133                        centinelAuthForm.Append("<input type=\"hidden\" name=\"TermUrl\" value=\"").Append(centinelTermUrl).Append("\">");

 134                        centinelAuthForm.Append("<input type=\"hidden\" name=\"MD\" value=\"\">");

 135                        centinelAuthForm.Append("<p>If you are not automatically redirected, please click to proceed with authentication of your card details via 3-D Secure.</p>");

 136                        centinelAuthForm.Append("<p><input type=\"submit\" value=\"SUBMIT\"></p>");

 137                        centinelAuthForm.Append("</form>");

 138                        centinelAuthForm.Append("</body>");

 139                        centinelAuthForm.Append("</html>");

 140                        Response.Clear();

 141                        Response.Write(centinelAuthForm.ToString());

 142                        Response.End();

 143                    }

 144                    else

 145                    {

 146                        //Customer not-enrolled, so just carry on with PayPal process

 147                        DoPayPalRequest(objLogFile, orderId, currencyCode, cardType, cardNumber, expDate,

 148                            startDate, issueNumber, securityCode,

 149                            firstName, lastName, street, city, state, zip, countryCode,

 150                            "", centinelEnrolled, "", centinelEciFlag, "");

 151                    }

 152                }

 153                else

 154                {

 155                    //Error in 3d secure authentication, so exit

 156                    Display3DSecureErrors(centinelErrorNo, centinelErrorDesc);

 157                }

 158            }

 159            else

 160            {

 161                //3D secure not required, so just carry on with PayPal process without 3DS details

 162                DoPayPalRequest(objLogFile, orderId, currencyCode, cardType, cardNumber, expDate,

 163                    startDate, issueNumber, securityCode,

 164                    firstName, lastName, street, city, state, zip, countryCode,

 165                    "", "", "", "", "");

 166            }

 167        }

 168        finally

 169        {

 170            objLogFile.WriteLine("");

 171            objLogFile.Close();

 172        }

 173    }

 174 }


 176 private void Do3DSecAuthentication()

 177 {

 178    //Open log file

 179    TextWriter objLogFile;

 180    objLogFile = File.AppendText(ConfigurationSettings.AppSettings["PaypalLogFilePath"]);

 181    try

 182    {

 183        //Log and save PaResPayload posted back from 3DS

 184        string centinelPaRes = Request.Form["PaRes"];

 185        objLogFile.WriteLine("... payload posted: " + centinelPaRes);


 187        //Retrieve posted card and user details from session

 188        int orderId = int.Parse(Request.QueryString["ID"]);

 189        string currencyCode = (string)Session["PendingOrderCurrencyCode"];

 190        string cardType = (string)Session["PendingOrderCardType"];

 191        string cardNumber = (string)Session["PendingOrderCardNumber"];

 192        string expDate = (string)Session["PendingOrderExpDate"];

 193        string startDate = (string)Session["PendingOrderStartDate"];

 194        int issueNumber = (int)Session["PendingOrderIssueNumber"];

 195        string securityCode = (string)Session["PendingOrderSecurityCode"];

 196        string firstName = (string)Session["PendingOrderFirstName"];

 197        string lastName = (string)Session["PendingOrderLastName"];

 198        string street = (string)Session["PendingOrderStreet"];

 199        string city = (string)Session["PendingOrderCity"];

 200        string state = (string)Session["PendingOrderState"];

 201        string zip = (string)Session["PendingOrderZip"];

 202        string countryCode = (string)Session["PendingOrderCountryCode"];


 204        //Re-fill form

 205        ddlCardType.SelectedIndex = -1;

 206        ddlCardType.Items.FindByValue(cardType).Selected = true;

 207        txtCardNumber.Text = cardNumber;

 208        ddlExpiryDateMonth.SelectedIndex = -1;

 209        ddlExpiryDateMonth.Items.FindByValue(expDate.Substring(0, 2)).Selected = true;

 210        ddlExpiryDateYear.SelectedIndex = -1;

 211        ddlExpiryDateYear.Items.FindByValue(expDate.Substring(2)).Selected = true;

 212        if (issueNumber > 0)

 213            txtIssueNumber.Text = issueNumber.ToString();

 214        if (startDate != "")

 215        {

 216            ddlStartDateMonth.SelectedIndex = -1;

 217            ddlStartDateMonth.Items.FindByValue(startDate.Substring(0, 2)).Selected = true;

 218            ddlStartDateYear.SelectedIndex = -1;

 219            ddlStartDateYear.Items.FindByValue(startDate.Substring(2)).Selected = true;

 220        }

 221        txtSecurityCode.Text = securityCode;


 223        //Set up second 3D secure call for authentication

 224        objLogFile.WriteLine("3D secure authenticate commenced: " + DateTime.Now.ToString());

 225        CentinelRequest centinelRequest = new CentinelRequest();

 226        centinelRequest.add("Version", ConfigurationSettings.AppSettings["CentinelMessageVersion"]);

 227        centinelRequest.add("MsgType", "cmpi_authenticate");

 228        centinelRequest.add("ProcessorId", ConfigurationSettings.AppSettings["CentinelProcessorId"]);

 229        centinelRequest.add("MerchantId", ConfigurationSettings.AppSettings["CentinelMerchantId"]);

 230        centinelRequest.add("TransactionPwd", ConfigurationSettings.AppSettings["CentinelTransactionPwd"]);

 231        centinelRequest.add("TransactionType", "C");

 232        centinelRequest.add("TransactionId", (string)Session["PendingOrderCentinelTransactionId"]);

 233        centinelRequest.add("PAResPayload", centinelPaRes);

 234        objLogFile.WriteLine("... request prepared: " + centinelRequest.getUnparsedRequest());


 236        //Make request and check response

 237        CentinelResponse centinelResponse = new CentinelResponse();

 238        string centinelErrorNo, centinelErrorDesc, centinelPAResStatus = "",

 239            centinelSignatureVerification = "", centinelCavv = "", centinelEciFlag = "", centinelXid = "";

 240        try

 241        {

 242            centinelResponse = centinelRequest.sendHTTP(ConfigurationSettings.AppSettings["CentinelTransactionUrl"], int.Parse(ConfigurationSettings.AppSettings["CentinelTimeout"]));

 243            objLogFile.WriteLine("... response received: " + centinelResponse.getUnparsedResponse());


 245            centinelErrorNo = centinelResponse.getValue("ErrorNo");

 246            centinelErrorDesc = centinelResponse.getValue("ErrorDesc");

 247            centinelPAResStatus = centinelResponse.getValue("PAResStatus");

 248            centinelSignatureVerification = centinelResponse.getValue("SignatureVerification");

 249            centinelCavv = centinelResponse.getValue("Cavv");

 250            centinelEciFlag = centinelResponse.getValue("EciFlag");

 251            centinelXid = centinelResponse.getValue("Xid");

 252        }

 253        catch

 254        {

 255            centinelErrorNo = "9040";

 256            centinelErrorDesc = "Communication error";

 257        }


 259        //Check 3d secure response

 260        if (centinelErrorNo == "0")

 261        {

 262            //No error, so check response

 263            if ((centinelPAResStatus == "Y" || centinelPAResStatus == "A" || centinelPAResStatus == "U") && centinelSignatureVerification == "Y")

 264            {

 265                //Response OK, so process PayPay payment

 266                DoPayPalRequest(objLogFile, orderId, currencyCode, cardType, cardNumber, expDate,

 267                    startDate, issueNumber, securityCode,

 268                    firstName, lastName, street, city, state, zip, countryCode,

 269                    centinelPAResStatus, "Y", centinelCavv, centinelEciFlag, centinelXid);

 270            }

 271            else

 272            {

 273                //Response not OK, so can't go further

 274                lblResultMessage.Text = "Sorry, but we cannot complete your order as your card has not been authorised with 3D secure";

 275            }

 276        }

 277        else

 278        {

 279            //Error in response

 280            Display3DSecureErrors(centinelErrorNo, centinelErrorDesc);

 281        }

 282    }

 283    finally

 284    {

 285        objLogFile.WriteLine("");

 286        objLogFile.Close();

 287    }

 288 }


 290 private void DoPayPalRequest(TextWriter objLogFile, int orderId, string currencyCode,

 291    string cardType, string cardNumber, string expDate,

 292    string startDate, int issueNumber, string securityCode,

 293    string firstName, string lastName, string street, string city, string state, string zip, string countryCode,

 294    string centinelPAResStatus, string centinelEnrolled, string centinelCavv, string centinelEciFlag, string centinelXid)

 295 {

 296    //Make payment with PayPal Direct Payments...

 297 }


 299 private void Display3DSecureErrors(string centinelErrorNo, string centinelErrorDesc)

 300 {

 301    lblResultMessage.Text = "Error in 3D secure authentication";

 302    lblResultMessage.Text += "<ul>";

 303    string[] centinelErrorNos = centinelErrorNo.Split(',');

 304    string[] centinelErrorDescs = centinelErrorDesc.Split(',');

 305    for (int i = 0; i < centinelErrorNos.Length; i++)

 306        lblResultMessage.Text += "<li>" + centinelErrorDescs[i] + " (" + centinelErrorNos[i].Trim() + ")" + "</li>";

 307    lblResultMessage.Text += "</ul>";

 308 }