Monday, 25 May 2015

2015 British General Election - Progressive Alliance What If?

Rather off topic for me on this blog but thought interesting nonetheless to publish... I was taken following the personally rather disappointing UK election results by a comment by the Green MP Caroline Lucas reported in the Guardian.

Reflecting on the result, she is quoted as saying...

The system is wrong and we should have electoral reform, but that could be some time coming. So we need other ways to work together in a progressive alliance. Where it is appropriate, only one progressive candidate could stand in a seat – a sort of electoral pact...

Which got me interested in the numbers, in particular considering what the change in the result might have been if such a pact was in place for 2015. Clearly there's a lot of assumption in this - not least the fact that I'm using the hindsight of the actual results to determine what might have been different, which is rather circular. It also assumes that every voter that would have voted say Liberal Democrat would be happy to vote for another party in the "alliance", which is unlikely to be 100% the case. It would certainly need some messaging effort between now and the next election to ensure people accepted that their party preference would be counted, albeit in a different constituency.

So to look at the numbers I downloaded the full set of results for all constituencies in England, Scotland and Wales, that is available from the British Election Study website. I then found all seats the Conservative won and worked out if they would still have won if one of two alliances were in place - firstly between the Liberal Democrats and the Greens, and secondly including Labour as well.

Having found the seats that would have changed hands in this scenario, I then looked at the actual votes cast for the "alliance" parties in these constituencies. I used this to calculate a fair means of allocating which actual party would be given the seat. Again this takes some liberties with timings - in reality the choice of which party was standing (and which was standing aside) in each seat would need to be made before voting of course. But it does seem like a reasonable basis for such a decision if something like this were planned for the next election.

Finally then looking at how this pans out. Firstly, in the scenario where there's an alliance between the Liberal Democrats and the Greens, there's a fairly minor change. Five seats would change hands, which given the breakdown I've used would give the Greens a further MP and the Liberal Democrats a slightly less disastrous loss of MPs to twelve instead of the eight they retained in reality. A significant Conservative majority would still have been the result.

Where things become more dramatic is if Labour is included. Pollitically this may be far-fetched, but electorially it would likely make sense - particularly in areas such as my home region of the South West, where they are third or fourth party at best in most constituencies. In this scenario, the Greens have five MPs, Liberal Democrats three times as many at twenty four and Labour an additional twenty six. The Conservatives remain the largest party but sixty short of an overall majority.

The data and analysis is available for view and review as a Google doc here.

Thursday, 21 May 2015

Sample Site for Merchello with Umbraco

For a while now, since last Codegarden in 2014 in fact, I've been planning to take a look at Merchello, an e-commerce framework for Umbraco. In the past I've made use of uCommerce which worked well for the project but the fact that, like Umbraco, Merchello is open-source is one factor that makes it attractive to consider for future projects.

So on and off for the past few weeks I've built out a very basic store site. Really doesn't look like much and has little in the way of bells and whistles, but implements a number of the core features required for an e-commerce website, namely:

  • Product category and product pages
  • Products with variants, with price and availability indicated for the particular variant selected
  • Basket page with quantity update and option to remove
  • Billing and shipping detail collection
  • Delivery method selection and application of postage & packing costs
  • Selection of payment method
  • Integration with a credit card processing provider, Authorize.Net
  • Presentation of confirmation screen

Nothing in the above is particularly off the beaten track when it comes to Merchello - there are a number of extension points available that I've not dug into yet.

That said, in building this out I did find myself digging into a number of documentation pages and examples (plus forum posts of course) which as is often the case with a fast moving project, sometimes struggle to keep up with the latest features.

I've also followed some stricter MVC patterns I like to use with Umbraco development, which is to use route hijacking, strongly typed POCO view models and mapping with Umbraco Mapper and AutoMapper from my Umbraco content and the Merchello APIs and objects respectively.

For both those reasons think it's worth sharing and may be of use for anyone else making a start with this product. Code is up at GitHub.

Thursday, 11 December 2014

Quick Guide to the Umbraco Source Code

Just a quick and belated cross-link to an article I recently had published on the 24 Days in Umbraco website. Looking at the Umbraco source code from the perspective of someone considering code contributions to the open-source project.

24days.in/umbraco/2014/traveller-guide

Thursday, 13 November 2014

Umbraco Mapper with Nested Archetypes

Following a previous blog post I got a comment asking about dealing with mapping nested Archetypes to a view model using Umbraco Mapper. As I noted in response, it's not that straightforward - the flexibility of the Archetype package means really you have to build something fairly custom for your particular type you are mapping to. Rather than squeeze too much into a comment though, thought the topic worth a post in itself.

As before, looking for a simple example, I've modelled a football match but I've changed it a little to use a nested Archetype. There's a top-level type that contains a single field set to which a text field and two nested score types can be added. The score type has fields for the team and the score.

From the content editor's perspective it looks like this:

I'm looking to map this to a view model that looks like this:

    public class FootballMatch
    {
        public FootballMatch()
        {
            Scores = new List<FootballMatchScore>();
        }

        public string Venue { get; set; }

        public IList<FootballMatchScore> Scores { get; set; }
    }

    public class FootballMatchScore
    {
        public string Team { get; set; }

        public int Score { get; set; }
    }

Which can then be used in the views like this:

    <h3>Today's Match at @Model.TodaysMatch.Venue</h3>

    @foreach (var score in Model.TodaysMatch.Scores)
    {
        <div>@score.Team @score.Score</div>
    }        

The trick as before is to register a custom mapping for the top-level type that will be mapped, like this:

    Mapper.AddCustomMapping(typeof(FootballMatch).FullName, ArchetypeMapper.MapFootballMatch);

And the implementation looks like this:

    public static object MapFootballMatch(IUmbracoMapper mapper, IPublishedContent contentToMapFrom, string propName, bool isRecursive)
    {
        // Get the strongly typed ArchetypeModel from the given property
        var archetype = contentToMapFrom.GetPropertyValue<ArchetypeModel>(propName, isRecursive);
        if (archetype != null)
        {
            // Pull out the first fieldset
            var fieldset = archetype.FirstOrDefault();
            if (fieldset != null)
            {
                // Instantiate an instance of type we are mapping to
                var result = new FootballMatch();

                // Convert the fieldset to a dictionary and map the top-level properties
                var fieldsetAsDictionary = fieldset.Properties.ToDictionary(
                     k => FirstCharToUpper(k.Alias),
                     v => v.Value);
                mapper.Map(fieldsetAsDictionary, result);

                // Given the known aliases for the nested type, loop through and map all the properties of each
                // of these too
                var aliases = new string[] { "homeTeam", "awayTeam" };
                foreach (var alias in aliases)
                {
                    var scoresModel = fieldset.GetValue<ArchetypeModel>(alias);

                    foreach (var innerFieldSet in scoresModel)
                    {
                        var innerFieldsetAsDictionary = innerFieldSet.Properties.ToDictionary(
                            k => FirstCharToUpper(k.Alias),
                            v => v.Value);
                        var score = new FootballMatchScore();
                        mapper.Map(innerFieldsetAsDictionary, score);
                        result.Scores.Add(score);
                    }
                }

                return result;
            }
        }

        return null;
    }

As I say, it would be nice to come up with something more generic to solve this type of problem, and we have managed this to an extent on a recent project that used a number of Archetypes. The flexibility of Archetype does perhaps come at a cost here from a developer perspective. It does seem fairly unavoidable that you'll have to do something quite custom depending on what exactly you are mapping from and too.

Nonetheless once done and set up as a custom mapping within the mapper, any instance of your view model type will be automatically mapped from the given Archetype.

Sunday, 2 November 2014

Using Umbraco Mapper with the Umbraco Grid

Like most people in the Umbraco community I've been keen to download and have a play with Umbraco 7.2, released in beta last week. There's a lot of new features, but the shiniest new toy is without doubt the grid.

This allows a developer to set up a very flexible interface for the editor to manage their content in a layout that closely reflects what they'll see on the website.

When it comes to rendering out the content of the grid, it's very simple. In your view you can just make a call to CurrentPage.GetGridHtml("myGridAlias") and this will spit out the HTML wrapped with bootstrapped elements. You can pass in overrides to use a different template, and create your own.

Nothing wrong with this at all but my preferred approach when rendering out Umbraco content is to create strongly typed view models and, via a hijacked route, map the Umbraco content to this model with the help of the Umbraco Mapper package. Once done, this provides a very clean view model for front-end developers to work with, avoiding the need to work with the more complex Umbraco APIs and dynamics in the views.

Clearly the output of the grid, even if strongly typed, is still going to be relatively "loose" in that we need to be able to handle all that we allow our editors to throw at it. Nonetheless, it seemed to a useful exercise to see how Umbraco Mapper could be used to handle mapping from the Grid data type - and am pleased to say it looks to be rather straightforward.

In order to have something to map too we need our own custom class. The following has been constructed to follow the structure of the JSON format that the grid data is stored in the database:

namespace MyApp.Models
{
    using System.Collections.Generic;
    using System.Web.Mvc;
    using Newtonsoft.Json;

    public class GridContent
    {
        public IList<GridSection> Sections { get; set; }

        public class GridSection
        {
            public int Size { get; set; }

            public IList<GridRow> Rows { get; set; }

            public class GridRow
            {
                public string Name { get; set; }

                public IList<GridArea> Areas { get; set; }

                public class GridArea
                {
                    public IList<GridControl> Controls { get; set; }

                    public class GridControl
                    {
                        public object Value { get; set; }

                        public GridEditor Editor { get; set; }

                        public object TypedValue 
                        {
                            get
                            {
                                switch (Editor.Alias)
                                {
                                    case "headline":
                                    case "quote":
                                        return Value.ToString();
                                    case "rte":
                                        return new MvcHtmlString(Value.ToString());
                                    case "media":
                                        return JsonConvert.DeserializeObject<GridContentMediaValue>(Value.ToString());
                                    default:
                                        return string.Empty;
                                }
                            }
                        }

                        public class GridEditor
                        {
                            public string Alias { get; set; }
                        }
                    }
                }
            }
        }
    }
}

A key property to note is TypedValue. This is used to convert the raw value saved for the grid control element into a typed value appropriate for the content that is stored. This might be a simple string or something more complex like a media item. The implementation of this property would need to be extended to handle all the types we allow our editors to use.

For example here's a complex type representing a media item, again following the JSON structure of the saved information:

namespace MyApp.Models
{
    public class GridContentMediaValue
    {
        public int Id { get; set; }

        public string Image { get; set; }

        public string Thumbnail { get; set; }
    }
}

Then for a given page, I've created a view model that looks like this, following the structure of my document type that uses the grid data type.

namespace MyApp.Models
{
    public class GridPageViewModel
    {
        public string Heading { get; set; }

        public GridContent Grid { get; set; }
    }
}

Umbraco Mapper has a handy feature called custom mappings. These can be set up when the mapper is instantiated - usually in a single location like a base controller - to effectively say, "whenever you are given a given type on a view model to map to, use this function".

With that feature, I'm able to do this in my controller:

namespace MyApp.Controllers
{
    using System.Web.Mvc;
    using Newtonsoft.Json;
    using Umbraco.Core.Models;
    using Umbraco.Web;
    using Umbraco.Web.Models;
    using Umbraco.Web.Mvc;
    using Umbraco72a.Models;
    using Zone.UmbracoMapper;

    public class GridPageController : RenderMvcController
    {
        public override ActionResult Index(RenderModel model)
        {
            var mapper = new UmbracoMapper();
            mapper.AddCustomMapping(typeof(GridContent).FullName, MapGridContent);

            var vm = new GridPageViewModel();
            mapper.Map(CurrentPage, vm);

            return View("Grid", vm);
        }

        public static object MapGridContent(IUmbracoMapper mapper, IPublishedContent contentToMapFrom, string propName, bool isRecursive)
        {
            var gridJson = contentToMapFrom.GetPropertyValue<string>(propName, isRecursive);
            return JsonConvert.DeserializeObject<GridContent>(gridJson);
        }   
 }
}

Which finally allows me to render my grid content in a very clean, strongly typed way, like this:

@model MyApp.Models.GridPageViewModel

<div>
 <h1>@Model.Heading</h1>

 @foreach (var section in Model.Grid.Sections)
 {
  <div>
   @foreach (var row in section.Rows)
   {
    <div>
     @foreach (var area in row.Areas)
     {
      <div>
       @foreach (var control in area.Controls)
       {
        <div class="@control.Editor.Alias">
         @switch (control.Editor.Alias)
         {
          case "media":
           var media = control.TypedValue as GridContentMediaValue;
           <img src="@media.Image" />
           break;
          default:
           <span>@control.TypedValue</span>
           break;
         }                                        
        </div>
       }
      </div>   
     }
    </div>
   }
  </div>
 }
</div>

For me this looks to provide a nice balance between allowing the editor the flexibility that the grid editor provides whilst retaining the strongly typed view model and mark-up generation code in the view. Would love to hear comments from anyone else playing with the grid editor and similarly considering best practices for using it.

Sunday, 21 September 2014

Unit Testing Entity Framework with Effort

Previous posts in this series

Introduction

In my previous post I discussed steps I've been taking with recent application to move more logic and behaviour into a richer domain model. This has a number of advantages, a significant one being ease of testing of this behaviour. As this is coded as methods on a standalone model class, it's very easy to test due to it's lack of dependencies. Nothing needs to be mocked or stubbed; an instance can just be instantiated in the test method and the appropriate methods called and results asserted.

Given the CQRS style approach I've taken elsewhere in the applications, MVC controllers are fairly light and have less value for testing. The main part of the application where logic lies is in the individual query and command handlers. These have a tight dependency on Entity Framework though so are significantly harder to test.

Approaches to Unit Testing Entity Framework

One track I expect a lot of developers have started down is to attempt to mock out the EF context, allowing you to return a known in memory collection or instance instead of hitting the database for your tests. EF 6 provides some good support for this. This is certainly fast, but the downside with this is that querying using LINQ to Objects is not the same as querying LINQ to Entities. There's a good degree of overlap so still some value in this, but writing tests this way runs the risk of having tests that pass in your tests but fail in the application.

Another approach is to simply use the database itself. Most people wouldn't class this a unit test any more, rather an integration one, as it suffers from two downsides. One is that it's going to be a lot slower than using an in-memory data store and the second that it's hard to ensure your data remains consistent. If one test changes the data, you have to make sure you have a fresh schema and data situation restored for further tests, which can take quite a bit of effort to maintain.

Introducing Effort

Effort is an open-source provider that generates an in-memory database from your EF model, and accurately represents (almost) all EF querying operations. Thus in each test you can instantiate a clean data store and run your tests against it. In my case this meant asserting that my queries return the correct results and view models, and commands persist the data as expected.

The big advantage here therefore is removing the brittleness of the tests in ensuring you have a known base schema and data to start each test. It's also fairly fast - I say fairly, as I still find half a second or so for each test, which isn't much but adds up of course if you have a lot.

One other small downside is the "almost" note above. There are some EF operations you can run - within SqlFunctions for example for date operations - that are SQL Server specfic. Effort attempts to remain database agnostic, so can't handle the use of these functions. I get round that by passing a flag indicating whether SQL specific functions are available to the query handler - if they are, as in the application itself, they are used. In the test they are not. So there's a small difference in behaviour between my tests and the real-world application - but for my application at least this is pretty insignificant.

Effort Example

There's good documentation on the Effort Codeplex site for it's set up and use, including seeding data from code or CSV files. I'll just note here some aspects I had to work around slightly differently - in particular as I am using ASP.Net Identity, which means the constructors available for "standard" EF aren't available and some modification is required. Many thanks to the author of the library for his help with this.

Firstly, here's an example of a test. This one is testing a query handler - confirming that given a particular query, the correct view model is populated and returned.

[TestMethod]
public void SurveyQuestionViewModelQueryHandler_WithValidQuery_ReturnsExpectedResults()
{
    // Arrange
    SetUpContextAndTestData();
    var responseId = CreateAndPersistResponse();

    var handler = new QuestionViewModelQueryHandler(Context);
    handler.SqlSpecificFunctionsAvailable = false;
    var worker = Context.OrganisationWorkers.Single(x => x.LastName == "Worker");
    var questionId = Context.Questions.Single(x => x.Code == "A6").Id;

    var query = new QuestionViewModelQuery {
        UserId = worker.Id,
        ResponseId = responseId,
        QuestionId = questionId,
    };

    // Act
    var result = handler.Retrieve(query).Result;

    // Assert
    Assert.AreEqual(questionId, result.QuestionId);
    Assert.AreEqual("Which of these are vegatables?", result.QuestionText);
    Assert.AreEqual("Section 1", result.SectionName);
    Assert.AreEqual(4, result.AnswerOptions.Count());
    
    Assert.AreEqual(0, result.SurveyProgressPercent);
    Assert.AreEqual(42, result.SectionProgressPercent);
}

The first step is to set up the in-memory context using the Effort library via the SetUpContextAndTestData() method, which I'll expand on in a moment. This creates the schema and a set of base data required for all tests. After that we call a helper method to instantiate a single survey response object (an entity in my application) that we need for this specific test.

The rest of the Arrange part of the test is involved with instantiating a handler object, setting the property on the handler that indicates of SQL specific functions can be called, and looking up some values from the in-memory database to create a query object.

The single line in the Act simply calls the Retrieve method on the handler and retrieves the result. In the Assert section we call several asserts, to ensure the resulting view model is populated as we are expecting.

Test Setup

Going back to the arrange steps, the following code is used to instantiate the in-memory database using the Effort library.

protected void SetUpContextAndTestData()
{
    InitContext();
    TestDataSeeder.SeedData(Context, true);
}

private void InitContext()
{
    var dataFolder = AppDomain.CurrentDomain.BaseDirectory;

    var connection = Effort.DbConnectionFactory.CreateTransient();
    var dummyContext = new ApplicationDbContext();
    var builder = new DbModelBuilder();

    var m = typeof(ApplicationDbContext).GetMethod("ConfigureModel", BindingFlags.NonPublic | BindingFlags.Instance);
    m.Invoke(dummyContext, new object[] { builder, false });

    var model = builder.Build(connection).Compile();

    Context = new ApplicationDbContext(connection, model, false);
    Context.Configuration.AutoDetectChangesEnabled = true;
}

The call to InitContext sets up the schema using a variation of the standard Effort instantiation to work with the constructors available in ASP.Net Identity. A dummy context is first created and then the Effor in-memory model is built from that. Please note if you aren't using ASP.Net Identity then you should follow the set-up code as detailed on the Codeplex site.

We then make a call to a method responsible for seeding the data. This actually leverages the existing code for seeding data following an EF migration. It passes a flag to indicate to the seeding method whether on not to call any SQL server specific migrations. For example I had some code to override conventions for various date fields to use the smalldatetime data type - as this is SQL specific, Effort won't be able to work with this. But as this setting isn't relevant for tests, it can safely be ignored.

Conclusion

I'd certainly recommend developers looking to test EF methods take a look at Effort. Pun intended, in using it there's not much effort involved in getting the parts of your application that traditionally would be quite hard to test, under your unit test coverage.

Sunday, 14 September 2014

Rich Domain Models and Entity Framework

Previous posts in this series

Working with a Rich Domain Model and Entity Framework

As described in previous posts in this series, I'm documenting a few changes to the way I've recently been building out ASP.Net MVC applications, following various influencers from blogs and books. One area I was keen to look at was creating a rich domain model - inspired by the Eric Evans classic book and some posts by Julie Lerman and Jimmy Bogard.

Previously I would have worked more with what is known as an anaemic domain model - which isn't quite as bad as it sounds... but means that when creating entities for use in Entity Framework Code First, they would in the most part be made of a simple set of properties with public getters and setters. With a rich domain model, we look to push more behaviour into these model classes and provide tighter means of control over their use by calling code from the rest of the application.

There are a number of means to do this - in this blog post I describe a number I've used.

Control via constructors

One first method is to remove the parameterless constructor that you might otherwise have, thus preventing calling code from simply newing up an instance of the class, without necessarily setting some required properties. In fact you can't completely remove it - EF requires it - but you can make it private.

Once that's in place you can create one or more constructors that will enforce the calling code to provide whatever properties are deemed required and thus not create what might by the terms of the application be deemed an invalid object. Here's an example from a survey application that defines an entity for a question:


    private Question()
    {
        Options = new List<QuestionOption>();
    }

    public Question(string code, string text, int orderInSection,
        QuestionSection section, QuestionType type)
        : this()
    {
        Code = code;
        Text = text;
        OrderInSection = orderInSection;
        Section = section;
        Type = type;
    }

Control via properties

A second step to control more fully how your class is used, is to make the property setters private. This will prevent calling code from simply setting values for them, instead you provide methods where appropriate groups of properties can be set together, and validation can be applied.

This example from the same application controls the setting of an allowable range for a numeric question's answer:


    public int? NumericalMin { get; private set; }

    public int? NumericalMax { get; private set; }

    public void SetNumericalRange(int min, int max)
    {
        if (min > max)
        {
            throw new ArgumentException(“Max parameter must be greater than the min parameter.");
        }

        NumericalMin = min;
        NumericalMax = max;
    }

Logic in the model

The last method to discuss involves pushing more of the application's logic into the rich domain model, rather than having this in various services or other components of the application. There's some major advantages to this, particularly when compared to making additional database requests to handle particular behaviours. The code is arguably easier to write and maintain, and certainly easier to test as you can simply create instances of the classes and test the behaviour methods.

You do need to take care though - in order to fulfil certain behaviours it might be necessary to instantiate your object from the database with a more fully populated object graph that you otherwise normally would. So it's important to bear in mind the database requests that your method will require.

This final example illustrates that balance and why it's certainly worth looking to move behaviour into model methods when you can. With the survey application, we have multiple choice questions that you can obtain a score from based on which options you select in your answer. I required a means of calculating the maximum score available on a question.

For a multiple select question with check-boxes, the maximum score would be the total associated with each of the options. For a single select though, where you can only select one via radio buttons, the maximum score would be the option that had the highest score associated with it.

The following code examples illustrate how this logic can be set up as a method of the Question class, which will be valid so long as the related Options for the question are also loaded. If that is the case though, the code is very easy to write, maintain and test.


    public int GetMaximumAvailableScore()
    {
        if (Type.Id == (int)QuestionTypeId.MultipleSelect)
        { 
            return Options.Sum(x => x.Score);
        }
        else if (Type.Id == (int)QuestionTypeId.SingleSelect)
        {
            return Options.Max(x => x.Score);
        }
        else
        {
            return 0;
        }
    }

    [TestMethod]
    public void Question_GetMaximumAvailableScoreForMultiSelect_ReturnsCorrectValue()
    {
        // Arrange
        var question = CreateMultiSelectQuestion();

        // Act
        var maxScore = question.GetMaximumAvailableScore();

        // Assert
        Assert.AreEqual(8, maxScore);
    }