That agile feeling

Do you remember being a junior developer? Or maybe, before you were a developer, being “that one who knows computers and Excel and stuff” in the office?

Remember how when someone said “it would be great if I didn’t have to do [some manual job that could be easily scripted]”, and you came back the next day with a hacked-together Excel macro or Access DB or shell script that did it for them?

Remember how pleased they were?

And, do you remember how they then said, “but it would be great if it could do [some other thing]” and you came back to them an hour later and showed them a magic button that did just that?

Remember how good that made you feel?

That right there is a short feedback cycle.

That’s what we’re trying to achieve every day with agile.

Using PowerShell to update DotNet Core version numbers

The problem: I have a DotNet Core package that I would like to build and publish to an internal NuGet feed using Visual Studio Team Services. If I was using a .nuspec or .csproj file, I could set the version number of the package during the build, but DotNet Core doesn’t let me do this.

The solution: Write a very small PowerShell script that will update project.json with a new version number, then plug that into the build.

The script

param(
[string]$path,
[string]$version
)
$project = Get-Content $path | ConvertFrom-Json
$project.version = $version
ConvertTo-Json -InputObject $project -Depth 32 | Out-File $path

So, what’s going on here? Not a lot: we’re reading project.json into an object, then modifying the version property (if you do this from command-line, you even get intellisense for project.json!), and then finally re-serializing the object to over-write the original file.

Annoyingly, this will screw up the formatting of project.json, but you don’t actually care about this because this file will only live for the duration of the build, and will never get back into source control.

Plugging it into VSTS

I’ve added a build variable to store the desired version number:

Package.Version = 1.0.$(Build.BuildId)-*

Then I’ve added a new PowerShell build step into the build as the first step. I’ve saved my file as Set-DotNetVersion.ps1, so we set up the following arguments:

vsts_ps_setversionnumber

That’s it! If you then call dotnet pack on your library, it’ll create a NuGet package with the version number that you specified above. My full build pipeline is:

  • PowerShell Script
  • Dotnet restore
  • Dotnet pack
  • Publish build artifacts

Custom Authentication in ASP.NET MVC Core

ASP.NET Core has really good out-of-the-box support for authorization and authentication via ASP.NET Identity. However, sometimes that’s not enough, and you need to roll your own logic.

In my case, I was writing an API for sending emails, which many different clients would connect to. Each client would be given an “API Key” that would identify and authorize their requests. I could have implemented this using HTTP Basic authentication, or using OAuth tokens, but instead I thought it’d be good to get my hands dirty in the base classes.

So, the scenario is that a client sends a request with a known header value, as in:

$Headers = @{
    Authorization = "abcdefg1234567"
}

$Data = @{
    To = "you@example.com";
    From = "me@example.com";
    Subject = "Hello!";
    Body = "How you doing?"
}

Invoke-WebRequest -Uri "https://myservice.com/emails/send" -Headers $Headers -Data $Data

The actual value of the key and the logic behind it isn’t relevant to this article (I’m using RSA private keys under the hood), but the point is that we have a method in our code to verify the authentication key.

To do this, we need to implement custom authorization middleware. This will be composed of three parts:

  • The middleware class
  • The middleware options
  • The authentication handler

The middleware inherits from Microsoft.AspNetCore.Authentication.AuthenticationMiddleware<TOptions>, where TOptions inherits from Microsoft.AspNetCore.Builder.AuthenticationOptions. It’s responsible for building new instances of our authentication handler, and for injecting any required dependencies into the handler. For the purposes of this example, let’s suppose that I have an interface to validate my API keys, like so:

public interface IApiKeyValidator
{
    Task<bool> ValidateAsync(string apiKey);
}

Let’s assume that I’ve implemented this somehow (maybe using Azure KeyVault), so my challenge now is to hook my custom logic up to MVC, so that its built-in authentication can kick in. The end result is that I can decorate my controllers with:

[Authorize(ActiveAuthenticationSchemes = "apikey")]

You can think of the middleware as the glue that binds the MVC framework with our business logic. We could start by writing:

public class ApiKeyAuthenticationOptions : AuthenticationOptions
{
    public const string DefaultHeaderName = "Authorization";
    public string HeaderName { get; set; } = DefaultHeaderName;
}

public class ApiKeyAuthenticationMiddleware : AuthenticationMiddleware<ApiKeyAuthenticationOptions>
{
    private IApiKeyValidator _validator;
    public ApiKeyAuthenticationMiddleware(
       IApiKeyValidator validator,  // custom dependency
       RequestDelegate next,
       IOptions<ApiKeyAuthenticationOptions> options,
       ILoggerFactory loggerFactory,
       UrlEncoder encoder)
       : base(next, options, loggerFactory, encoder)
    {
        _validator = validator;
    }

    protected override AuthenticationHandler<ApiKeyAuthenticationOptions> CreateHandler()
    {
        return new ApiKeyAuthenticationHandler(_validator);
    }
}

This is all just glue code. The real meat lies in ApiKeyAuthenticationHandler, which can be stubbed out as follows:

public class ApiKeyAuthenticationHandler : AuthenticationHandler<ApiKeyAuthenticationOptions>
{
    private IApiKeyValidator _validator;
    public ApiKeyAuthenticationHandler(IApiKeyValidator validator)
    {
        _validator = validator;
    }

    protected override Task<AuthenticateResult> HandleAuthenticateAsync()
    {
        throw new NotImplementedException();
    }
}

As you can probably tell, our logic lives in the HandleAuthenticateAsync method. In here, we can basically do whatever we want – we can inject any dependency that the application knows about through the middleware, and the handler has access to the full request context. We could check the URL, querystring, form values, anything. For simplicity, here’s a really cut-down implementation:

StringValues headerValue;
if (!Context.Headers.TryGetValue(Options.HeaderName, out headerValue))
{
    return AuthenticateResult.Fail("Missing or malformed 'Authorization' header.");
}

var apiKey = headerValue.First();
if (!_validator.ValidateAsync(apiKey))
{
    return AuthenticateResult.Fail("Invalid API key.");
}

// success! Now we just need to create the auth ticket
var identity = new ClaimsIdentity("apikey"); // the name of our auth scheme
// you could add any custom claims here
var ticket = new AuthenticationTicket(new ClaimsPrincipal(identity), null, "apikey");
return AuthenticateResult.Success(ticket);

I’ve kept this example simple, but at this point you have to power to construct any sort of claims identity that you like, with as much or as little information as you need.

Finally, we just need to tell MVC about this middleware in Startup.cs:

public void ConfigureServices(IServiceCollection services)
{
    // the implementation of this is left to the reader's imagination
    services.AddTransient<IApiKeyValidator, MyApiKeyValidatorImpl>();
    // etc...
}

public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
{
    app.UseMiddleware<ApiKeyAuthenticationMiddleware>();
    // etc...
}

And that’s it! Custom business logic, dependency injection and MVC all working together beautifully.

Typescript interfaces for C# developers

Or, how not to make the mistakes I made with Typescript.

As a long-time .NET developer, I took the plunge into TypeScript a couple of years ago after much uhm-ing and ah-ing. I learned that you could write interfaces in TypeScript, and I immediately starting doing them wrong.

In .NET, an interface is something that you expect to be implemented in a class. For example:

public interface IFoo {
    string Bar { get; }
    string Baz(string bar);
}

public class FunkyFoo {
    public string Bar { get; set; }
    public void Baz(string bar) => $"Hello {bar}";
}

You might code against the interface, but you would only really create interfaces if you expected several different concrete classes to implement that contract.

When I started writing TypeScript, I wrote interfaces in much the same fashion:

interface IFoo {
    string Bar;
    Baz(bar: string);
}

class FunkyFoo {
    public Bar: string;
    public Baz(bar: string): string {
        return `Hello ${bar}`;
    }
}

The problem is that, while this works, it’s not really giving you the full benefit of TS interfaces. TypeScript, being a super-set of JavaScript, is a functional language – unlike .NET, the classes aren’t the most important thing in it. Using the approach above, I was writing code like:

let foo: Array<IFoo>;
$.getJSON("foo/bar", json => {
    for (let i = 0; i < json.length; i++) {
        foo.push(new FunkyFoo(json[i]));
    }
}

Ugh. Horrible and messy. If I’d used interfaces properly, I’d have something like this:

interface Foo {
  bar: string;
  baz: string;
}

// ...

let foo: Array<Foo>;
$.getJSON("foo/bar", json => {
    foo = json;
});

The realisation that dawned on me was that interfaces are just data contracts – nothing more. Due to the loose typing of JavaScript, I could quite happily say that an object was of type Foo, without doing any conversion.

So, what should I have been using interfaces for? Well, firstly, data contracts as seen above – basically just a way to define what data I expect to get from/send to the server. Secondly, for grouping sets of repeated parameters, as in:

function postComment(email: string, firstName: string, lastName: string, comment: string) {
  // etc
}

// becomes:
interface UserInfo {
    email: string;
    firstName: string;
    lastName: string;
}

// now the method is much cleaner,
// and we can re-use these parameters
function postComment(user: UserInfo, comment: string) {
  // etc
}

I’ve ended up thinking of interfaces as more akin to records in F# – just basic DTOs. Doing this has actually made my code better – now I have a clean separation between data and functionality, rather than mixing both in the same structure.

The disaster of OOP, and what we can salvage

Object Oriented Programming is a disaster, or at least that’s what Brian Will thinks, and I’m not about to disagree. I share his disillusionment with OOP, and I have always had a sneaking admiration for the PHP and JavaScript developers who just seem to get stuff done much faster than their static-typed object brethren.

However, one paragraph did stick out:

In [the] original vision, an object-oriented program is composed of a graph of objects, each an island of state unto itself. Other objects do not read or write the state of other objects directly but instead send messages.

Just because OOP as we write it today hasn’t produced the benefits we sought, doesn’t mean that there is no value in the original concepts. The difference between now and then is one of scale: how many of you out there are actually writing single programs, where a system is composed of objects that all live in a single app domain and communicate via messages?

I don’t think I’ve ever written such an app (it’s much harder in the web world in any case, where the app simply has no state from one request to the next). Where I have been writing actual, run-in-memory-UI-and-everything applications, they’ve been small utility apps that I’ve thrown together using RAD, that simply wrapped functionality that already existed in databases or services.

That, right there, is the part where OOP hasn’t failed – it has just, like the Roman Empire, transformed into something else. Instead of objects, think services – think how an enterprise system is composed of many disparate systems, services and databases, each of which are tied together by some form of message. In the worst case, they share state promiscuously and become an un-maintainable tangle, but in the best cases, we have services that sit serenely in the heavens and send messages back and forth to each other.

Or we could just hack it together with PHP and buy a bigger server.

When to Inject a Dependency

Let’s take a simple scenario: say that you’re developing a holiday request system (because the previous MS Access-cum-Excel-cum-Word jalopy that the boss wrote ten years ago has finally corrupted beyond the point of no return). You’re given the basic parameters, and then, if you’re like me, you start pseudo-coding:

class HolidayManager
{
  void RequestHoliday(Employee emp, DateTime start, DateTime end)
  {
    // TODO: sensible error messages and basic validation stuff
    if (start < DateTime.Now) throw new ArgumentOutOfRangeException();
    if (end < start) throw new ArgumentOutOfRangeException();

    // does the user have any holiday available?
    if (database.GetHolidaysRemaining(emp) == 0)
      throw new OutOfHolidayException("TODO: should we use a return code?");

    // assume that this call returns us a unique ID for this holiday
    var holidayId = database.AddHoliday(emp, start, end, HolidayStatus.Requested);

    // tell the user
    emailer.SendConfirmation(emp, start, end);

    // tell their line manager
    var manager = database.GetLineManager(emp);
    if (manager != null) // they might be the Big Boss
      emailer.SendRequestForApprovalEmail(emp, start, finish, holidayId);
    else
      this.ApproveHoliday(holidayId, HolidayStatus.Approved);
  }

  void ApproveHoliday(int id, HolidayStatus status)
  {
    // assume this struct has info about the holiday
    var holiday = database.GetHoliday(id);
    if (holiday.Status == HolidayStatus.Requested)
    {
        database.SetApproval(id, status);
        int daysLeft = database.CalculateRemainingDays(holiday.Employee);

        // send email to the requester (assume that holiday includes an email address)
        emailer.SendApprovalEmail(holiday, status, daysLeft);
    }
    else
      throw new InvalidOperationException("already approved");
  }
}

What we have here is a basic sketch of our business logic – we don’t care about how the database works, and we don’t care how emails go out, because that’s not important to us at this stage.

If, whilst pseudo-coding, you find yourself writing GetStuffFromDatabase or somesortofqueue.Push(msg), then congratulations – you’ve identified your dependencies!

Whenever you find yourself writing a quick note to cover up what you know will be a complicated process (retrieving data from a store, passing messages to a queue, interacting with a third party service, etc.), then this is an excellent candidate for a dependency.

Looking at our example, we’ve got two dependencies already:

  • database
  • emailer

Now, my point here is not to design the best holiday application imaginable – the code above is just a sketch. My point is that we’ve now identified some application boundaries, namely the boundary between business logic and services.

Let’s take this a little further: in our emailer implementation, we might have further dependencies. Say that our emailer implementation needs to format an email (given data and a template) and then send it. We might switch the formatter at some point in the future (using RazorEngine, NVelocity or just string.Format) and we might change the delivery mechanism (because the business now has a central email queue, and we need to submit messages to that instead of using SMTP directly). In this case, the business logic of the email engine is:

class HolidayEmailer {
  IEmailFormatter formatter;
  IEmailSender sender;
  IEmailLogger logger;

  void SendEmail(someObject email)
  {
    var message = new System.Net.MailMessage();
    message.To.Add(email.To);
    message.Subject = email.Subject;
    message.Body = formatter.Format(email);

    sender.Send(message);
    logger.Log(message);
  }
}

What this means is that we have many levels of DI – from the point of view of our holiday app, it has an abstraction that it can call to send emails, and we can unit test our object against a mock. We can also test our emailer implementation, because we’ve abstracted the formatting, sending and logging, which means that we can test what happens if any of these fail, or produce weird results.

Don’t outsource your business logic

Just another Monday morning, and Dwayne The Developer is ready to start cranking code. The business needs an object to make, track and approve holiday requests. Dwayne, being a conscientious sort, and having heard that Interfaces Are A Good Thing™, starts by writing:

interface IHolidays {
  // returns the holiday ID from the database
  int Request(Employee emp, DateTime start, DateTime finish);
  void Approve(Employee manager, int holidayId);
  void Reject(Employee manager, int holidayId, string reason);
  bool IsApproved(int holidayId);
}

Yay, Liskov and all that good stuff, right?

Wrong.

Sharon The Architect happens to be wandering by Dwayne’s desk, and sees what he’s writing. She asks why he’s made the business object an interface, and, not being satisfied with a few incoherent mumbles about SOLID, points out that Dwayne has just outsourced the business logic of this module, without gaining any of the benefits of dependency injection or inversion of control.

Interfaces are much like outsourcing: a software house might outsource non-core tasks such as designing the public website, internationalising the UI, designing icons or writing a custom SAP integration module. A sensible software house would seldom outsource its “secret sauce” – the core code that makes or breaks the app (I’ve seen some people do this, and it never ends well).

An interface essentially outsources a task: it says “here is a method that does X; I don’t care who does it or how, just do it and deliver me the result”.

There point is that there is no point in ever having a different implementation of IHolidays, because that object IS the application.

These things are good candidates for wrapping inside interfaces:

  • Database calls
  • SMTP and SMS APIs
  • Message queueing systems
  • File systems
  • Third-party APIs

Each of these is something with complicated internal logic that isn’t relevant to the business logic that your real objects are implementing – think of them as “stuff that does stuff and returns a result, where I don’t really know or care how that stuff happens”.

Because they aren’t part of the business logic, they’re excellent candidates for mocking during unit tests. We don’t need to test whether System.IO.File.Delete works; we just need to know what happens to our code when it does or doesn’t.

So what should our holiday object look like? Tune in next time to find out…

On Elephant Traps

Elephant trap (software): ground that looks firm and takes one’s initial weight, but which subsequently drops out from under you, leaving you in a dark pit, staring up at the disapproving face of your project manager as they scatter the ashes of your estimates over you like sarcastic confetti.

The difference between real elephant traps and software ones is that the software traps aren’t intentional (and don’t actually trap elephants, which is Bad™). Instead, software traps are created by our desire to avoid risk, but in the process create massive unknown risks that far outweigh the initial risk we thought we were avoiding.

A real example from my career was when we decided to move from using GUIDs as primary keys to integers. This was a demonstrably Good Thing To Do, but it was also a huge change: our application logic lived primarily in stored procedures, and all of these took GUIDs as keys (not to mention our ADO.NET application code, web pages, etc). In our desire to avoid risk, we made a fateful decision: fudge it.

Where possible, we continued to use the GUIDs, and left the columns in place in the database. We created SQL functions to look up the real integer keys from GUIDs, and vice-versa, and peppered our stored procedures with these; meanwhile, the website kept passing GUIDs from page to page.

What we thought we were doing was avoiding risk:

change = risk ∴ fewer LoC changed = less risk

Unfortunately, this is false, and in fact is one of the main causes of risk in software. By being clever, we made the system less obvious – new developers had to get their heads around the fact that the keys passed via the website weren’t the actual database keys. This might have been OK, except that newer pages bypassed the GUIDs and just used integers, not to mention some pages that had to be modified to use integers to get better performance. Gradually, stored procedures evolved that could take either a GUID or an integer as a key – and would use one if the other was null (except when developers got confused, passed both in and the procedure crashed).

Smoke & Mirrors

smokemirrorsIf code is less obvious, it takes longer for a developer to fully understand it – and the result is that developers are more likely to inadvertently break stuff because they don’t realise that the layer of smoke-and-mirrors is pretending that it’s something else to satisfy the needs of an old bit of VB6. In effect, we’re disguising complexity, and thus we create elephant traps.

As I see it, this is a legacy of old, poorly-understood code. In such projects, the application is so poorly known that developers cannot be sure what the effects of making code changes will be. The natural instinct therefore is to make changes in such a way as to change the code as little as possible – in practise, this usually means pushing changes down to the database, because a) it’s the lowest level, and b) it can be changed live, so if it breaks it can be fixed without a build.

Change != Risk

The problem we often face is that we have no way of knowing what effects our changes will have, and so our natural instinct is to make fewer changes. This results in brittle software and strangles innovation. No-one is going to risk adding a cool new feature if they’re too scared to change the page layout because no-one understands how that custom JavaScript-generated CSS engine resizes page content.

We should make breaking changes as soon as possible: the sooner we break things, the sooner we know the scale of the problem we face, and the better we understand the system (breaking something is, despite what Gandalf said, a fantastic way of finding out what something is).

More Tests = Less Risk

The answer is tests. An application with a comprehensive suite of unit tests, integration tests, automated UI tests and a manual tester has a high degree of safety – a developer just has to run the unit tests to see whether they’ve broken anything fundamental. To get the best out of this, the application should have a short feedback cycle – if it takes a day to get it running locally, developers will make as few changes as possible. If it’s weeks between builds and integration tests, then people will start resenting fixing weeks-old mistakes because they’re now working on something else (and in the worst case, these bugs will become Business As Usual).

A well-run project with unit tests that can be run in seconds, continuous integration builds and tests on every push, and regular builds to testing with, at the very least, a five minute manual smoke test, is one where developers are not afraid to make changes.

Don’t be mean to elephants! They never forget…