Designing better Web API’s – Part I

This was originally published on Oct-22-2014

At the core of many value added offerings on the internet is the ability to compose that value using web based programmatic interfaces. This paper looks at how to expose the atomic (‘composable’) or higher value based (composed) interfaces using Standards based Web API protocols. We will also discuss how to ‘productionalize’ this with the implementation of key aspects such as security, versioning, self-describing and the use of Hypermedia (HATEOAS).

To begin with, the basic question that I typically ask myself is, “what am I providing and how do I expect my consumers to consume my API?” I also try to envision any related API’s that I should be exposing as part of the current API, for instance, if I am exposing a Financial Portfolio API, I may also wish to consider related Accounts API and Transactions (Debit/Credit Posts) API. I also need to decide whether I am exposing this related API as Hypermedia based relationships or using standard HTTP verbs.

What makes an API?

Well, for starters I need to know how I will be exposing individual operations or distinct pieces of functionality within my Web API. These operations are also called URI Endpoints or the Interface Addresses. Additionally I will expose the type of action that I want to perform on some data using this URI Endpoint. The common HTTP verbs fit very well with these actions and include GET, PUT, POST, DELETE. So I could have an operation called, “GetAccountDetails” for instance where GET is the verb.

What makes a RESTful API?

RESTful API’s are very much like the RPC type of API described above with the distinction that they expose only the objects (Data) that they are engaged in (e.g. Account) and expose operations using verbs as resources. In particular, they use HTTP verbs (GET, PUT, DELETE, POST) as resources. It is also very HTTP centric and uses specific HTTP commands to get to the results from operations. Another important thing to note is that RESTful API dictates that the server be stateless in that the server should not maintain or hold on to any state information between multiple calls to it. RESTful API will also not dictate the format of the results that it returns to its consumers. Instead, it allows its consumers to engage in content negotiation with it and specify the format that it is interested in (e.g. XML, JSON). In addition, RESTful API may include in the payload of the results, links to related operations and API as discussed in the second paragraph above. It does this using something called, “Hypermedia.As.The.Engine.Of.Media.State”  or HATEOAS.

So we know that RESTful API uses Resources. Resources are a representation of real world objects or entities such as Account, Customer, etc. Resources also have the notion of relationships of different objects that are usually nested like a tree. (e.g. Financial Portfolio à Accounts à Transactions à Products). These relationships are not like an Entity Relationship that is Normalized, instead it tends to be hierarchical in nature.

These Resources are exposed as Uniform Resource Indicators or URI paths and may contain Query Strings, but only for non-data elements and operations. These Query Strings are usually used for utilitarian purposes such as Pagination, Sorting, Filtering, Grouping, etc.

RESTful Basics (yes there is more)

A great start to this is a brief read on Chapter 5 of Roy Fielding’s Doctoral dissertation (copyright: Fielding, Roy Thomas. Architectural Styles and the Design of Network-based Software Architectures. Doctoral dissertation, University of California, Irvine, 2000.) at

In particular,

  1. Separation of the client and the server
  2. Each request must be stateless
  3. We must cache the data results as much as possible, i.e. wherever they are cacheable.
  4. The API must have a uniform interface or a regular pattern on how to get to the resources.


Can we image the “Web” without the Anchor tag or hyperlinks? In the same spirit, or design, hypermedia or links is an ideal way to allow our consumers to navigate through our API and its various resources. These are in particular links in resources that describe how to further manipulate data. In the end, we create self-describing and documenting API.

So, when should I be using REST?

RESTful API’s have a lot going for them, they tend to be easy to consume and easy to maintain. They also afford fantastic scalability due to its stateless nature and performance due to its cacheability.

However if your operations cannot be represented as distinct resources or you need to hold state information between calls, you may wish to explore other design patterns such as Remote Procedure Calls. In general, they are a great way to expose resources over the HTTP protocol if your consumers are going to naturally be consuming your API as such.

The Design Process

Here we will look deeper into the design of RESTful WEB API’s including how to use the HTTP verbs, status codes, associations, ETags. We will also look into some techniques on how to craft the results that we will be returning and how to work with pagination and returning partial results, and how to work with non-resource based API’s.

  1. Nouns, not Verbs: To start with, as mentioned earlier, use objects or Nouns instead of Verbs to describe the URI endpoints and use their plural versions. So for instance, instead of GetAccount, define Accounts.
  2. Indentifiers: next, we will need to identify individual items within the collection of results that we return. To do this, we will be using Identifiers (which does not need to be a relational database primary key) e.g.
  3. Operations or Verbs: Here is a mapping of HTTP verbs used with resources to commonly understood actions against a database.


  1. Status Codes: Certain calls to the API will need to return a status of some sort. Typical examples include a success or an error condition. The standard way to do this with RESTful Web API is to use HTTP Status Codes. A quick sampling includes,
Status Code Description Status Code Description Status Code Description
200 OK 201 Created 202 Accepted
302 Found 304 Not Modified 307 Temp Redirect
308 Perm. Redirect 400 Bad Request 401 Not Authorized
403 Forbidden 404 Not Found 405 Method not Allowed
409 Conflict 500 Internal Server Error    


  1. Above all, keeping the status codes (how many we use) simple is key.

Returning the desired Content Type (ACCEPT HEADERS)

We can also support multiple ways to format our results such as XML or JSON and indicate the ones that we support by using the HTTP Accept Header

Accept: text/xml, application/json, application/jsonp, application/xml+rss, application/xml+atom

The ordering above will tell the server that the first content type (text/xml above) is the default if not specified.

If your consumer will not be able to modify Accept Headers (e.g. they are calling your API from an application such as Excel) then another way to identify the desired content type could be to use a Query String parameter called e.g. format and specifying the content type (this is not recommended, use only if all else is not possible to do).


As described earlier, typically we will have related objects or entities to the resource that we are working with. The example we used earlier was that when working with a Financial Portfolio resource, we may also have related resources for Accounts, Transactions and Products.

It is very useful to identify these associations and the way to do this is to use URI Navigation. So for instance, to get all Accounts for a given Financial Portfolio,, we can use a related URI Navigation such as Note how the Accounts URI is associated with or contextual to the portfolio instance.

We can obviously further enhance our API calls using Filters, Pagination, Searches, etc., using Query Strings such as

Entity Tags or ETags (Caching Support)

Entity Tags allow us to tag our results with a pseudo version number or an ETag. A caching directive can then use this ETag to ascertain when to refresh the cache (if the ETag changes). Specifically this is facilitated by the consumer where the client specifies a HTTP Header directive, “If-None-Match” with no HTTP body such as,

If-None-Match: “134679235689124578”

This tells the server to only send results if it does not match the previous ETag specified in the If-None-Match. In this case the server will send back a Status Code of 304 or Not Modified.

Other ways to use the ETag would be to test if information has changed on the server prior to updating the back-end (with a PUT call) using a, “If-Match” HTTP Header directive. If the If-Match fails, the server will send back a status code of 412 or “Precondition Failed” as specified in the Header.

ETags also come in 2 flavors. The regular strength brew and a weak tag. A regular ETag signifies that the results are completely identical, byte-for-byte whereas a weak tag (noted with a “W/” prefix) signifies that the results may not be structurally identical but mean the same, i.e. semantically identical.

ETags are included in the HTTP Header,

ETag: W/”134679235689124578”

What to return?

Your API may return different types of results including, a single result, or a collection as results.

Represent single results as simple objects, including complex types. The result keys or element names should have a consistent naming convention (I like to use camelCase personally).

When returning collections, it is good to separate the metadata about the collection from the actual results. e.g.

“resultCount”: 682,
“id”: 124578,
“firstName”: “Aref”,


If our result contains a lot of information, it is good to provide pagination support. There are many ways to accomplish this and my preferred way is to use Query Strings. Let us first look at what the client would need to implement pagination.

Page Size: an indication of how many items to return per page,

Previous Link: Link to the previous page,

Next Link: Link to the next page,

First Link: Link to the first page,

Last Link: Link to the last page.

A possible client call to our API could look like,

Which tells the server that the consumer wishes to get the first page with 10 items. The server could then possibly return in the metadata part of the results,

“pagination”: [{
“totalCount”: 16973,

“result”: {…}]

Note: Another way to do this is to use Take and Skip instead of Page Size. So the client could tell the server to take the next 10 result items after skipping the first 50 items.

Picking specific fields or members (Partial Items)

One way to reduce the network traffic is for the client to specify to the server what fields it is interested in and to only send back results for those fields.

One way to do this is to use Query String parameters such as,field3,field4,field7

This may work in simple cases but be mindful of querystring length limitations and this gets tricky with complex nesting of fields.

We can also partially update a resource by using the HTTP PATCH verb. This is usually used in conjunction with ETags to check for concurrency of the specified fields and values.


First of all, we will assume that you have conducted exercises in data classification, threat classification and availability classification of your resources. This will dictate whether you require to secure your API or not. Security needs to be a foremost architectural concern in our API design and not an afterthought.

There are many types of threats that we would like protection from. Some of these include,

Internal or physical attacks on the servers – these could be brute-force intrusion and penetration attacks. This could also include physical destruction of your assets.

Man-in-the-middle attacks – eavesdroppers that are listening in on the traffic flow for both the data and the meta-data of the traffic.

Malevolent users and hackers – do not trust the browser implicitly.

We may need to protect the data at rest and in transit. We will typically want to protect the data in transit using some sort of TLS such as SSL. Additionally within our API implementation we may want to ensure additional levels of detections such as IP and domain filtering, authentication and authorization.

Cross Domain Security (where are the calls to your API from?)

By design, an AJAX call from the browser in a given domain will not be allowed into a different domain. We will have to design for it if this is intended. For instance if our API is only for internal consumption then there is no need to enable cross domain calls into our API.

If we need to support Cross Domain calls then we have 2 options.

  1. Support JSONP as a format (remember for this your URI has to include a callback Query String to the JavaScript function exposed as part of JSONP).

JSONP is essentially JSON with (Javascript) Padding. The container of the payload is a Javascript surrounded body but the “innards” is really JSON. This essentially presents the result as a Javascript that is then executed. This executed function then fetches the data that the browser needs. This is okay as the browser is used to receiving Javascript from multiple domains such as Google APIs, JQuery, Getbootstrap, etc. Here is an example,

  1. We have a Javascript function shipped to the client called refreshAccounts(data){…} This will be expecting some data (input parameter) that will be coming from a server located in a cross domain.
  2. The client AJAX will issue a call to a RESTful API resource, say: the client will also specify the Accept Header to be of type, Accept: application/javascript
  3. This will return something like http://”a
  4. Then the internally defined Javascript function refreshAccounts(data) would receive the payload like: refreshAccounts({“accountNumber”: 124578, “balance”: 136987.2345});

This is probably best explained as a sequence diagram as illustrated below:



There are obviously security implications to this as we are intentionally bypassing the protection provided by the browsers’ implementation of the, “Same Origin Policy”. This is best elucidated at stackexchange (

Implementing Web API Security using the .NET Framework version 2

.NET Web API v2 framework is independent of the hosting container. That is, it is not dependent on ASP.NET (System.Web) or IIS. Back in Web API v1, Authentication relied on IIS (WebModule) and added additional options to host this ourselves in the hosting layer or the HttpMessageHandler layer of the authentication pipeline. As much flexibility as HttpMessageHandler offered, it was somewhat disconnected from the container (IIS). Authorization could be done directly within the Controller or using Autorization Filters.


Version 2, introduced a new hosting infrastructure called OWIN and a specific implementation with the Katana Project ( In this new scheme, The OWIN adapter will abstract away all security interests from the underlying container (IIS). In version 2, the MessageHandler pattern is deprecated and instead, we are encouraged to use Filters for Authentication and Authorization. Using the Authentication Filters, we can move the authentication logic closer to the Controllers and farther away from OWIN.

Look into Project Helios ( ) from Microsoft that does away with ASP.NET (System.Web) layer and lets OWIN talk directly to Process/Host.

OWIN Middleware

namespace WebAPIBlog


public class AuthenticationMiddleware


private readonly Func<IDictionary<string, object>, Task> _next;


public AuthenticationMiddleware(Func<IDictionary<string, object>, Task> next)


_next = next;



public async Task Invoke(IDictionary<string, object> env)


// inspect env and do credential validation, then set principal


env[“server.user”] = CreatePrincipal();

await _next(env);





The user identity is simply an entry in the dictionary called server.user. The adapters in both direction takes care that it materializes in ASP.NET as a Principal.

Katana Authentication Middleware

public class Startup{    public void Configuration(IAppBuilder app)    {        app.UseCookieAuthentication(new CookieAuthenticationExtensions        {            AuthenticationType = “Cookies”,        });         app.UseGoogleAuthentication(new GoogleAuthenticationExtensions        {            AuthenticationType = “Google”,        });         app.UseOAuthBearerAuthentication(new Microsoft.Owin.Security.OAuth.OAuthBearerAuthenticationOptions        {            AuthenticationType = “Bearer”,            AuthenticationMode = Microsoft.Owin.Security.AuthenticationMode.Active        });    }}



Even though the use of MessageHandler is no longer recommended, it is still supported and useful in some cases such as for Cross Site calls using CORS.

public class MyHandler : System.Net.Http.DelegatingHandler{    protected async protected override Task<System.Net.Http.HttpResponseMessage> SendAsync(        System.Net.Http.HttpRequestMessage request,         System.Threading.CancellationToken cancellationToken)        {                // inspect request                var response = await base.SendAsync(request, cancellationToken);                 // inspect response                    return response;        }}


In between the request and response we can manipulate the data above.

Authentication Filter

This filter is part of the overall security pipeline. Typically we will use this filter to invoke the Katana middleware. Every Katana Authentication middleware has a name (e.g. Cookies, Bearer, Google, etc.). We register the middleware in the Authentication Filter pipeline. So, inside the App_Start\WebApiConfig.cs class, we can add,

public static class WebApiConfig    {        public static void Register(HttpConfiguration config)        {            // Web API configuration and services            // Configure Web API to use only bearer token authentication.            config.SuppressDefaultHostAuthentication();            config.Filters.Add(new HostAuthenticationFilter(OAuthDefaults.AuthenticationType));            config.Filters.Add(new HostAuthenticationFilter(“Bearer”));             // Web API routes            config.MapHttpAttributeRoutes();             config.Routes.MapHttpRoute(                name: “DefaultApi”,                routeTemplate: “api/{controller}/{id}”,                defaults: new { id = RouteParameter.Optional }            );        }             }}


Now, in our controller, we can do something like,

namespace WebAPIBlog.Controllers{    [Authorize]    [HostAuthentication(“Bearer”)]  // default authentication type for Katana    public class ValuesController : ApiController    {        // GET api/values        [OverrideAuthentication]        [HostAuthentication(“Google”)]  // override authentication type for Katana        public IEnumerable<string> Get()        {            return new string[] { “value1”, “value2” };        }

 Authorization Filter

This remains as before including the use of the following annotations,

[Authorize] at the Controller or Action level to enforce non-Anonymous use of those resources,

[AllowAnonymous] at the Controller or Action level to indicate or override use of those resources,

[Authorize(Roles = “Admin”)] at the Controller or Action level to indicate specific roles that need to be authorized.

[Authorize][HostAuthentication(“Bearer”)]  // default authentication type for Katanapublic class ValuesController : ApiController{    // GET api/values    [AllowAnonymous]    [OverrideAuthentication]    [HostAuthentication(“Google”)]  // override authentication type for Katana    public IEnumerable<string> Get()    {        return new string[] { “value1”, “value2” };    }     // GET api/values/5    [Authorize(Roles = “Admin, SuperAdmin”, Users = “sa”)]    public string Get(int id)    {        return “value”;    }


This ensures that the Web API sends back a HTTP Status Code of 401 if not authorized or authenticated.

How to acquire the Client Identity? RequestContext !

In the past we have been able to acquire the client identity by looking at the Thread.CurrentPrincipal. Version 2 instead recommends the use of RequestContext.

The RequestContext is part of the HttpRequestMessage and incidentally the ApiController.User object now points to the RequestContext object. Keep in mind that now this value can be null so we MUST test for it before trying to use it.

This provides much better testability and avoids misuse of threads. Here is how to set this up.

First, let us configure our Web.Config,

<?xml version=”1.0″?> <!– For more information on using Web.config transformation visit –> <configuration xmlns:xdt=””>  <system.web>    <compilation debug=”true”         targetFramework=”4.5″ />    <httpRuntime targetFramework=”4.5″/>  </system.web&gt; <system.webServer>  <modules>    <add name=”BlogModule”               type=” WebAPIBlog.Pipeline.HttpModule”/>  </modules></system.webServer></configuration>


This refers to the APIBlog.Pipeline.HttpModule as defined below,namespace WebAPIBlog.Pipeline{    public class HttpModule : IHttpModule    {         public void Dispose()        {            throw new NotImplementedException();        }         public void Init(HttpApplication context)        {            context.BeginRequest += context_BeginRequest;        }         private void context_BeginRequest(object sender, EventArgs e)        {            System.Diagnostics.Debug.Write(“HttpModule”, HttpContext.Current.User.Identity);        }    }}


Now how to get to the User context – Claims Principal from within our Controllers?

As we can see below, we can either use the ApiController.User reference

namespace WebAPIBlog.Controllers{    [Authorize]    [HostAuthentication(“Bearer”)]  // default authentication type for Katana    public class ValuesController : ApiController    {        // GET api/values        [AllowAnonymous]        [OverrideAuthentication]        [HostAuthentication(“Google”)]  // override authentication type for Katana        public IEnumerable<string> Get()        {            var currentUser = User;            return new string[] { “value1”, “value2” };        } …

Or the System.Net.Http.Request.GetRequestContext().Principal Claim like so:

namespace WebAPIBlog.Controllers{    [Authorize]    [HostAuthentication(“Bearer”)]  // default authentication type for Katana    public class ValuesController : ApiController    {        // GET api/values        [AllowAnonymous]        [OverrideAuthentication]        [HostAuthentication(“Google”)]  // override authentication type for Katana        public IEnumerable<string> Get()        {            var currentUser = System.Net.Http.Request.GetRequestContext().Principal;            return new string[] { “value1”, “value2” };        }


Both of the above is identical.

Now for supporting OWIN,

if not already installed, use Nuget to get 2 packages, 1. Microsoft.Owin.Host.System.Web and 2. Microsoft ASP.NET Web API 2.2 OWIN. Now to bootstrap OWIN to the project ensure that the Startup class’s Configuration method has the following defined:

using System;using System.Collections.Generic;using System.Linq;using Microsoft.Owin;using Owin; [assembly: OwinStartup(typeof(WebAPIBlog.Startup))] namespace WebAPIBlog{    public partial class Startup    {        public void Configuration(IAppBuilder app)        {            var config = new System.Web.Http.HttpConfiguration();            config.Routes.MapHttpRoute(                “default”,                “api/{controller}”);             app.UseWebApi(config);             ConfigureAuth(app);        }    }}


Now, to add some OWIN Middleware… Create a new class

public class BlogMiddleware    {        private System.Threading.Tasks.Task _next;        // pass me the next middleware to call        public BlogMiddleware(Func<IDictionary<string, object>, System.Threading.Tasks.Task> next)        {            _next = next;         }         // method that does the actual work        public async System.Threading.Tasks.Task Invoke(IDictionary<string, object> env)        {            // we can use the env dictionary above for a key called server.user to retrieve the Principal Claim            //  or we can use the Katana, strongly typed helper class            var ctx = new Microsoft.Owin.OwinContext(env);            System.Diagnostics.Debug.Write(ctx.Request.User);             // call the next thing in the pipeline and pass along the environment…            await _next(env);   // this is how we can chain multiple middlewares together.        }    }}


Now, wire up this class, BlogMiddleware in the Startup.cs like so,

namespace WebAPIBlog{    public partial class Startup    {        public void Configuration(IAppBuilder app)        {            var config = new System.Web.Http.HttpConfiguration();            config.Routes.MapHttpRoute(                “default”,                “api/{controller}”);             app.Use(typeof(BlogMiddleware));             app.UseWebApi(config);             ConfigureAuth(app);        }    }}


Implementing the Authentication Filter

By convention we need to name our filters with a Filter suffix. So, in our case, we will create a WebApiBlogAuthenticationFilterAttribute class that implements the Attribute and IAuthenticationFilter interface.

namespace WebAPIBlog.Pipeline{    public class WebApiBlogAuthenticationFilterAttribute : Attribute, System.Web.Http.Filters.IAuthenticationFilter    {         public async System.Threading.Tasks.Task AuthenticateAsync(System.Web.Http.Filters.HttpAuthenticationContext context, System.Threading.CancellationToken cancellationToken)        {            // to get what is already in the pipeline access the RequestContext otherwise can use context.Principal            System.Diagnostics.Debug.Write(context.ActionContext.RequestContext.Principal);                        // we can now wire this up globally by adding configuration.Filters.add(…) in the Startup.cs or            // just add it per controller as an attribute annotation like            // [WebApiBlogAuthenticationFilter]            // public class MyController : ApiController        }         public async System.Threading.Tasks.Task ChallengeAsync(System.Web.Http.Filters.HttpAuthenticationChallengeContext context, System.Threading.CancellationToken cancellationToken)        {            throw new NotImplementedException();        }         public bool AllowMultiple        {            get { return false; }        }    }}


Authorization Filters

Similar to adding the Authentication Filter, we will create a class again with the Attribute suffix in its name. Let us call it WebApiBlogAuthorizationFilterAttribute and this time we will just override a base class called AuthorizeAttribute like so,

namespace WebAPIBlog.Pipeline{    public class WebApiBlogAuthorizationFilterAttribute : System.Web.Http.AuthorizeAttribute    {        // override the IsAuthorized method here        protected override bool IsAuthorized(System.Web.Http.Controllers.HttpActionContext actionContext)        {            // implement our custom authorization policy here.            System.Diagnostics.Debug.Write(actionContext.RequestContext.Principal);             // return base.IsAuthorized(actionContext);            return true;         }    }}


Simulating the Katana Authentication Middleware (for testing)

To mock a Claims Principal we can harness this into the Middleware implementation class that we saw earlier

public class BlogMiddleware{    private System.Threading.Tasks.Task _next;    // pass me the next middleware to call    public BlogMiddleware(Func<IDictionary<string, object>, System.Threading.Tasks.Task> next)    {        _next = next;     }     // method that does the actual work    public async System.Threading.Tasks.Task Invoke(IDictionary<string, object> env)    {        // create a mock claims Principal using GenericIdentity        var context = new Microsoft.Owin.OwinContext(env);         // authentication        context.Request.User = new System.Security.Principal.GenericPrincipal(new System.Security.Principal.GenericIdentity(“aref”), new string[]{});         // we can use the env dictionary above for a key called server.user to retrieve the Principal Claim        //  or we can use the Katana, strongly typed helper class        var ctx = new Microsoft.Owin.OwinContext(env);        System.Diagnostics.Debug.Write(ctx.Request.User);         // call the next thing in the pipeline and pass along the environment…        await _next(env);   // this is how we can chain multiple middlewares together.    }}


Hosting Options

If you recall, we do not have to use the IIS/ASP.NET hosting infrastructure (System.Web) because of the abstraction provided by OWIN.

Specifically, we will start by adding a Nuget package, called OwinHost. Now go to the Project Properties, click the Web tab on the left and scroll down to the Servers section and select OwinHost from the drop-down list.

We can also use the prerelease project called Helios to replace the use of System.Web. To do this, we will need to Nuget the package, Microsoft.Owin.Host.IIS

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s