RefM – A Microservice Case Study – The Undocumented API
Loading...
X

RefM – A Microservice Case Study

clip_image002

Introduction

Microservice Architecture is an emerging approach to modern application architecture where applications are composed of autonomous, independently deployed, scaled, and managed services. This approach to service architecture along with the benefits of cloud platforms provides the scalable, resilient, cross platform foundation necessary for Modern Applications.

This post takes the logical architecture defined in my previous post Modern Apps and Microservices and applies it to the design and implementation of a Microservice called RefM. RefM provides a common utility needed by nearly all applications, reference data.

RefM Data Model

Reference data is any kind of data that is used to categorize other data or for relating data to information beyond the boundaries of the enterprise. Sometimes called lookup data, it is organized by domain such as ‘states’ or ‘countries’ and is defined by entities made up of codes and values such as 2 letter state abbreviations and the state name:

  • domain = “States”
  • code = ‘MA’
  • value = ‘Massachusetts’

Reference data can be further defined through attributes. Attributes can be used to provide detail about a piece of reference data as in the example above data such as population, longitude, latitude and so on. Reference data is used in the presentation tier quite often, so a sequence attribute is typically used to provide detail to the presentation tier on the order in which the data should appears, Alphanumeric Ascending or Descending. Also, entities can be sequenced to be first or last in a list, for example, United States could be forced to the top of an alphabetical list of countries.

The design of a Microservice begins with data modeling. You may choose to have an on-the-wire data model and an in-the-store data model. This will be determined by the complexity of the data model as well as how the model is stored in the database.

The on-the-wire data model represents the data as it is sent over the wire and consumed by the client. These are the models that will drive the REST API design. The in-the-store model represents how the data is represented in the storage medium. Since there could be two data models, data transformation is a common operation that is performed by a Microservice.

RefM will use a No-SQL data store so the on-the-wire model will be identical to the in-the-store model. The model will be both stored and transmitted in JSON format which will make it easy to work with on both the client and the server.

The RefM Data Model consists of Entities and Attributes. An Entity belongs to a domain (States) and provides a code (MA) and a value (Massachusetts). A use-case of the RefM Microservice is that you will be able to request a list of all entities that belong to a particular domain; a list of the 50 states or all the countries in the world or all the zip codes for Boston.

Entities are also able to link to another. For example, each zip code from Massachusetts could link to the ‘MA’ state entity. The use-case is that you could request all the zip codes for the state of MA.

Each Entity has a property for sequence with the default being Alpha-Ascending and also supporting Alpha-Descending, First and Last.

Finally, in order to provide extensibility in the data model and to provide additional functionality for applications, an Entity may contain a list of Attributes where an Attribute is a name-value pair. An example would be that each state Entity would contain attributes for population, current governor, square miles, etc.

clip_image004

When implemented, a unique ID and a Cache Time-To-Live is provided in order to support both the physical storage medium (DocumentDb No-SQL) and the in-memory cache (Redis).

clip_image006

RefM API

Once you have the data model, the next step is to define the API. As we will be using the REST protocol, we will want to follow best practices for the design of REST API’s. A highly recommend resource for best practices in designing REST API’s is Apigee’s e-book entitled Web API Design. Here are a few of the recommendations form their book:

Keep your base URL simple and intuitive Use the name of the Microservice as the base of your URL à /ref
Use nouns to define paths to the resources, keep verbs out of your base URL’s Use the language of your data model to define paths à /ref/entities/domain/{domain}
Use HTTP Verbs(GET, POST, PUT, DELETE) to operate on the collections and elements GET /ref/entities – returns a collection of entities, POST /ref/entities – creates an entity
Provide robust descriptive error handling that leverage HTTP status codes 200-OK, 400-Bad Request, 500-Internal Server Error, 401-Unauthorized
Version your API’s Place versioning at the start of the path à/v1/ref/entities/domain/{domain}
Compliment your API’s with an SDK At a minimum provide code snippets in popular programming languages such as JavaScript and C#

RefM is a Microservice that provides read/write access to reference data and since reference data is typically administered by the team responsible for enterprise data, the ability to write reference data is restricted to the administrator team. API’s related to administrative operations will add the ‘admin’ noun to their paths to differentiate them from the public API’s. Following the rules outlined above for API design, here is both our public (developer) API and our private (admin) API.

clip_image008

Note that there is room for this API to expand. This is V1 and provides enough functionality to get up and running. Looking forward it is possible to imagine the next version providing public GET operations based on Attributes and private POST and PUT operations that work on batches.

RefM Interfaces

Below are the C# interface definitions for our API’s. These interfaces will be used by both the service and the SDK. Note the use of namespaces to provide organization for our definitions. There will be some instances where we will use the same class name in both the public, private, server and client implementations and so namespaces will help us avoid name collision and help developers an easy way to find what they are looking for.

clip_image010 clip_image012

 

RefM Project Dependency Tree

Now that we have our Data Model and Public and Private Interfaces defined, let’s consider how we are going to define our projects and the dependencies they have on each other. This project and dependency model will inform the Continuous Integration build process.

clip_image014

Model The project that defines the Microservice Data Model
Public API Interface The project that defines the Public API interface
Public API Service Implementation The project that implements the public interface. This is where domain logic and data access are implemented.
Public API REST Endpoint The project that implements the public REST Endpoint. This code handles the incoming REST messages and routes them to the appropriate method of the Service Implementation class
Public API SDK The project that implements the public interface for client consumption. This is where client requests are packaged for invocation over the wire protocol, HTTP for example.
Private API Interface The project that defines the Private API interface
Public API Service Implementation The project that implements the private interface. This is where domain logic and data access are implemented.
Private API REST Endpoint The project that implements the private REST Endpoint. This code handles the incoming REST messages and routes them to the appropriate method of the Service Implementation class
Private API SDK The project that implements the private interface for client consumption. This is where client requests are packaged for invocation over the wire protocol, HTTP for example.

RefM Services

The RefM Services implement the public and private interfaces, accessing the physical store, providing in-memory caching where applicable and providing rich error handling. Here is a sample code snippet from the RefM Public API Service Implementation:

clip_image016

The TRY block:

  • Line 31 – define the query against the No-SQL Db to retrieve all entities in a particular domain
  • Line 32 – grab the JSON for the entity list from cache
  • Line 33 – if found in cache, instantiate else fire query and return results from Db
  • Move to the FINALLY block

The CATCH block:

  • Line 37 to 38 – If an error occurs in calling the Db, return HTTP Error Bad Request with the detailed error encapsulated

The FINALLY block:

  • Lines 42 to 46 – If the database query return an empty set, return a Not Found error
  • Lines 48 to 49 – Otherwise cache the JSON representation of the list and return

This code leverages two common framework classes not covered by this article, _dbase and _cache. These classes are found in a referenced framework library and handle the interaction with any databases, message queues, and in-memory cache sub-systems required. In the code below they are initialized to use Azure Redis Cache and Azure DocumentDb.

clip_image018

RefM REST Endpoints

Since we have externalized the definition of the model, interface, and implementation of our services, the REST Endpoint project is wafer thin. Its purpose is to leverage the ASP.NET WebAPI technology stack for receiving and routing incoming HTTP requests to our service. It defines the URL routes and their mapping to a set of methods that forward on the request to our service class:

clip_image020

  • Line 10 – Define member of type RefM Service Class
  • Line 14 – Instantiate RefM Service Class
  • Line 17 to 18 – ASP.NET Web API Route definition for GET operation for All Entities by Domain
  • Line 21 – forward the call to the RefM Service object

Note that the routes do not identify the version. It is our recommendation that your services be placed under API Management using products such as Apigee, Azure API Management, 3Scale or some equivalent. These systems provide an additional layer of security, policy injection, version control and analytics that are a must have for hosted services.

To demonstrate how versioning could be handled using API Management here are screen shots from the Azure API Management System for the API above. API Management provides a proxy to our API and it is in the proxy where the versioning will be managed. Note the use of versioning in the URL that will be exposed by API Management.

clip_image022

Here is the GET operation for all Entities from a particular Domain:

clip_image024

In addition to these administrative features, API management systems provide a portal for developers to be able to register and subscribe to API’s. In return, they receive an API Key that is used on all subsequent calls providing analytic insight into each and every invocation.

Here is a screen shot from the Azure API Management Developer Portal for the RefM Developer API console. The developer console provides documentation, code samples, and the ability to execute the API to see how the results are formatted:

clip_image026

clip_image028

RefM SDKs

The purpose of an SDK is to make it easy for developers to adopt your software. There is also a burden placed on you and the developer; keeping current. This is a reasonable price to pay for the ease of use. SDKs also come in various shapes and sizes from code snippets to full blown sample apps to that plus downloadable libraries for various languages and platforms.

For purposes of this discussion, we have created an SDK for C# developers. The first code we will examine is the SDK class definition, RefM, taking its name from the Microservice. Note that the SDK class implements the same interface as the RefM Service class.

clip_image030

  • Line 13 – The SDK leverages another Microservice is the BlueMetal portfolio called LogM. It is used to log exceptions, alerts, notifications, and events. It uses a message Q for delivery of these messages to a central repository used for reporting and analytics
  • Line 15 and 16 – The SDK exposes two public properties, one for the developer key and the other for the API URL. Exposing these as properties allows the developer to specify the location of the Microservice which could be local or in the cloud. If the Developer Key is not present, the API call is still made and it is assumed that the service is not under management.

Next, we will take a look at the implementation for the client side of ‘Get All Entities by Domain’:

clip_image032

  • Line 31 to 38 – If no API URL is specified, log and throw an error
  • Line 40 – Begin to build up the URI
  • Line 42 to 45 – If the DevKey is present append it to the URL
  • Line 47 – Invoke the API
  • Line 49 – Instantiate the JSON as an in memory object
  • Line 51 to 58 – Handle any exceptions
  • Line 60 – Return the list of Entities

RefM in Action

One of the features of Azure’s API Management system is that is provides code snippets demonstrating how to call an API in various languages. Here are some samples of the code snippets that Azure API Management provides:

clip_image034

clip_image036

We could have stopped there and leverage this built in documentation for API invocation. We went that extra mile and provided a C# SDK. If you do go this extra mile, it is recommend that you leverage services such as NuGet, GitHub, CodePlex or an equivalent to distribute your SDK.

In addition to downloading the SDK from one of these locations, the developer would register on the API Developer Portal and subscribe to the API of interest. Once that process is complete, the developer would have a key that they would use on each and every call to the API.

Our SDK makes it easy for the developer to supply their developer key and set the location of the service and finally to call the API. Here is a code snippet that demonstrates calling the RefM Admin SDK:

clip_image038

This code demonstrates how an administrator would create an entity:

  • Declare and instantiate an instance of RefM Admin SDK class
  • Set the developer key and API URL for v1 of the RefM Admin REST Endpoint
  • Create and initialize an entity for the state of Massachusetts
  • Call the API to create the Entity

In this next example, we demonstrate how a developer would use the RefM Dev SDK to get a list of all the states in the US and populate a ListBox as well as how to look up a single state and grab the population attribute and display in a TextBox:

clip_image040

The second code snippet demonstrates how a developer could use the RefM Dev SDK:

  • Declare and instantiate an instance of the RefM Dev SDK class
  • Set the developer key and API URL for v1 the RefM Dev REST Endpoint
  • Request a list of the 50 states
  • Populate a ListBox with the names of the 50 states
  • Lookup the state of Massachusetts
  • Display the population of the state of Massachusetts in a TextBox

If you are interested in learning more about Microservices and how they can be applied to your next development project, please contact me at bobf@bluemetal.com or reach out to our Sales Directors:

In BostonMatt BienfangVice President

Managing Director of Sales

mattbien@bluemetal.com

617.800.0332

In Chicago Jay MahlendorfClient Director

jaym@bluemetal.com

312.219.9943

In New York Brendan FitzgeraldClient Director

bfitz@bluemetal.com

215.850.6569

3 observations on “RefM – A Microservice Case Study
  1. Chris Schick

    Hi Bob,

    Thanks for the article, I like the simplicity of your design and enjoyed the MS Virtual Academy titled ‘Exploring Microservices in Docker and Microsoft Azure’. Any chance you could make the REFM code available? You know, so I can get a head start. I would like to build off of your implementation as a base but I’m extremely challenged for time.

    Thanks,

    Chris

     
    Reply
  2. Chris

    Hello Bob,

    Thanks for the article. Can you provide some insight into how the ID of the ModelBase is set? Is this set in the database and/or by the Admin application during creation?

    Thanks,

    Chris

     
    Reply
  3. Pingback: Dew Drop – January 15, 2015 (#1934) | Morning Dew

Leave Your Observation

Your email address will not be published. Required fields are marked *

Read previous post:
Modern Apps and Microservices

Introduction This post is part one of a two part series that delves into an emerging approach to modern application architecture...

Close