Ryan Tablada

Christian, UX Tycoon, Instructor at The Iron Yard

Checkout my screencasts at EmberGrep

Checkout my newest book on Leanpub

API Archives

Creating a Simple TODO MVC API with API Kit

Jan/20/2016

For a long time, I've wanted a small JSON based API server that I could stand up in minutes with relationships. I decided that Node and Mongoose give me a lot of flexibility for small projects that I just want to try things out on.

I've been messing around with this stack and had a fairly large setup called express-shell that I use to make a more handrolled site using Express and Mongoose. But, express-shell focused on server rendering and there was a lot of things around sessions and mail that I didn't need for micro-services and smaller APIs.

So, I had a list of things that I wanted:

  • Quick start up
  • CLI Resource Generators
  • Support for One to One, Many to Many, and Has Many relationships (with foreign keys)
  • JSON output to match JSON API or namespaced JSON resources
  • OAuth 2 Bearer Grant Support
  • Easy (or auto) registration of public and protected resources

After a while of work, I've created api-kit and to go along with things generator-api-kit a Yeoman generator package to facilitate the creation of APIs.

Getting Started

To get started, install Yeoman and generator-api-kit:

npm install -g yo generator-api-kit

Then create a new API Kit project:

yo api-kit

This will create the boilerplate for our API project and install all the required Node modules.

Generating the Todo Resource for Todo MVC

One of the key points of API Kit was that it has to build quick JSON based APIs right from the command-line. So, to match Todo MVC, let's create a todos resource with two properties: an isComplete field which will be a boolean value and a title field which will be a string of what our user wants to do. We can now generate this using the api-kit:resource generator:

yo api-kit:resource todo isComplete:Boolean title:String

Here we are saying that we want to create a todo model with our different fields and data types.

This will create a few files for us:

  • app/models/todo.js - A Mongoose model describing our model schema
  • app/http/resources/public/todos.js - A set of Express.js routes for standard CRUD
  • app/transformers/todo.js - A Mystique transformer to map data in and out of our API

Using our API

Now, we can serve our API by running npm start.

If we use something like Postman, we can go to http://localhost:3000/api/todos and we'll see the following response back from our API:

{ "todos": [] }

Let's create a new Todo for our app by sending the following JSON to our API as a POST to http://localhost:3000/api/todos:

{ "todo": { "title": "Buy Milk", "isCompleted": false } }

And now the server responds with:

{ "todo": { "id": "56a01982f8630e7f4a771879", "title": "Buy Milk" } }

NOTE Your id will vary

And if we make another GET request to http://localhost:3000/api/todos:

{ "todos": [ { "id": "56a01982f8630e7f4a771879", "title": "Buy Milk" } ] }

What's Next

In the upcoming weeks I hope to document API Kit a bit more including:

  • Deploying to Heroku
  • Creating Related Records
  • User Authentication
  • Interacting with this data from a single page app (likely Ember.js)

Why I Am Deprecating Eloquent-Ember

Jun/04/2014

If you follow me, you know that I am heavily involved in both the Ember and Laravel communities. Thus, I have taken it upon myself to help define the development story for working with Ember in Laravel projects as well as digging into what is happening at a lower level in both projects. This not only includes articles, but also creating reusable code both in the Laravel/PHP world and in the Ember world.

One of my first contributions to this effort was the creation of Eloquent-Ember. This package added functionality to your Eloquent models to allow for somewhat compatibility with the Pre 1.0 JSON API standard used by Ember's JSONAdapter. In today's Ember field, Eloquent-Ember is a closer relation to the ActiveModelAdapter available in Ember Data 1.0.0-beta.

However, I am officially deprecating Eloquent-Ember for a few major reasons. Primary of which is that I admit that it is a pattern of bad design.

Design Implications

Eloquent-Ember was a prime example of heavy coupling for to make Ember specific APIs easier to build. I always knew that this was a stop-gap solution and was not particularly the right way of going about things. It completely tied every instance of a model to output data in a way that Ember would understand.

This meant that my API presentation logic was deeply nested in the database layer of my application. Now everything in my application had to talk Ember speak. This meant my business logic and server generated content (including packages that hooked into Eloquent) had to be modified or abstracted.

This started to show itself as the API for Eloquent-Ember became more and more awkward as features were tacked on. Properties to define what relationships to load when outputting JSON, other's that would override this array and could be set on the fly. It all was a bit of a mess.

Taking Notes From Ember Data

Having taken some time to dive deeper into Ember and Ember Data's code, I found that DS.Models were never affected by the API that they were attached to. Instead, Ember uses Adapters and Serializers to get the data in a state that DS.Models can work with. This also means that down to a per request basis, the way that data is handled and interpreted can be customized to fit the usecase. The Model is then a standard version which can focus only on keeping up with properties and relationships. This is a strong pattern which allows for a ton of flexibility and means that even the craziest of data sources can be consumed and brought in to DS.Models. Some of the Adapters and Serializers I have seen range from consuming JSONAPI standards, XML APIs, CouchDB interactions (with authentication), and more.

What this means for the Laravel side of things

On the Laravel side of things we can actually mimic this pattern where our Eloquent models are decoupled from our Ember API implementation. So to put things in 1-to-1 terminology:

DS.Model - EloquentModel DS.Adapter - Controller DS.Serializer - Serializer Service

DS.Adapter & Controllers

In Ember, the implementation of an adapter really is a communication layer of where to grab data back and forth between your API and the Client-Side model. This is a bit similar to an API Controller (and a bit of the Router) on the server-side of things.

A controller sees the input that is being sent (HTTP headers, Request Payload, etc), and translates that into something that can then fetch data. But at the most basic level, the data needs to be formatting to something that the Business logic on the server will understand. This is where calling out to a Serializer helps.

DS.Serializer & Serializer Services

Serializers have one role to play in a system: translating data. If we look at the public API for the various Serializers in Ember Data, there are really two functions that we are publicly facing - serialize and normalize

serialize is taking data from the current system and translating it into something that another system will understand. normalize does the opposite taking external data and translating it into something that is meaningful to our existing business logic.

Implementation Of A Basic Adapter

In my own opinion, I do really like the new API standard expected by Ember's RESTAdapter (and the even thinner JSONAdapter). It's explicit with room to grow further API features that would not only work with Ember, but other services as well. Essentially I would like an easy path to build APIs where with just a few lines of code, I can rather reliably know that my response would look like this:

json { "users": [ "firstName": "John", "lastName": "Doe", "hasOneRelationship": 2, "embeddedAddress": { "street": "123 1st Street", "state": "NY", }, "hasManyRelationship": [ 1, 2, 3 ] ], "meta": { "status": 500, "page": 1, "lastPage": 50, "perPage": 10, "authenticatedUser": "..." } }

But, I don't want to have to sacrifice standardized interfaces in the Laravel world to do so. This means bringing the presentation logic out of the Model, even out of the controller, and focusing on a serializer pattern.

Progress To The Future

I have written a bit on some of my early attempts to build out this pattern in Laravel (beware that code has bugs!). And I have an example application showing this pattern (and using some Ember CLI Awesomeness).

I am continuing to work on solidifying and having a good test base for this before creating a package for distribution. If you look at the Laracon Ember application, the final API for creating APIs will be very similar to what is there. So you should be able to return arrays, Collections, or Eloquent Models from a controller action and have it format the response accordingly.

php public function index() { return Users::paginate(20); }

However, I am still working on the stories for both more customization (API Keys, Auto-Loaded Relationships, custom Serializers, etc), as well as more convenience (base repositories to handle queries, etc). I also am looking at making the meta data story really powerful so that this becomes a powerful API tool for all uses (not just Ember). Along those lines, I am looking at using well tested utilities such as Fractal which could help keep the package well tested with very little effort on my part.

Let me know what you think about this approach, and anything you think I may have missed the ball on.

Clean Transformed APIs

May/08/2014

In the past, when working with Javascript libraries that required diffrent JSON formatting, I have base models that extended Eloqent and changed the toArray function. It was super hacky and required a ton of rework for each different data format that I needed and I couldn't support multiple API formats. Plus anything in my application (including external libraries) would have to be informed of the changes in the toArray functionality and contract.

Not to mention, my model class required rewrites for even the most minor change in the structure of data. It really stunk!

After spending some time in the guts of Ember and Ember Data's source code, I realized a cool pattern. They used serializers to take in and consume data formats. And you could extend them to your heart's content!

From there, I looked at my Laravel project and thought where this would fit. If I put the logic in the model again with just a tacked on serializer, I would run into the same issues with breaking the contract and committing to one data format. Plus if I published something that directly mapped to the database, I think that Phil Sturgeon would run me down with his bike.

Bike Chase

In the past, I did this manually in repository, but that doesn't feel right. Why should a data fetching object have to also present the data? Nope, I want the repository to do as little as possible (it should do one thing and do it GREAT).

Then I thought that the controller would be a cool way to handle this. But, I knew I wanted to keep my controller super sexy and thin. This meant extending a base controller. It wouldn't do much good if each controller had to know how to inject some object, which gives the chance that I mess up the declaration of the __construct function. In reality, if I really wanted to, I wanted to have something where my API controller could look like this:

```php <?php

class UsersController extends MagicSauceController { public function index() { return User::all(); } } ```

This is no troll either! This wouldn't be my final implementation (I still have to add HTTP header token authentication, authorization, and more,) but I wanted that code to work.

Luckily, Laravel is pretty smart about how the router talks to controllers. If you use layouts, you've got some idea to how this works. In Illuminate\Routing\Controller there is a callAction function that is used for communication between the router and the controller method you write on a day to day basis.

From here, I am able to hook into the what a controller method returns and then use my serializer class on that response. So, here is a basic version of a my serializing BaseController:

```php <?php namespace Api;

use Illuminate\Routing\Controller; use Illuminate\Support\Collection;

abstract class BaseController extends Controller { protected $serializer = 'Services\CamelCaseSerializer';

protected function serializeOutput($values)
{
    return $this->serializer->serialize($values);
}

protected function needsSerialization($response)
{
    return      is_array($response)
            ||  is_a($response, 'Illuminate\\Support\\Collection')
            ||  is_a($response, 'Illuminate\\Database\\Eloquent\\Model');
}

protected function addOutputKey($response)
{
    if (is_a($response, 'Illuminate\\Database\\Eloquent\\Model')) {
        $key = $this->getSingularOutputKey();
    } else {
        $key = $this->getPluralOutputKey();
    }

    return new Collection(array($key => $response));
}

protected function getSingularOutputKey()
{
    $key = str_singular(class_basename($this));

    return $this->serializer->serializeKey($key);
}

protected function getPluralOutputKey()
{
    $key = str_plural(class_basename($this));

    return $this->serializer->serializeKey($key);
}

/**
 * Execute an action on the controller.
 *
 * @param string  $method
 * @param array   $parameters
 * @return \Symfony\Component\HttpFoundation\Response
 */
public function callAction($method, $parameters)
{
    $this->setupSerializer();

    $response = call_user_func_array(array($this, $method), $parameters);

    // Check if the response should be serialized
    if ($this->needsSerialization($response))
    {
        $response = $this->serializeOutput($response);
    }

    // Check if the response should be keyed
    if ($this->needsSerialization($response))
    {
        $response = $this->addOutputKey($response);
    }

    return $response;
}

protected function setupSerializer()
{
    $this->serializer = app($this->serializer);
}

} ```

This controller has know idea what data transformation the serializer is doing. All of the transformation lies within the Services\CamelCaseSerializer class.

```php <?php namespace Services;

use Illuminate\Support\Str;

class CamelCaseSerializer { public function __construct(Str $str) { $this->str = $str; }

public function serialize($data)
{
    switch ($data) {
        case is_a($data, 'Illuminate\\Support\\Collection'):
            $serializedData = $this->serializeCollection($data);
            break;
        case is_a($data, 'Illuminate\\Database\\Eloquent\\Model'):
            $serializedData = $this->serializeEloquentModel($data);
            break;
        case is_array($data):
            $serializedData = $this->serializeArray($data);
            break;
        default:
            $serializedData = $data;
    }

    return $serializedData;
}

public function serializeKey($key)
{
    return $this->str->camel($key);
}

protected function serializeCollection($collection)
{
    return $collection->map(function($data) {
        return $this->serialize($data);
    });
}

protected function serializeEloquentModel($model)
{
    return $this->serializeArray($model->toArray());
}

protected function serializeArray(array $data)
{
    $serializedData = array();

    foreach ($data as $key => $value) {
        $key = $this->serializeKey($key);
        $value = $this->serialize($value);

        $serializedData[$key] = $value;
    }

    return $serializedData;
}

} ```

The output from the User controller from earlier looks a bit like this:

json { "users": [ { "id": 1, "email": "foo@example.com", "userName": "foo_bar" } ] }

Now, if I wanted to create a serializer that transforms my data some other way, but only for a single controller or group of controllers, I can just set $serializer on that controller instance to MyMagicSerializer's class name.

As Jeffrey Way pointed out, this is still a very simple implementation. Before pulling this out of my project and building a package, I hope to do some of the following:

  • Inject status codes and meta data
  • Handle paginators (right now those would be all messed up)
  • Handle errors and set statuses and messages