With the constant growth of the number of internet-connected devices, the requirements for Web APIs is on the rise more than ever. So much so that common web frameworks are often being repurposed to serve as platforms for Rest API development. However, since none of them was primarily developed to handle a Rest API project, using them as-is quickly increases the project complexity which results in longer development times as well as increased maintenance costs.
Goals
In this document we'll attempt to answer a question that is as old as development itself – how to properly structure a project so it is:
- easy to get around in, even for new developers
- Large projects, more often than not, have many developers coming in and out during their lifecycle; making the familiarization easier reduces the required on-boarding time, helps project's constant growth and reduces maintenance cost during the project's lifetime.
- easy to maintain and upgrade in the future
- An API project should have future upgrades in mind, the simplest starting point is versioning your API's endpoints, which makes it much easier to implement new standards as they arrive (looking at you, JSON:API). The project's structure needs to be ready for that and require minimal changes when a new standard comes in. This will greatly reduce the upgrade costs as no time will be spent on project restructuring and legacy code refactoring.
- not weighed down by its size
- Some structures work very well with small projects, but one thing you learn over time is that rarely ever a small project remains a small project. Larger projects can often suffer because of a "small project approach" in the early phases, increasing their cost which would've been avoided with better planning. This structure avoids this problem as it should work well for both small and large projects and hopefully, you'll be able to try it out in both scenarios.
This structure allows development of multi-module applications, which is exactly what I did in the attached example, however it works equally well with a single module.
What we're not covering
This post is not going into the details of code implementation; however, a sample project is supplied and you can easily base your projects off it. We're not going into details on how to properly manage your API output, as it is a topic for another day. Simple "JsonResponses" are used in this example instead. For a real-world application, standardizing your API's output is a must. Security implementation is also not the subject of this text; however, you will probably notice security benefits of the presented approach even though they are not mentioned.
The framework
I'm using Symfony as it is a commonly used framework for web-based API development. A framework should never dictate the application structure, and that is exactly what Symfony 4 is trying not to do. With Symfony 3, this was different and the first mistake almost every team working on a Symfony-based project did was use its "bundled structure". That structure is only meant to be used if the code in that structure is planned to be reused as-is in other applications, which, in most cases, is not true.
The API project structure
To demonstrate the structure, I created a sample project which is provided with this text. This project actually has 2 applications which share some resources – "Consumer" application, which is meant to allow API access from the consumer side and the "Admin" application which is built to handle API for the administrator access. Admin application has access to all resources of the Consumer application, while Consumer application can only access its own resources. They are meant to be accessed from different domains (e.g. www.your-api-site.tld and admin.your-api-site.tld). We'll start from the root folder of the project which also holds the composer.json file and the vendor folder.
Root structure
``` ├─ bin/ ├─ config/ ├─ docs/ ├─ migrations/ ├─ public/ ├─ src/ ├─ tests/ ├─ translations/ ├─ var/ └─ vendor/ ... ```
The root folders are all named in lower case. You're probably familiar with the use of most if not all of them, but let's go over them so we're on the same page.
`bin` and `public` folders contain the front controllers for cli and web access, respectively. `config` folder contains the global settings, which are inherited by all the applications in the project. `docs` is meant for Swagger API documentation, it can be omitted if you don't plan to use swagger, or you intend to use Swagger Annotations (not recommended, as it reduces code readability, YAML is much more readable and it doesn't clutter the controllers). `migrations` will hold all the migrations for all apps. `src` holds the source code of our apps and is the namespace root for our project. In the provided example that is the "SampleApp". `tests` is meant for unit tests, should you use them. `translations` should contain translation libraries. `var` folder is used for all app-generated files. `vendor` is generated by composer and contains all the dependencies.
Bin folder
In a default Symfony installation the `bin` folder would only contain the `console` front controller for the CLI. If your project contained only one app (module), this would remain the same, but in multi-app environment like the provided example bin has more CLI front controllers:
``` ... ├─ bin/ │ ├─ admin │ ├─ console │ └─ consumer ... ```
In this setup, commands executed on the `admin` file will be executed only in the Admin application. The same goes for the Consumer. However, console executes the same command in both applications. This is required for automated deployments or scripts executed after composer update or install.
Config folder
The `config` folder has the default contents for a Symfony app. The only thing to remember is that this is to be considered as a "global" config in this setup. Each module/application can have its own config folder which extends or overrides the global config. E.g. a global config can have a global entity manager configured to work with the main database of the project. A module can define its own entity manager that works only with the schema specific for that module (no other module will have access to it).
Docs folder
Docs is meant to hold swagger definitions. E.g.:
``` ... ├─ docs/ │ ├─ admin.yaml │ ├─ common.yaml │ └─ consumer.yaml ... ```
Never define your swagger docs using annotations. They are hard to maintain, don't indent automatically and reduce code readability.
Migrations
Migrations are to be sorted as follows:
```
...
├─ migrations/
│ ├─ Admin/
│ │ ├─ Version2019{...}.php
│ │ ├─ Version2019{...}.php
│ │ └─ ...
│ └─ Consumer/
│ ├─ Version2019{...}.php
│ ├─ Version2019{...}.php
│ └─ ...
...
```
Public folder
In most cases the public folder contains only the front controller for the entire project - `index.php`. In multi-app solutions either the front controller can be refactored to detect domain and decide which app's Kernel to start, or the following solution can be employed:
``` ... ├─ public/ │ ├─ admin/ │ │ └─ index.php │ └─ consumer/ │ └─ index.php ... ```
That way Apache or Nginx decide which front controller to call depending on the domain the request is coming from.
Src folder
The main folder this document is about, in some applications this is the `app` folder. It is the root of our project's namespace and contains the main app folders along with the shared resources folder `Core`:
``` ... ├─ src/ │ ├─ Admin/ │ ├─ Consumer/ │ └─ Core/ ... ```
The "Core" and the shared components
Core can hold any shared components like Subscribers, Exceptions, project-level Security implementations, etc. However, it should not hold Controllers/Actions, Entities or any other resources that are specific to an application in the project. In the provided example, an ExceptionSubscriber and RequestSubscriber are supplied. These affect request and response handling for all applications within the project. RequestSubscriber verifies validity of incoming requests. In the provided example it is rather simple; however, I suggest a proper content negotiator implementation. ExceptionSubscriber catches thrown exceptions and makes an API-friendly Response from them. Another subscriber that is common among APIs is a ResponseSubscriber that is meant to handle the responses properly instead of using the JsonResponse like we do in the provided example. A proper approach would be to return resources or representation objects from controllers and serialize them according to the requested content type in the ResponseSubscriber. Doing serialization in every controller would result in a lot of repeated code and it is not in accordance with the design pattern we're using (more about that in a bit).
The application folders
The main application folders are where the most of the development work is happening. They will grow as the application grows and, at some point, a new version for the API might get introduced. To handle the growth of the application gracefully, split your application's features in groups. E.g. user-related features like authentication, account management and similar go into "Account" or "User" folder. Features related to handling payments or issuing invoices into "Finance". By using this logic, the structure will end up looking something like this:
``` ... ├─ src/ │ ├─ Admin/ │ │ ├─ ... │ │ ... │ │ └─ Kernel.php │ ├─ Consumer/ │ │ ├─ Account/ │ │ │ ├─ Action/ │ │ │ │ ├─ Login.php │ │ │ │ ├─ Register.php │ │ │ │ └─ ResetPassword.php │ │ │ ├─ Command/ │ │ │ │ ├─ LoginCommand.php │ │ │ │ ├─ RegisterCommand.php │ │ │ │ └─ ResetPasswordCommand.php │ │ │ ├─ Entity/ │ │ │ │ └─ User.php │ │ │ ├─ Exception/ │ │ │ │ └─ InvalidCredentialsException.php │ │ │ ├─ Handler/ │ │ │ │ ├─ LoginHandler.php │ │ │ │ ├─ RegisterHandler.php │ │ │ │ └─ ResetPasswordHandler.php │ │ │ └─ Repository/ │ │ ├─ Catalog/ │ │ │ ├─ Action/ │ │ │ ├─ Command/ │ │ │ ├─ Entity/ │ │ │ └─ Handler/ │ │ ├─ config/ │ │ │ ├─ packages/ │ │ │ ├─ routes/ │ │ │ └─ services.yaml │ │ ├─ Finance/ │ │ │ ├─ Action/ │ │ │ ├─ Command/ │ │ │ ├─ Entity/ │ │ │ ├─ Event/ │ │ │ └─ Handler/ │ │ └─ Kernel.php │ └─ Core/ │ ├─ Exception/ │ │ ├─ ResourceExpiredException.php │ │ ├─ UnsupportedContentTypeException.php │ │ └─ ValidationException.php │ ├─ Subscriber/ │ │ ├─ ExceptionSubscriber.php │ │ ├─ RequestSubscriber.php │ │ └─ ResponseSubscriber.php │ └─ Traits/ │ └─ Timestampable.php ... ```
In small projects, such grouping can be omitted, but it should be introduced if the project continues to grow, as the size of the project will start suffocating it.
Here's an example of a smaller project setup:
``` ... ├─ src/ │ ├─ API/ │ │ ├─ Action/ │ │ │ ├─ Auth/ │ │ │ │ ├─ Login.php │ │ │ │ ├─ Register.php │ │ │ │ └─ ResetPassword.php │ │ │ ├─ ViewSomething.php │ │ │ └─ CreateSomething.php │ │ ├─ Command/ │ │ │ ├─ Auth/ │ │ │ │ ├─ LoginCommand.php │ │ │ │ ├─ RegisterCommand.php │ │ │ │ └─ ResetPasswordCommand.php │ │ │ ├─ CreateSomethingCommand.php │ │ │ └─ ViewSomethingCommand.php │ │ ├─ config/ │ │ │ ├─ packages/ │ │ │ ├─ routes/ │ │ │ └─ services.yaml │ │ ├─ Entity/ │ │ │ ├─ Something.php │ │ │ └─ User.php │ │ ├─ Event / │ │ ├─ Exception/ │ │ │ └─ InvalidCredentialsException.php │ │ ├─ Handler/ │ │ │ ├─ Auth/ │ │ │ │ ├─ LoginHandler.php │ │ │ │ ├─ RegisterHandler.php │ │ │ │ └─ ResetPasswordHandler.php │ │ │ └─ SomethingHandler.php │ │ ├─ Repository/ │ │ └─ Kernel.php │ └─ Core/ │ ├─ Exception/ │ │ ├─ ResourceExpiredException.php │ │ ├─ UnsupportedContentTypeException.php │ │ └─ ValidationException.php │ ├─ Subscriber/ │ │ ├─ ExceptionSubscriber.php │ │ ├─ RequestSubscriber.php │ │ └─ ResponseSubscriber.php │ └─ Traits/ │ └─ Timestampable.php ... ```
The design patterns used
The structures above introduce changes to the common MVC pattern. If you're into design patterns you've probably already recognized the elements of an ADR (Action-Domain-Responder) pattern as well as the Command Pattern.
The first thing you'll notice is that the Controllers are replaced with Actions. Instead of having a Controller file containing multiple methods (actions), defining several endpoints, this structure uses Actions where each file defines a single endpoint. These Action files are actually Symfony's Invokable Controllers, consisting of a constructor (used for autowiring) and an invoke method, which is invoked when accessing that endpoint. This results in much smaller files, which are much easier to read as well as navigate to. No more opening the controller and scrolling down from method to method, looking for the endpoint you want to work on.
As soon as the request hits an Action, it attempts to instantiate a Command required for handling the request. The commands should contain a factory method which takes the input data and returns an instance of the command. All assertions and validations are to be done in the command's factory method. If anything is wrong with the input data (missing parameter, invalid input, etc.) the factory method must throw an exception. This exception is then caught by the ExceptionSubscriber and a proper 422 response is generated. The command, once instantiated, must contain all the data required for the handler to handle the request. The data in a command instance must be valid, and require no further validation.
Once instantiated, the command is handled via the Command Bus. Matching handler is called and most of the logic happens there. In the example project I've created the simplest handlers as proofs of concept. In a real project, the handler should return a Collection containing Resources or even a complete Representation, depending on your setup. The action should just return a Representation, which is then serialized by the already mentioned ResponseSubscriber.
Requirements
To be able to have a working structure as mentioned above, a few dependencies exist (apart from the Symfony 4 skeleton, of course):
- league/tactician-bundle
- Command pattern implementation
- beberlei/assert
- Input validation in Command's factory method
- harmbandstra/swagger-ui-bundle
- Easy documentation of API's endpoints, we've had its required folders included in the text above
- Easy documentation of API's endpoints, we've had its required folders included in the text above
Recommendations
The following were not covered in this text; however, I strongly suggest you look into them and start using them in your future API projects:
- jms/serializer-bundle
- Absolutely mandatory for supporting multiple content type outputs, allows for easy creation of serializable objects
- lcobucci/jwt
- For jwt-based authentication
- willdurand/hateoas-bundle
- An implementation of HATEOAS REST architecture
- enm/json-api-server-bundle
- An implementation of a JSON:API architecture
Further reading:
- Action-Domain-Responder design pattern: https://en.wikipedia.org/wiki/Action%E2%80%93domain%E2%80%93responder
- Command pattern: https://en.wikipedia.org/wiki/Command_pattern
- JSON:API: https://jsonapi.org/
- HATEOAS: https://en.wikipedia.org/wiki/HATEOAS
- League Tactician (Command pattern): https://tactician.thephpleague.com/