API Documentation - Test Documentation

A post in continuation of the topic of experimental solutions , from where the code for an example will be reused. In a previous post, I touched on the topic of how to write tests for a simple service, when it acts as a black box and from the test code we do not have direct access to the code of the program under test. Once again, I’ll dwell on the fact that the service under test was implemented in the Go language, and the tests for the service in the Ruby language and the RSpec testing framework. The stack was chosen from our own preferences and is not of key importance to the topic under consideration. In this article I want to consider the issue of documenting the API, again using a not-so-standard solution.

There are several basic approaches to writing documentation:

  • Just write the documentation manually.
  • Automatically generate documentation for comments in code.
  • Automatic generation of code documentation.

In the first case, the disadvantages are obvious, you must manually maintain the documentation, in addition, the text of the documentation is separated from the code. In the second case, documentation is most often generated from comments on classes, methods, and functions. In this case, it is much easier to maintain an up-to-date description, in addition, there may be additional checks, such as the fact that the signature of the function matches the description of the parameters in the comment. Automatic code generation should eliminate the problem of supporting an up-to-date description of API methods. These methods are widely used for documenting libraries. However, automatic generation is extremely rare for the WEB API, one of the possible reasons is that all such generators are framework dependent and require some support for reflection from the web framework and programming language.

Now briefly consider a few less common ways to compile (and use) documentation:

  • We write machine-readable documentation, generate code for the server and client code template on it.
  • Having machine-readable documentation, we check in the tests whether the API behavior matches the description in the documentation, for example, that the response of the method corresponds to what is written in the documentation.

I’ll explain right away why API documentation (including machine-readable ones) may be needed before the API implementation itself is written. Most often, this situation occurs when both the API server and the client code are being developed simultaneously. Having machine-readable documentation at the stage of client development, it becomes possible to generate both the API client itself and the server emulating real answers.

I also want to note that such protocols as GRPC, Apache Thrift and the like actually make you first write (and then maintain) a machine-readable description of the API and only then write an implementation, which can undoubtedly carry some annoying effect from the need to constantly edit the files with the protocol description, but on the other hand, we are always sure that at least the description of the signatures corresponds to reality.

And the last (in this text) way of obtaining documentation:

  • We generate documentation for API methods for a real response from the server. Requests to the server are encoded for tests.

And now this is the subject of the title. Since the test code is an integral part of the system under test, you can quite cheaply put additional functionality on the tests. I will highlight the advantages of this method:

  • Inside the documentation we get 100% relevant examples of answers from API methods, and all possible usage scenarios if they are separately covered by tests.
  • We can automatically generate documentation without making changes to the source code of the service. Thus, we are no longer dependent on the support of auto documentation with a specific framework or technology,
  • In general, we may not have access to the source code of the service, but we can obtain, including machine-readable documentation, and use all its advantages.

In this example, the Swagger specification will be used to document the API, I will skip the general description of this tool, since there are enough such reviews. But I note that this is machine-readable documentation that allows you to generate both human documentation and server and client code templates for working with the described API.

In the vastness of the network, rspec-rails-swagger and rswag gems were found . Both unfortunately have a minimum, but binding to rails. Gems have quite detailed documentation with examples and fairly simple code. As an experiment, I untied the rspec-rails-swagger gem from Rails and connected to existing functional tests.

The description of the test for generating documentation for the method is as follows:

describe 'swagger_docs' do
    let(:movies_resp_body) { File.read('spec/fixtures/movies.json') }
    path '/movies' do
        operation "GET", summary: "respond 200 OK" do
            parameter :rating, in: :query, type: :string, required: false, description: "filter by rating"
            response 200, description: "successful" do
                    type: :array,
                    items: {
                        type: :string,

This code runs execution tests and uses the Rspec syntax extension to specify the meta information that will be used to generate the swagger file.

Run rspec through a long command:

bundle exec rspec -f RSpec::Rails::Swagger::Formatter --order defined -t swagger_object

The -t flag filters run tests only by those that use special gem syntax. The -f flag enables the formatter to output the result as a swagger json file.
The command output will produce a valid swagger file that can be loaded into swagger-ui or try to use it to generate a client for the API.

In the end, we have functional tests on the Rspec framework, along the way generating swagger documentation, from which you can quickly get the client API blank for any other programming language. Full listing example .

In summary:

  • Automated documentation guarantees some minimal match between your expectations from the API and the harsh changeable reality.
  • In some cases, the developer is physically forced to first describe the API methods and only then their implementation (GRPC, Apache Thrift protocols).
  • Automatically generated machine-readable documentation can simplify life both at the development stage and at the stage of support and work with the API.
  • Using special tools, you can generate machine-readable documentation without making changes to the source code.