Awesome
Openapi Generator
An openapiv3 generator which can generate micro-services for GO Language
Content
- Openapi Generator
- Openapi Document
- Develop In Generated Project
- Todo
Introduction
We know openapi is cool, usually we use swagger or postman for api testing, documentation and design. However, design, develop, testing and documentation is redundant work and spend us lot of time.
This generator able to generate micro-services with below capability:
- working gin mock server, you can modified it to become real microservice
- unit test for all restapi
- build in swagger-ui to allow you test api easily
- easily generate docker image
- build in support security scheme
Quickstart
Make sure you api document comply to Requirements, then you can generate micro-service source code:
- Download openapigenerator
- prepare openapi-v3 spec file, (tested in .yaml only)
- Execute below command :
./openapigenerator-mac.bin --apifile="simpleapi.yaml" --targetfolder=myproject --projectname="project1" --port="9000" --lang="go"
- You can use Visual Studio Code or others IDE to open
~/myproject
:
cd myproject
code .
- Copy .env.default to .env, define appropriate value, like API keys and etc
- Run below command and done:
make && ./project1
- Try access rest api interface: http://localhost:9000/doc/swagger-ui/index.html
Dockerizing
Project come with few command help you run microservice in docker environment. Check Makefile
to know more detail
- create docker container in devnetwork.
cd myproject
make dockerremove ; make docker && make dockerrun
make dockershell # access docker shell
2 You may edit Makefile
if you wish to pass create docker container with more useful setting
Openapi Document
Pure openapi not good enough for prepare complete microservices, there is additional guide line shall follow at below section.
Requirements
This project has below requirement:
- api document written in openapi v3
- api document shall save in .yaml format
- define operationId in every api in path, with value:
- Alphabet value which can use to define as programming function title
- No special characters allowed, except prefix with '-' (avoid generate route handle)
- example:
GetData
,SaveData
,-SaveData
- All element shall refere from #components/schema, example:
- parameters
- request body
- response content
Not Supported Features
- openconnect security schemes
- apiKey only supported at header, others area is not supported (cookie, query...)
Define Environment Variables
We can use Specification Extensions to declare environment variable for microservices, it will add into .env.default
.
Example:
x-env-vars:
MONGO_SERVER: mongodbserver
MONGO_DB: db1
MONGO_USER:
MONGO_PASS:
Define Error Schema
You can define special schema "Error", which shall include 2 field: err_code
and err_msg
as below example.
http request for post
,put
which you may submit requestbody.
Error:
type: object
properties:
err_code:
type: string
example: "ER-MG-001"
err_msg:
description: A human readable error message
type: string
example: "Mongodb not connected"
version:
type: string
example: v1
How it work:
- This schema can define as error 4xx response for http request with requestbody (post, put, patch)
- when the actual request body not compatible with schema defination, the
err_msg
will display error like below:
{
"err_code": "ERR_INPUT_VALIDATION",
"err_msg": "Key: 'Model_Book.Bookid' Error:Field validation for 'Bookid' failed on the 'required' tag",
"version": "v1"
}
Require 2XX and 4XX In Every Response
For good practise, every http request shall define at least 1 response for 2xx and 4xx. There is hardcoded behaviour which will identity 2xx as success response, and 4xx is failed response.
Develop In Generated Project
After code generated, we can start over development by change the generated code. You shall prepare below dev environment:
- Visual Studio Code
- Go Sdk
- Docker (optional if you play with docker)
Project Overview
The generated project prepare under below structure:
- written in go language
- Running on top of gin web service
- It work as complete mock server to service micro-service as stated in openapi document.
Development Concept
To make the micro-service perform real task, we shall perform development, which involve below scope:
-
Modify route handle to perform real task:
route
is restapi request likeGET /myapi
,POST /myapi/res1
route handle
is programming function, which will trigger when client access specificroute
.- Every
route
connect to ownroute handle
route
is terminology fromgin
, it have similar meaning with openapipath
-
Unit test (optional)
-
Prepare environment for deployment like:
- environment variables in
.env
or.env.docker
- prepre dockers images
- prepare database for handle
- environment variables in
On and off, depends on project requirement you need to perform modification at openapi document, and regenerate the code again, and again. It is important to know how to regenerate the code without overwrite your modification.
Technically it has few rules:
- Remain all file with prefix Z*.go remain unchange (openapi/Z*.go test/Z*.go)
- During development, create new
.go
file in folderopenapi/
- Created file should not start with
Z
, to avoid it mixed with generated file - Root level file as below guideline:
go.main
: always overwrite by generatorgo.mod
: always overwrite by generatorMakefile
: always overwrite by generator.env.default
: always overwrite by generator.env
: free to change.env.docker
: free to change
File Structure
Below is some explanation generated code
main.go
: entry pointgo.mod
: go project setting.env.default
: template for.env
Makefike
: store command for build project, you may change if you have complex requirementDockerfile
: template for generate docker image, change it if you have special need on dockeropenapi/
: store all go program, you shall perform develop in this folder onlyopenapi/ZModel_*.go
: All openapi schema will generate as goModel
here (structs)openapi/ZRouterRegistry.go
: List all the route/path in this microservice, and connect to which route handleopenapi/ZRouterHandle.go
: Keep all default auto generateroute handle
, base on name defined in openapi's path's operationId.openapi/ZSecurity*.go
: security schemes setting according openapi documentopenapi/ZServer.go
: store code to run gin serverapi/api.yaml
: store openapi document, used by swagger-uidist/*
: project binary build into this folder
Schemas Or Models
Openapi's schema equivalent to model
in this project, and all model store as openapi/ZModel_*.go
. In go, Model
is kind of struct, suitable interface, getter/setter/validator was prepared.
The model is important cause it act as pattern of api output. Route handle comply output pattern using specific model
.
Routes
Route
equivalent to rest api request method
+ path
, example
- Get /api/service1
- Post /api/v1/service2
- Put /api
- Delete /api/crud
Route Handles
Route handle
is programmingfunction
likeGetStudents()
which is trigger byroute
- openapi document
operationId
declare name ofroute handle
, and generator will help you prepare dummy route handle.
Develop Real Route Handle
To provide real data in micro-service, we perform 3 step below:
Transfer Route Handle Into New File
- Create new file
openapi/routehandles.go
(or others name you like) Cut and paste
specific function fromopenapi/ZRouteHandle.go
intoopenapi/routehandles.go
. Let's assumegetMemoryInfo(c *gin.Context){}
- remain
getMemoryInfo()
inZRouteHandle.go
will cause duplicate function and cause error
- remain
- Edit content of
openapi/routehandles.go
to serve real data:
// auto generate by generator
package openapi
import (
"fmt"
"runtime"
"github.com/gin-gonic/gin"
// "net/http"
)
func getMemoryInfo(c *gin.Context) {
var m runtime.MemStats
runtime.ReadMemStats(&m)
// For info on each, see: https://golang.org/pkg/runtime/#MemStats
allocated := bToMb(m.TotalAlloc)
available := bToMb(m.Sys)
percent := int((allocated/available)*100) / 100
allocatedstr := fmt.Sprintf("%v MB", allocated)
availablestr := fmt.Sprintf("%v MB", available)
percentstr := fmt.Sprintf("%v percent", percent)
c.Header("Content-Type", "application/json")
//
// type Model_MemoriesInfo struct{
// Percent string `json:"percent" binding:""` //
// Total string `json:"total" binding:""` //
// Used string `json:"used" binding:""` //
// Version string `json:"version"`
// }
data := Model_MemoriesInfo{} //Model_MemoriesInfo defined at openapi/ZModel_MemoriesInfo.go
data.SetTotal(percentstr)
data.SetPercent(availablestr)
data.SetUsed(allocatedstr)
data.SetVersion("v1")
c.JSON(200, data)
}
func bToMb(b uint64) uint64 {
return b / 1024 / 1024
}
Define Handle Exists In Document
- Edit the original openapi document, add entry
x-operationId-exists
:
# bottom of file
x-operationId-exists:
getMemoryInfo: true
- Try regenerate the source code,
getMemoryInfo()
no longer generate inZRouteHandle.go
Create Environment Variables
Lot of time, we wish to use configure microservice according deployment requirement, such as:
-
Define database server location and credentials
-
On/Off specific functions
-
Define apikey and etc
-
Define environment variable in openapi document using
x-env-vars
and regenerate the source code. Example in openapi document:
# bottom of file
x-operationId-exists:
getMemoryInfo: true
x-env-vars:
APIVERSION: v1.1
Generated .env.default
will automatically add below entry
APIVERSION=v1.1
- load environment variable in route handle using
godotenv.load()
andos.Getenv("APIVERSION")
. Example of updated code:
package openapi
import (
"fmt"
"os"
"runtime"
"github.com/gin-gonic/gin"
"github.com/joho/godotenv"
log "github.com/sirupsen/logrus"
// "net/http"
)
func getMemoryInfo(c *gin.Context) {
err := godotenv.Load()
if err != nil {
log.Fatal("Error loading .env file")
}
verionno := os.Getenv("APIVERSION")
var m runtime.MemStats
runtime.ReadMemStats(&m)
// For info on each, see: https://golang.org/pkg/runtime/#MemStats
allocated := bToMb(m.TotalAlloc)
available := bToMb(m.Sys)
percent := int((allocated/available)*100) / 100
allocatedstr := fmt.Sprintf("%v MB", allocated)
availablestr := fmt.Sprintf("%v MB", available)
percentstr := fmt.Sprintf("%v percent", percent)
c.Header("Content-Type", "application/json")
//
// type Model_MemoriesInfo struct{
// Percent string `json:"percent" binding:""` //
// Total string `json:"total" binding:""` //
// Used string `json:"used" binding:""` //
// }
data := Model_MemoriesInfo{} //Model_MemoriesInfo defined at openapi/ZModel_MemoriesInfo.go
data.SetTotal(percentstr)
data.SetPercent(availablestr)
data.SetUsed(allocatedstr)
data.SetVersion(verionno)
c.JSON(200, data)
}
func bToMb(b uint64) uint64 {
return b / 1024 / 1024
}
- define environment variable via:
- edit
.env
in non-docker build - edit
.env.docker
in docker build - or, modify it in command line,
export X_API_Key=12345
- edit
Build Project
We shall build the project to run the micro-service, follow below instruction.
Simple Build
- run below command:
make
- To run the service: Linux/Mac:
./<your-project-name>
Windows: double click the filename <your-project-name>
Build For Distribution
If you wish to build binary for specific OS and distribute manually. Folow step
- Run below command:
make windows #build for windows
make linux #build for linux
make mac #build for mac
make mac-arm #build for mac m1 type processor
- Distrubute file in
dist/
, along with suitable.env
file
Build For Docker
We can build and run docker image via below step:
- Copy
.env.default
to.env.docker
, and define appropriate content - run command:
make dockerremove #remove existing docker
make docker # create docker image only
make dockerrun # create docker container
or we can simplified as single row command:
make dockerremove; make docker && make dockerrun
- We can change Docker image build from different source by edit
Dockerfile
. Like replacealpine
toubuntu
- We can change docker image/container/network by edit
Makefile
Unit Test
Generator help you prepare reasonable structure for unit test, however you need to copy content into suitable file to prevent effort overwritten when regenerate code.
You need to install grc
package for unit test
https://jakeholmquist.medium.com/add-some-fun-to-your-cli-with-grc-ea868df985b6
Define Real Test
- expand folder ./test, open any file (assume ZGet_Memory_test.go)
- scroll down and follow comment, copy content into test/Get_Memory.go (File name shall follow comment)
// copy and modify below content and put into new file Get_Memory.go (in this test folder)
/*
package test
import (
"io"
...
- after step 2, error in ZGet_Memory_test.go should disappear
- Modify
Get_Memory.go
if neccessary to serve real content - repeat same step 1-4 for others file until all error disappear:
- if request body is required,
FunctionName_RequestBody()
will fill in sample data which defined in openapi document - You can change whole structure of unit test as long as the original function name remain
- Don't change any file start with Z*.go, cause it will overwritten by generator
- if request body is required,
Execute Unit Test
Ensure microservice is activated, run below command and see the result:
make apitest
Secure Microservice
We can secure restapi via define secruityScheme in openapi document.
Secure With Apikey
- Add security Schemes. Below is apiKey example:
components:
securitySchemes:
BasicApiKey:
type: apiKey
in: header
name: X-Api-Key #environment variable X_Api_Key will prepare automatically
- Define which api use it:
paths:
/api1:
get:
summary: welcome
description: show msg undefine resource
operationId: "welcome"
security:
- BasicApiKey: [] # Get /api1 require X-Api-Key defined in header
- Re-generate the source code:
X_Api_Key
will automatically prepare in.env.default
openapi/ZSecurity_BasicApiKeyy.go
generated- unit test template use env var X_Api_Key in header
- Update
.env
and.env.docker
if both file exists, and restart the micro-service - Try the
Get /api
again with headerX-Api-Key
as:
http get localhost:<portno>/api1 X-Api-Key:<you-key-code>
Todo
- Add
x-generator-setting
to direct set project name, port number and etc suitable configuration - rename this project to prevent crash name with official openapi-generator
- support more component type
- client generators for different kind of languages
- add some common template for
- crud for different kind of database
- messaging template for sms, email, push notifications
- openapi 3.1
- more securityschemes
- more complete data validation