Home

Awesome

Camera API for micropython

ESP32 Port

This project aims to support various cameras (e.g. OV2640, OV5640) on different MicroPython ports, starting with the ESP32 port. The project implements a general API, has precompiled FW images and supports a lot of cameras out of the box. Defaults are set to work with the OV2640. At the moment, this is a micropython user module, but it might get in the micropython repo in the future. The API is stable, but it might change without previous announce.

Content

Precomiled FW (the easy way)

If you are not familiar with building custom firmware, visit the releases page to download firmware that suits your board. There are over 20 precompiled board images with the latest micropython!

Using the API

Importing the camera module

from camera import Camera, GrabMode, PixelFormat, FrameSize, GainCeiling

Creating a camera object

Camera construction using defaults. This is the case if you are using a non-generic precompiled firmware or if you specified the camera model or pins in mpconfigboard.h during your build. Then you can just call the construction without any keyword arguments.

cam = Camera()

or with relevant keyword arguments:

cam = Camera(pixel_format=PixelFormat.JPEG,
    frame_size=FrameSize.QVGA,
    jpeg_quality=90,
    fb_count=2,
    grab_mode=GrabMode.WHEN_EMPTY)

When using a generic precompiled firmware, the camera constructor requires specific keyword arguments (namely the camera pins to be used). These pins are just examples and if used as-is, a error will occur. Adapt them to your board!

cam = Camera(
    data_pins=[1,2,3,4,5,6,7,8],
    vsync_pin=9,
    href_pin=10,
    sda_pin=11,
    scl_pin=12,
    pclk_pin=13,
    xclk_pin=14,
    xclk_freq=20000000,
    powerdown_pin=-1,
    reset_pin=-1,
)

Keyword arguments for construction:

Default values:

The following keyword arguments have default values:

Initializing the camera

cam.init()

Capture image

img = cam.capture()

Arguments for capture

Convert image to another format

You can either convert the image with the capture method directly passing the desired output format:

img_rgb888 = cam.capture(PixelFormat.RGB888) #capture image as configured (e.g. JPEG), convert it to RGB888 and return the converted image

Or you can first capture the image and then convert it to the desired PixelFormat with the convert method. Doing so you can have both, the captured and the converted image. Note that more memory will be used.

img = cam.capture()
img_rgb888 = cam.convert(PixelFormat.RGB888) #converts the last captured image to RGB888 and returns the converted image

Convertion supported

Camera reconfiguration

cam.reconfigure(pixel_format=PixelFormat.JPEG,frame_size=FrameSize.QVGA,grab_mode=GrabMode.LATEST, fb_count=2)

Keyword arguments for reconfigure

Additional methods

Here are just a few examples:

cam.set_quality(90)  # The quality goes from 0% to 100%, meaning 100% is the highest but has probably no compression
cam.set_bmp_out(True) # Enables convertion to bmp when capturing image
camera.get_brightness()
camera.set_vflip(True) #Enable vertical flip

See autocompletions in Thonny in order to see the list of methods. If you want more insides in the methods and what they actually do, you can find a very good documentation here. Note that each method requires a "get_" or "set_" prefix, depending on the desired action.

To get the version of the camera driver used:

import camera
vers = camera.Version()

Additional information

The FW images support the following cameras out of the box, but is therefore big: OV7670, OV7725, OV2640, OV3660, OV5640, NT99141, GC2145, GC032A, GC0308, BF3005, BF20A6, SC030IOT

Build your custom FW

Setting up the build environment (DIY method)

To build the project, follow these instructions:

  espressif/esp32-camera:
    git: https://github.com/espressif/esp32-camera.git

Alternatively, you can clone the https://github.com/espressif/esp32-camera repository inside the esp-idf/components folder instead of altering the idf_component.yml file.

Add camera configurations to your board (optional, but recommended)

Supported camera models

This project supports various boards with camera interface out of the box. You typically only need to add a single line to your board config file ("mpconfigboard.h). Example (don't forget to add the empty line at the bottom):

#define MICROPY_CAMERA_MODEL_WROVER_KIT       1

Below is a list of supported MICROPY_CAMERA_MODEL_xxx definitions:

For unsupported camera models

If your board is not yet supported, add the following lines to your board config-file "mpconfigboard.h" with the respective pins and camera parameters. Otherwise, you will need to pass all parameters during construction. Example for Xiao sense:

#define MICROPY_CAMERA_PIN_D0       (15)
#define MICROPY_CAMERA_PIN_D1       (17)
#define MICROPY_CAMERA_PIN_D2       (18)
#define MICROPY_CAMERA_PIN_D3       (16)
#define MICROPY_CAMERA_PIN_D4       (14)
#define MICROPY_CAMERA_PIN_D5       (12)
#define MICROPY_CAMERA_PIN_D6       (11)
#define MICROPY_CAMERA_PIN_D7       (48)
#define MICROPY_CAMERA_PIN_PCLK     (13)
#define MICROPY_CAMERA_PIN_VSYNC    (38)
#define MICROPY_CAMERA_PIN_HREF     (47)
#define MICROPY_CAMERA_PIN_XCLK     (10)
#define MICROPY_CAMERA_PIN_PWDN     (-1)
#define MICROPY_CAMERA_PIN_RESET    (-1)
#define MICROPY_CAMERA_PIN_SIOD     (40)        // SDA
#define MICROPY_CAMERA_PIN_SIOC     (39)        // SCL
#define MICROPY_CAMERA_XCLK_FREQ    (20000000)  // Frequencies are normally either 10 MHz or 20 MHz
#define MICROPY_CAMERA_FB_COUNT     (2)         // The value is between 1 (slow) and 2 (fast, but more load on CPU and more ram usage)
#define MICROPY_CAMERA_JPEG_QUALITY (85)        // Quality of JPEG output in percent. Higher means higher quality.
#define MICROPY_CAMERA_GRAB_MODE    (1)         // 0=WHEN_EMPTY (might have old data, but less resources), 1=LATEST (best, but more resources)

Customize additional camera settings

If you want to customize additional camera setting or reduce the FW size by removing support for unused camera sensors, then take a look at the kconfig file of the esp32-camera driver and specify these on the sdkconfig file of your board.

Build the API

To build the project, you could do it the following way:

. <path2esp-idf>/esp-idf/export.sh
cd MyESPCam/micropython/ports/esp32
make USER_C_MODULES=../../../../micropython-camera-API/src/micropython.cmake BOARD=<Your-Board> clean
make USER_C_MODULES=../../../../micropython-camera-API/src/micropython.cmake BOARD=<Your-Board> submodules
make USER_C_MODULES=../../../../micropython-camera-API/src/micropython.cmake BOARD=<Your-Board> all

If you experience problems, visit MicroPython external C modules.

Notes

Benchmark

I didn't use a calibrated osziloscope, but here is a FPS benchmark with my ESP32S3 (xclck_freq = 20MHz, GrabMode=LATEST, fb_count = 1, jpeg_quality=85%) and OV2640. Using fb_count=2 theoretically can double the FPS (see JPEG with fb_count=2). This might also aplly for other PixelFormats.

Frame SizeGRAYSCALERGB565YUV422JPEGJPEG -> RGB565JPEG -> RGB888JPEG (fb=2)
R96X9612.512.512.5No imgNo imgNo imgNo img
QQVGA12.512.512.525252550
QCIF111111.525252550
HQVGA12.512.512.52516.716.750
R240X24012.512.511.52516.712.550
QVGA12111225252550
CIF12.5No imgNo img6.38.38.312.5
HVGA332.512.56.36.325
VGA33312.53.63.625
SVGA33312.52.82.525
XGANo imgNo imgNo img6.31.61.612.5
HDNo imgNo imgNo img6.31.41.312.5
SXGA2226.31112.5
UXGANo imgNo imgNo img6.30.70.712.5

Looking at the results: image conversion make only sense for frame sized below QVGA or if capturing the image in the intended pixelformat and frame size combination fails.

Troubleshooting

You can find information on the following sites:

Donate

If you enjoy this work and would like to share your enjoyment, please feel free to donate and contribute to the project. Thanks! :blush: