Skip to main content

Camera Usage in Seeed Studio XIAO ESP32S3 Sense

tip

The content of this tutorial is only applicable to Seeed Studio XIAO ESP32S3 Sense.

In this tutorial, we will walk you through using the camera module on the XIAO ESP32S3 Sense. This tutorial is divided into the following parts, first, we will explain the camera function provided by ESP32 and its functions. Secondly, we will bring you how to use the camera in two dimensions: taking pictures and recording videos, and finally, we will create some interesting projects around taking pictures and recording videos.

Seeed Studio XIAO ESP32S3 Sense
Get One Now 🖱️

Getting Started​

This tutorial may involve the use of microSD card, camera, antenna, etc. Please prepare the following materials and install them correctly according to your project needs.

Installation of antenna​

On the bottom left of the front of XIAO ESP32S3, there is a separate "WiFi/BT Antenna Connector". In order to get better WiFi/Bluetooth signal, you need to take out the antenna inside the package and install it on the connector.

There is a little trick to the installation of the antenna, if you press down hard on it directly, you will find it very difficult to press and your fingers will hurt! The correct way to install the antenna is to put one side of the antenna connector into the connector block first, then press down a little on the other side, and the antenna will be installed.

Installation of expansion boards (for Sense)​

Installing the expansion board is very simple, you just need to align the connector on the expansion board with the B2B connector on the XIAO ESP32S3, press it hard and hear a "click", the installation is complete.

We now have a new fully XIAO ESP32S3 Sense-compatible powerful camera, the OV5640, on our shelves, and if you purchase it, you can replace the camera to use it.

Get One Now 🖱️

If you need to know the detailed parameter information of ov5640, you can refer to the following chart.

tip

All the programs about cameras in the Wiki are compatible with both OV5640 and OV2640 cameras.

Prepare the microSD card​

XIAO ESP32S3 Sense supports microSD cards up to 32GB, so if you are ready to buy a microSD card for XIAO, please refer to this specification. And format the microSD card to FAT32 format before using the microSD card.

After formatting, you can insert the microSD card into the microSD card slot. Please note the direction of insertion, the side with the gold finger should face inward.

Camera slot circuit design for expansion boards​

The XIAO ESP32S3 Sense card slot occupies 14 GPIOs of the ESP32-S3, and the pin details of the occupancy are shown in the table below.

ESP32-S3 GPIOCameraESP32-S3 GPIOCamera
GPIO10XMCLKGPIO11DVP_Y8
GPIO12DVP_Y7GPIO13DVP_PCLK
GPIO14DVP_Y6GPIO15DVP_Y2
GPIO16DVP_Y5GPIO17DVP_Y3
GPIO18DVP_Y4GPIO38DVP_VSYNC
GPIO39CAM_SCLGPIO40CAM_SDA
GPIO47DVP_HREFGPIO48DVP_Y9

Turn on the PSRAM option​

The PSRAM of ESP32 refers to the external PSRAM (Pseudo Static Random Access Memory) on the ESP32 chip, which provides additional memory space to increase the available memory of the ESP32 system. In the ESP32 system, PSRAM has the following main uses:

  1. Extending the available RAM: The built-in RAM of ESP32 is limited, especially for some applications that need a lot of memory, such as image processing, audio processing, etc., the built-in RAM may not be enough. By using PSRAM, the available RAM of ESP32 can be extended to meet the needs of these applications.

  2. Accelerated memory access: Since PSRAM is external memory, the access speed is slower than the internal RAM, however it can be used as a cache or temporary memory to accelerate memory access and data processing.

  3. Storage buffers: For applications that require large buffers, such as network buffers, audio buffers, etc., PSRAM can provide enough storage space to avoid out-of-memory situations.

For the content of this tutorial, you need to turn on the PSRAM function of the Arduino IDE to ensure that the camera works properly.

Camera Library Overview​

Before we start, we recommend you to read this chapter to understand the common camera functions. So that you can use these functions to complete your own project development or be able to read the program more easily.

Part I: esp_camera.h​

  1. Configuration structure for camera initialization.

The following is an example of the configuration, just fill in according to the actual pin situation.

static camera_config_t camera_example_config = {
        .pin_pwdn       = PWDN_GPIO_NUM,
        .pin_reset      = RESET_GPIO_NUM,
        .pin_xclk       = XCLK_GPIO_NUM,
        .pin_sccb_sda   = SIOD_GPIO_NUM,
        .pin_sccb_scl   = SIOC_GPIO_NUM,
        .pin_d7         = Y9_GPIO_NUM,
        .pin_d6         = Y8_GPIO_NUM,
        .pin_d5         = Y7_GPIO_NUM,
        .pin_d4         = Y6_GPIO_NUM,
        .pin_d3         = Y5_GPIO_NUM,
        .pin_d2         = Y4_GPIO_NUM,
        .pin_d1         = Y3_GPIO_NUM,
        .pin_d0         = Y2_GPIO_NUM,
        .pin_vsync      = VSYNC_GPIO_NUM,
        .pin_href       = HREF_GPIO_NUM,
        .pin_pclk       = PCLK_GPIO_NUM,

        .xclk_freq_hz   = 20000000, // The clock frequency of the image sensor
        .fb_location = CAMERA_FB_IN_PSRAM; // Set the frame buffer storage location
        .pixel_format   = PIXFORMAT_JPEG, // The pixel format of the image: PIXFORMAT_ + YUV422|GRAYSCALE|RGB565|JPEG
        .frame_size     = FRAMESIZE_UXGA, // The resolution size of the image: FRAMESIZE_ + QVGA|CIF|VGA|SVGA|XGA|SXGA|UXGA
        .jpeg_quality   = 12, // The quality of the JPEG image, ranging from 0 to 63.
        .fb_count       = 2, // The number of frame buffers to use.
        .grab_mode      = CAMERA_GRAB_WHEN_EMPTY //  The image capture mode.
    };
  1. Initialize the camera driver.

After configuring camera_example_config in the above format, we need to use this function to initialize the camera driver.

esp_err_t esp_camera_init(const camera_config_t* config);

  • Input Parameters: Camera configuration parameters

  • Output: ESP_OK on success

note

Currently this function can only be called once and there is no way to de-initialize this module.

  1. Obtain pointer to a frame buffer.
camera_fb_t* esp_camera_fb_get();

Data structure of camera frame buffer:

typedef struct {
    uint8_t * buf;              /*!< Pointer to the pixel data */
    size_t len;                 /*!< Length of the buffer in bytes */
    size_t width;               /*!< Width of the buffer in pixels */
    size_t height;              /*!< Height of the buffer in pixels */
    pixformat_t format;         /*!< Format of the pixel data */
    struct timeval timestamp;   /*!< Timestamp since boot of the first DMA buffer of the frame */
} camera_fb_t;
  1. Return the frame buffer to be reused again.
void esp_camera_fb_return(camera_fb_t * fb);
  • Input Parameters: Pointer to the frame buffer
  1. Get a pointer to the image sensor control structure.
sensor_t * esp_camera_sensor_get();
  • Output: pointer to the sensor
  1. Save camera settings to non-volatile-storage (NVS).
esp_err_t esp_camera_save_to_nvs(const char *key);
  • Input Parameters: A unique nvs key name for the camera settings
  1. Load camera settings from non-volatile-storage (NVS).
esp_err_t esp_camera_load_from_nvs(const char *key);
  • Input Parameters: A unique nvs key name for the camera settings

Part II: img_converters.h​

  1. Convert image buffer to JPEG.
bool fmt2jpg_cb(uint8_t *src, size_t src_len, uint16_t width, uint16_t height, pixformat_t format, uint8_t quality, jpg_out_cb cb, void * arg);

  • Input Parameters:

    • src: Source buffer in RGB565, RGB888, YUYV or GRAYSCALE format
    • src_len: Length in bytes of the source buffer
    • width: Width in pixels of the source image
    • height: Height in pixels of the source image
    • format: Format of the source image
    • quality: JPEG quality of the resulting image
    • cp: Callback to be called to write the bytes of the output JPEG
    • arg: Pointer to be passed to the callback

  • Output: true on success

  1. Convert camera frame buffer to JPEG.
bool frame2jpg_cb(camera_fb_t * fb, uint8_t quality, jpg_out_cb cb, void * arg);

  • Input Parameters:

    • fb: Source camera frame buffer
    • quality: JPEG quality of the resulting image
    • cp: Callback to be called to write the bytes of the output JPEG
    • arg: Pointer to be passed to the callback

  • Output: true on success

  1. Convert image buffer to JPEG buffer.
bool fmt2jpg(uint8_t *src, size_t src_len, uint16_t width, uint16_t height, pixformat_t format, uint8_t quality, uint8_t ** out, size_t * out_len);

  • Input Parameters:

    • src: Source buffer in RGB565, RGB888, YUYV or GRAYSCALE format
    • src_len: Length in bytes of the source buffer
    • width: Width in pixels of the source image
    • height: Height in pixels of the source image
    • format: Format of the source image
    • quality: JPEG quality of the resulting image
    • out: Pointer to be populated with the address of the resulting buffer. You MUST free the pointer once you are done with it.
    • out_len: Pointer to be populated with the length of the output buffer

  • Output: true on success

  1. Convert camera frame buffer to JPEG buffer.
bool frame2jpg(camera_fb_t * fb, uint8_t quality, uint8_t ** out, size_t * out_len);

  • Input Parameters:

    • fb: Source camera frame buffer
    • quality: JPEG quality of the resulting image
    • out: Pointer to be populated with the address of the resulting buffer
    • out_len: Pointer to be populated with the length of the output buffer

  • Output: true on success

  1. Convert image buffer to BMP buffer.
bool fmt2bmp(uint8_t *src, size_t src_len, uint16_t width, uint16_t height, pixformat_t format, uint8_t ** out, size_t * out_len);

  • Input Parameters:

    • src: Source buffer in RGB565, RGB888, YUYV or GRAYSCALE format
    • src_len: Length in bytes of the source buffer
    • width: Width in pixels of the source image
    • height: Height in pixels of the source image
    • format: Format of the source image
    • quality: JPEG quality of the resulting image
    • out: Pointer to be populated with the address of the resulting buffer.
    • out_len: Pointer to be populated with the length of the output buffer

  • Output: true on success

  1. Convert camera frame buffer to BMP buffer.
bool frame2bmp(camera_fb_t * fb, uint8_t ** out, size_t * out_len);

  • Input Parameters:

    • fb: Source camera frame buffer
    • quality: JPEG quality of the resulting image
    • cp: Callback to be called to write the bytes of the output JPEG
    • arg: Pointer to be passed to the callback

  • Output: true on success

Part III: app_httpd.cpp​

note

This part of the library introduction is based on the Create a video preservation terminal -- Based WebServer section. This library is mainly used to perform image acquisition and face recognition functions for the web server. It is not directly included in the onboard package of ESP.

  1. Face recognition function.
static int run_face_recognition(fb_data_t *fb, std::list<dl::detect::result_t> *results)
  • Input Parameters:
    • fb: a pointer to a struct representing a frame buffer containing image data.
    • results: a pointer to a list of detected face results.
  1. Handles HTTP requests for BMP image files.
static esp_err_t bmp_handler(httpd_req_t *req)
  • Input Parameters: a pointer to a struct representing the HTTP request.
  1. Encodes JPEG image data in a streaming fashion.
static size_t jpg_encode_stream(void *arg, size_t index, const void *data, size_t len)
  • Input Parameters:
    • arg: a pointer to a user-defined argument that is passed to the function.
    • index: an index value indicating the current position within the image data.
    • data: a pointer to a buffer containing the image data to be encoded.
    • len: the length of the data buffer.
  1. Handles HTTP requests for capturing and streaming images from camera.
static esp_err_t capture_handler(httpd_req_t *req)
  • Input Parameters: a pointer to a struct representing the HTTP request.
  1. Handles HTTP requests for streaming video from a camera.
static esp_err_t stream_handler(httpd_req_t *req)
  • Input Parameters: a pointer to a struct representing the HTTP request.
  1. Initializes and starts a camera server that captures and streams video over HTTP.
void startCameraServer()

Taking photos with the camera​

Next we start with the most basic usage of the camera, for example, we will first use the camera to complete the image acquisition. The first project we will use the microSD card, the main task of this program is to get the camera footage every minute and then save the footage to the microSD card.

Before you start, do what I did and install the microSD card and the camera.

You can find the complete program code and the required dependency files below this link.

Download the Code

Here is the Arduino program for this project.

#include "esp_camera.h"
#include "FS.h"
#include "SD.h"
#include "SPI.h"

#define CAMERA_MODEL_XIAO_ESP32S3 // Has PSRAM

#include "camera_pins.h"
#include "ESP32_OV5640_AF.h"   // OV5640 auto-focus library (github.com/0015/ESP32-OV5640-AF)

OV5640 ov5640 = OV5640();

#define OV5640_AF_FOCUS_FRAMESIZE FRAMESIZE_SXGA // AF focusing requires >=1280x1024
#define OV5640_FOCUS_TIMEOUT_MS  8000

unsigned long lastCaptureTime = 0; // Last shooting time
int imageCount = 1;                // File Counter
bool camera_sign = false;          // Check camera status
bool sd_sign = false;              // Check sd status

// Save pictures to SD card
void photo_save(const char * fileName) {
  // Take a photo
  camera_fb_t *fb = esp_camera_fb_get();
  if (!fb) {
    Serial.println("Failed to get camera frame buffer");
    return;
  }
  // Save photo to file
  writeFile(SD, fileName, fb->buf, fb->len);
  
  // Release image buffer
  esp_camera_fb_return(fb);

  Serial.println("Photo saved to file");
}

// SD card write file
void writeFile(fs::FS &fs, const char * path, uint8_t * data, size_t len){
    Serial.printf("Writing file: %s\r\n", path);

    File file = fs.open(path, FILE_WRITE);
    if(!file){
        Serial.println("Failed to open file for writing");
        return;
    }
    if(file.write(data, len) == len){
        Serial.println("File written");
    } else {
        Serial.println("Write failed");
    }
    file.close();
}

// Wait for the OV5640 continuous auto-focus to settle (FW_STATUS == 0x10 FOCUSED)
bool waitForOv5640Focus() {
  const uint32_t started = millis();
  uint8_t status = 0;
  while (millis() - started < OV5640_FOCUS_TIMEOUT_MS) {
    status = ov5640.getFWStatus();
    if (status == 0x10) {
      Serial.printf("OV5640 focus settled: FW_STATUS=0x%02X\n", status);
      return true;
    }
    delay(100);
  }
  Serial.printf("ERROR: OV5640 focus not settled within %u ms; last FW_STATUS=0x%02X\n",
                OV5640_FOCUS_TIMEOUT_MS, status);
  return false;
}

void setup() {
  Serial.begin(115200);
  while(!Serial); // When the serial monitor is turned on, the program starts to execute

  camera_config_t config;
  config.ledc_channel = LEDC_CHANNEL_0;
  config.ledc_timer = LEDC_TIMER_0;
  config.pin_d0 = Y2_GPIO_NUM;
  config.pin_d1 = Y3_GPIO_NUM;
  config.pin_d2 = Y4_GPIO_NUM;
  config.pin_d3 = Y5_GPIO_NUM;
  config.pin_d4 = Y6_GPIO_NUM;
  config.pin_d5 = Y7_GPIO_NUM;
  config.pin_d6 = Y8_GPIO_NUM;
  config.pin_d7 = Y9_GPIO_NUM;
  config.pin_xclk = XCLK_GPIO_NUM;
  config.pin_pclk = PCLK_GPIO_NUM;
  config.pin_vsync = VSYNC_GPIO_NUM;
  config.pin_href = HREF_GPIO_NUM;
  config.pin_sscb_sda = SIOD_GPIO_NUM;
  config.pin_sscb_scl = SIOC_GPIO_NUM;
  config.pin_pwdn = PWDN_GPIO_NUM;
  config.pin_reset = RESET_GPIO_NUM;
  config.xclk_freq_hz = 20000000;
  config.frame_size = FRAMESIZE_UXGA;
  config.pixel_format = PIXFORMAT_JPEG; // for streaming
  config.grab_mode = CAMERA_GRAB_WHEN_EMPTY;
  config.fb_location = CAMERA_FB_IN_PSRAM;
  config.jpeg_quality = 12;
  config.fb_count = 1;
  
  // if PSRAM IC present, init with UXGA resolution and higher JPEG quality
  //                      for larger pre-allocated frame buffer.
  if(config.pixel_format == PIXFORMAT_JPEG){
    if(psramFound()){
      config.jpeg_quality = 10;
      config.fb_count = 2;
      config.grab_mode = CAMERA_GRAB_LATEST;
    } else {
      // Limit the frame size when PSRAM is not available
      config.frame_size = FRAMESIZE_SVGA;
      config.fb_location = CAMERA_FB_IN_DRAM;
    }
  } else {
    // Best option for face detection/recognition
    config.frame_size = FRAMESIZE_240X240;
#if CONFIG_IDF_TARGET_ESP32S3
    config.fb_count = 2;
#endif
  }

  // camera init
  esp_err_t err = esp_camera_init(&config);
  if (err != ESP_OK) {
    Serial.printf("Camera init failed with error 0x%x", err);
    return;
  }
  
  camera_sign = true; // Camera initialization check passes

  // --- OV5640 auto-focus init (a key step missing from factory firmware; without it the lens defaults out of focus) ---
  sensor_t *s = esp_camera_sensor_get();
  if (s && s->id.PID == OV5640_PID) {
    Serial.println("OV5640 detected, initializing auto-focus...");
    // Switch to SXGA for focusing (the AF firmware needs >=1280x1024 to evaluate contrast)
    if (s->set_framesize(s, OV5640_AF_FOCUS_FRAMESIZE) != 0) {
      Serial.println("WARN: cannot set SXGA for AF focus, captures will be out of focus");
    } else if (!ov5640.start(s)) {
      Serial.println("WARN: OV5640 CHIPID check failed, AF skipped");
      s->set_framesize(s, config.frame_size); // restore the capture framesize
    } else {
      int r1 = ov5640.focusInit();                       // load focus firmware, 0 = success
      int r2 = (r1 == 0) ? ov5640.autoFocusMode() : -1;  // enable continuous auto-focus
      Serial.printf("  focusInit=%d autoFocusMode=%d\n", r1, r2);
      if (r1 == 0 && r2 == 0 && waitForOv5640Focus()) {
        Serial.println("OV5640 AF ready.");
      } else {
        Serial.println("ERROR: OV5640 AF setup failed; captures will be out of focus.");
      }
      // Restore the capture framesize
      if (s->set_framesize(s, config.frame_size) != 0) {
        Serial.println("WARN: cannot restore capture framesize");
      }
    }
  }

  // Initialize SD card
  if(!SD.begin(21)){
    Serial.println("Card Mount Failed");
    return;
  }
  uint8_t cardType = SD.cardType();

  // Determine if the type of SD card is available
  if(cardType == CARD_NONE){
    Serial.println("No SD card attached");
    return;
  }

  Serial.print("SD Card Type: ");
  if(cardType == CARD_MMC){
    Serial.println("MMC");
  } else if(cardType == CARD_SD){
    Serial.println("SDSC");
  } else if(cardType == CARD_SDHC){
    Serial.println("SDHC");
  } else {
    Serial.println("UNKNOWN");
  }

  sd_sign = true; // sd initialization check passes

  Serial.println("Photos will begin in one minute, please be ready.");
}

void loop() {
  // Camera & SD available, start taking pictures
  if(camera_sign && sd_sign){
    // Get the current time
    unsigned long now = millis();
  
    //If it has been more than 1 minute since the last shot, take a picture and save it to the SD card
    if ((now - lastCaptureTime) >= 60000) {
      char filename[32];
      sprintf(filename, "/image%d.jpg", imageCount);
      photo_save(filename);
      Serial.printf("Saved picture: %s\r\n", filename);
      Serial.println("Photos will begin in one minute, please be ready.");
      imageCount++;
      lastCaptureTime = now;
    }
  }
}
note

The compilation and upload of this program requires two other dependencies, please go to GitHub and download them in full.

Please upload the program for XIAO ESP32S3, after the program is uploaded successfully, please turn on the serial monitor, adjust the camera to face the object you want to shoot, wait for one minute, the shot will be saved to the SD card. Next, XIAO will take a photo every minute.

Remove the microSD card and with the help of the card reader, you can see the photos saved inside the card.

Program annotation​

The program starts with the camera and SD card libraries we need to use, and the import of some pin dependency files we defined for the XIAO ESP32S3.

Then for easy reading, we define two functions one after another, one is the function photo_save() that saves the captured image to the SD card, and the other is the function writeFile() that writes the file.

// Save pictures to SD card
void photo_save(const char * fileName) {
  // Take a photo
  camera_fb_t *fb = esp_camera_fb_get();
  if (!fb) {
    Serial.println("Failed to get camera frame buffer");
    return;
  }
  // Save photo to file
  writeFile(SD, fileName, fb->buf, fb->len);
  
  // Release image buffer
  esp_camera_fb_return(fb);

  Serial.println("Photo saved to file");
}

In the function that saves the image to the microSD card, two main tasks are accomplished. The first one is the acquisition of the picture and the second one is the call to the function that writes the file.

Getting the image can be done with esp_camera_fb_get(), the image information will be saved in the pointer fb, and then we can write the buf of fb to the SD card.

In the Setup() function, a large segment of the program is configuring the camera pins and camera initialization, we can just apply it directly by default. If you have requirements for the camera's pixels or quality, you can adjust the values inside according to the functions described in the Camera Library Overview chapter.

The last thing to do in the loop() function is to control the photos to be taken every minute and follow the incremental number as the file name suffix of the taken photos.

if(camera_sign && sd_sign){
    // Get the current time
    unsigned long now = millis();
  
    //If it has been more than 1 minute since the last shot, take a picture and save it to the SD card
    if ((now - lastCaptureTime) >= 60000) {
      char filename[32];
      sprintf(filename, "/image%d.jpg", imageCount);
      photo_save(filename);
      Serial.printf("Saved picture：%s\n", filename);
      Serial.println("Photos will begin in one minute, please be ready.");
      imageCount++;
      lastCaptureTime = now;
    }
  }

Before executing the loop(), we configure two flag checks camera_sign and sd_sign. This ensures that the task of taking and saving pictures must be run after the camera and SD card checks have been successfully executed in Setup().

Project I: Making a handheld camera​

Next, we use the above theoretical knowledge to create a super small photo artifact. The end result of this project is that the live camera feed is displayed on the Seeed Studio Round Display for XIAO, and when you lock the object you want to photograph, touch the screen and take a picture to record on the microSD card.

Preliminary Preparation​

Before starting this project, you will need to prepare the following hardware in advance.

Seeed Studio XIAO ESP32S3 SenseSeeed Studio Round Display for XIAO
Get One Now 🖱️
Get One Now 🖱️

Since this project will use Round Display for XIAO, please read the contents of the Wiki environment configuration of the display expansion board, install the necessary libraries and configure the TFT environment before you run the routines of this project.

Since the XIAO EPS32S3 Sense is designed with three pull-up resistors R4~R6 connected to the SD card slot, and the round display also has pull-up resistors, the two SD card slots conflict when both boards are used at the same time. There are two ways to solve this problem:

  • Option 1 — no hardware change (used by the example code in this project): keep J3 intact and use the microSD card slot on the XIAO ESP32S3 Sense itself. As demonstrated by engineer Mjrovai, this is possible at software level — the code below initializes the SD card with SD_CS_PIN 21 (the Sense card reader). For details, refer to his methods and procedures.
  • Option 2 — cut J3 and use the Round Display slot: cut off J3 on the XIAO ESP32S3 Sense expansion board. After disconnecting J3, the SD card slot on XIAO ESP32S3 Sense will no longer work, so you need to insert the microSD card into the SD card slot on the Round Display, and change SD_CS_PIN in the code from 21 to D2.
tip

If you choose Option 2, we recommend that you remove the camera module first to avoid scratching the camera when you cut the J3 connection with the blade.

Next, please install the microSD card, XIAO ESP32S3 Sense and Round Display in order.

Specific operation​

You can find the complete program code and the required dependency files below this link.

Download the Code

Here is the Arduino program for this project.

/*
 * This code was adapted from the original, developed by Seeed Studio:
 * https://wiki.seeedstudio.com/xiao_esp32s3_camera_usage/#project-i-making-a-handheld-camera
 *
 * Important changes:
 *  - It is not necessary to to cut off J3 on the XIAO ESP32S3 Sense expansion board.
 *  It is possible to use the XIAO's SD Card Reader. For that you need use SD_CS_PIN as 21.
 *
 * - The camera buffer data should be captured as RGB565 (raw image) with a 240x240 frame size
 * to be displayed on the round display. This raw image should be converted to jpeg before save
 * in the SD card. This can be done with the line:
 * bool ok = frame2jpg(fb, 12, &out_buf, &out_len);
 *
 * - The XCLK_FREQ_HZ should be reduced from 20KHz to 10KHz in order to prevent
 * the message "no EV-VSYNC-OVF message" that appears on Serial Monitor (probably due the time
 * added for JPEG conversion.
 *
 * Adapted by MRovai @02June23
 *
 * Generic OV5640 auto-focus patch (only applies to the OV5640; fixed-focus modules
 * such as the OV3660/NT99141 are skipped automatically):
 *  - After esp_camera_init, run focusInit / autoFocusMode only when PID == OV5640_PID
 *  - In RGB565 mode do not switch to SXGA (it would overflow the fb buffer);
 *    rely on the OV5640's internal full-size image for focus evaluation
*/

#include <Arduino.h>
#include <TFT_eSPI.h>
#include <SPI.h>
#include "esp_camera.h"
#include "ESP32_OV5640_AF.h"   // OV5640 auto-focus library (github.com/0015/ESP32-OV5640-AF), OV5640 only
#include "FS.h"
#include "SD.h"

#define CAMERA_MODEL_XIAO_ESP32S3 // Has PSRAM
#define TOUCH_INT D7

/*
 * NOTE: This sketch keeps J3 intact and uses the microSD card slot on the
 * XIAO ESP32S3 Sense itself (SD_CS_PIN 21), so no hardware change is needed
 * (Mjrovai's software-level method). If you have cut J3 and want to use the
 * Round Display card reader instead, comment out the 21 line and use D2.
 */

#define SD_CS_PIN 21 // ESP32S3 Sense SD Card Reader
//#define SD_CS_PIN D2 // XIAO Round Display SD Card Reader (requires J3 cut)

#include "camera_pins.h"

// Width and height of round display
const int camera_width = 240;
const int camera_height = 240;

// File Counter
int imageCount = 1;
bool camera_sign = false;          // Check camera status
bool sd_sign = false;              // Check sd status

TFT_eSPI tft = TFT_eSPI();

// --- OV5640 AF (OV5640 module only) ---
OV5640 ov5640 = OV5640();
#define OV5640_FOCUS_TIMEOUT_MS 8000

// Wait for the OV5640 continuous auto-focus to settle (FW_STATUS == 0x10 FOCUSED)
bool waitForOv5640Focus() {
  const uint32_t started = millis();
  uint8_t status = 0;
  while (millis() - started < OV5640_FOCUS_TIMEOUT_MS) {
    status = ov5640.getFWStatus();
    if (status == 0x10) {
      Serial.printf("OV5640 focus settled: FW_STATUS=0x%02X\n", status);
      return true;
    }
    delay(100);
  }
  Serial.printf("WARN: OV5640 focus not settled within %u ms; last FW_STATUS=0x%02X\n",
                OV5640_FOCUS_TIMEOUT_MS, status);
  return false;
}

// SD card write file
void writeFile(fs::FS &fs, const char * path, uint8_t * data, size_t len){
    Serial.printf("Writing file: %s\n", path);

    File file = fs.open(path, FILE_WRITE);
    if(!file){
        Serial.println("Failed to open file for writing");
        return;
    }
    if(file.write(data, len) == len){
        Serial.println("File written");
    } else {
        Serial.println("Write failed");
    }
    file.close();
}

bool display_is_pressed(void)
{
    if(digitalRead(TOUCH_INT) != LOW) {
        delay(3);
        if(digitalRead(TOUCH_INT) != LOW)
        return false;
    }
    return true;
}

void setup() {
  Serial.begin(115200);
  delay(300);
  Serial.println();
  Serial.println("==== round_display_take_picture_fix ====");

  // Camera pinout
  camera_config_t config;
  config.ledc_channel = LEDC_CHANNEL_0;
  config.ledc_timer = LEDC_TIMER_0;
  config.pin_d0 = Y2_GPIO_NUM;
  config.pin_d1 = Y3_GPIO_NUM;
  config.pin_d2 = Y4_GPIO_NUM;
  config.pin_d3 = Y5_GPIO_NUM;
  config.pin_d4 = Y6_GPIO_NUM;
  config.pin_d5 = Y7_GPIO_NUM;
  config.pin_d6 = Y8_GPIO_NUM;
  config.pin_d7 = Y9_GPIO_NUM;
  config.pin_xclk = XCLK_GPIO_NUM;
  config.pin_pclk = PCLK_GPIO_NUM;
  config.pin_vsync = VSYNC_GPIO_NUM;
  config.pin_href = HREF_GPIO_NUM;
  config.pin_sscb_sda = SIOD_GPIO_NUM;
  config.pin_sscb_scl = SIOC_GPIO_NUM;
  config.pin_pwdn = PWDN_GPIO_NUM;
  config.pin_reset = RESET_GPIO_NUM;
  config.xclk_freq_hz = 10000000; // Reduced XCLK_FREQ_HZ from 20KHz to 10KHz (no EV-VSYNC-OVF message)
  config.frame_size = FRAMESIZE_240X240;
  config.pixel_format = PIXFORMAT_RGB565;
  config.grab_mode = CAMERA_GRAB_WHEN_EMPTY;
  // RGB565 240x240 uses a single internal-DRAM buffer: TFT_eSPI's SPI DMA can read
  // DRAM directly, while reading from PSRAM crashes pushColors (see the Setup66 + PSRAM issue notes)
  config.fb_location = CAMERA_FB_IN_DRAM;
  config.jpeg_quality = 12;
  config.fb_count = 1;

  // camera init
  esp_err_t err = esp_camera_init(&config);
  if (err != ESP_OK) {
    Serial.printf("Camera init failed with error 0x%x", err);
    return;
  }
  Serial.println("Camera ready");
  camera_sign = true; // Camera initialization check passes

  // --- OV5640 auto-focus init (OV5640 only; fixed-focus modules are skipped) ---
  // A key step missing from factory firmware: esp_camera_init does not set up AF,
  // so the OV5640 defaults to an out-of-focus lens.
  // In RGB565 mode do not switch to SXGA (it would overflow the fb buffer);
  // rely on the OV5640's internal full-size image for focus evaluation.
  sensor_t *s = esp_camera_sensor_get();
  if (s && s->id.PID == OV5640_PID) {
    Serial.println("OV5640 detected, initializing auto-focus...");
    if (!ov5640.start(s)) {
      Serial.println("WARN: OV5640 CHIPID check failed, AF skipped");
    } else {
      int r1 = ov5640.focusInit();                       // load focus firmware, 0 = success
      int r2 = (r1 == 0) ? ov5640.autoFocusMode() : -1;  // enable continuous auto-focus
      Serial.printf("  focusInit=%d autoFocusMode=%d\n", r1, r2);
      if (r1 == 0 && r2 == 0) {
        waitForOv5640Focus();
      } else {
        Serial.println("ERROR: OV5640 AF setup failed; captures will be out of focus.");
      }
    }
  } else {
    Serial.printf("PID=0x%04X (not OV5640), AF skipped - fixed-focus module works as-is.\n",
                  s ? s->id.PID : 0);
  }

  // Display initialization
  tft.init();
  pinMode(TFT_BL, OUTPUT);
  digitalWrite(TFT_BL, HIGH);  // turn on the backlight
  tft.setRotation(1);
  tft.fillScreen(TFT_WHITE);

  // Initialize SD card - Card Reader under the camera (21)
  // Change to D2 for using Display Card Reader
  if(!SD.begin(SD_CS_PIN)){
    Serial.println("Card Mount Failed");
    return;
  }
  uint8_t cardType = SD.cardType();

  // Determine if the type of SD card is available
  if(cardType == CARD_NONE){
    Serial.println("No SD card attached");
    return;
  }

  Serial.print("SD Card Type: ");
  if(cardType == CARD_MMC){
    Serial.println("MMC");
  } else if(cardType == CARD_SD){
    Serial.println("SDSC");
  } else if(cardType == CARD_SDHC){
    Serial.println("SDHC");
  } else {
    Serial.println("UNKNOWN");
  }

  sd_sign = true; // sd initialization check passes
}

void loop() {
  if( sd_sign && camera_sign){

    // Take a photo
    camera_fb_t *fb = esp_camera_fb_get();
    if (!fb) {
      Serial.println("Failed to get camera frame buffer");
      return;
    }

    if(display_is_pressed()){
      Serial.println("display is touched");
      char filename[32];
      sprintf(filename, "/image%d.jpg", imageCount);

      // RGB565 -> JPEG, then save to SD (frame2jpg returns bool: true = success)
      size_t out_len = 0;
      uint8_t* out_buf = NULL;
      bool ok = frame2jpg(fb, 12, &out_buf, &out_len);
      if (ok) {
        // Save photo to file
        writeFile(SD, filename, out_buf, out_len);
        Serial.printf("Saved picture: %s\n", filename);
        imageCount++;
        free(out_buf);
      } else {
        Serial.println("JPEG conversion failed");
      }
    }

    // Draw image on the round display
    uint8_t* buf = fb->buf;
    uint32_t len = fb->len;
    tft.startWrite();
    tft.setAddrWindow(0, 0, camera_width, camera_height);
    tft.pushColors(buf, len);
    tft.endWrite();

    // Release image buffer
    esp_camera_fb_return(fb);

    delay(10);
  }
}

Upload the program to XIAO ESP32S3 Sense, if the screen does not light up after successful upload, you may need to click the Reset button on XIAO, then you will see the monitoring screen displayed on the Round Display in real time. Click anywhere on the screen, the image will be recorded and saved in the microSD card.

Program annotation​

The configuration of the camera and the microSD card are the previous contents, so we will not repeat them here. For the use of microSD card, you can refer to the XIAO ESP32S3 Sense file system Wiki to learn how to use it.

// Take a photo
camera_fb_t *fb = esp_camera_fb_get();
if (!fb) {
  Serial.println("Failed to get camera frame buffer");
  return;
}

...
  
// Release image buffer
esp_camera_fb_return(fb);

delay(10);

The above program, which is the basic block of code for calling the camera, is divided into three parts: screen capture, exception exit, and releasing the photo buffer.

if(display_is_pressed()){
  Serial.println("display is touched");
  char filename[32];
  sprintf(filename, "/image%d.jpg", imageCount);
  // Save photo to file
  writeFile(SD, filename, fb->buf, fb->len);
  Serial.printf("Saved picture：%s\n", filename);
  imageCount++;
}

The above program is used to checks if the screen is touched. If it is, the code saves the captured image to a file on an microSD card.

// Decode JPEG images
uint8_t* buf = fb->buf;
uint32_t len = fb->len;
tft.startWrite();
tft.setAddrWindow(0, 0, camera_width, camera_height);
tft.pushColors(buf, len);
tft.endWrite();

This part of the code displays the captured image on a screen. It first retrieves the image buffer and its length from the camera_fb_t structure. Then, it sets up the screen to receive the image data and displays the image on the screen using the pushColors() function.

Record short videos and save to microSD card​

note

We do not recommend video encoding export on top of MCU because the current supported encoding library resources are too small and the operation is very complicated and tedious.

This example does not involve video encoding, and the exported video is an MJPG composite of AVI per frame, so the video recording may not be particularly good and satisfying. The purpose of this tutorial is to provide you with simple methods and ideas for recording short videos, and we welcome partners who have better solutions to submit PR to us.

In the previous chapters, we mastered how to use the camera to capture images. We know that a single image is stitched together to make a moving video picture. Based on this theory, our project in this chapter will guide you how to write a program to record a 10-second video every 1 minute and save it in a microSD card.

You can find the complete program code and the required dependency files below this link.

Download the Code

Here is the Arduino program for this project.

#include "esp_camera.h"
#include "FS.h"
#include "SD.h"
#include "SPI.h"
#include "esp_timer.h"

#define CAMERA_MODEL_XIAO_ESP32S3 // Has PSRAM

#include "camera_pins.h"

const int SD_PIN_CS = 21;

File videoFile;
bool camera_sign = false;
bool sd_sign = false;
unsigned long lastCaptureTime = 0;
unsigned long captureDuration = 10000; // 10 seconds
int imageCount = 0;

void setup() {
  Serial.begin(115200);
  while(!Serial);
  
  // Initialize the camera
  camera_config_t config;
  config.ledc_channel = LEDC_CHANNEL_0;
  config.ledc_timer = LEDC_TIMER_0;
  config.pin_d0 = Y2_GPIO_NUM;
  config.pin_d1 = Y3_GPIO_NUM;
  config.pin_d2 = Y4_GPIO_NUM;
  config.pin_d3 = Y5_GPIO_NUM;
  config.pin_d4 = Y6_GPIO_NUM;
  config.pin_d5 = Y7_GPIO_NUM;
  config.pin_d6 = Y8_GPIO_NUM;
  config.pin_d7 = Y9_GPIO_NUM;
  config.pin_xclk = XCLK_GPIO_NUM;
  config.pin_pclk = PCLK_GPIO_NUM;
  config.pin_vsync = VSYNC_GPIO_NUM;
  config.pin_href = HREF_GPIO_NUM;
  config.pin_sscb_sda = SIOD_GPIO_NUM;
  config.pin_sscb_scl = SIOC_GPIO_NUM;
  config.pin_pwdn = PWDN_GPIO_NUM;
  config.pin_reset = RESET_GPIO_NUM;
  config.xclk_freq_hz = 20000000;
  config.pixel_format = PIXFORMAT_JPEG;
  config.frame_size = FRAMESIZE_SVGA;
  config.grab_mode = CAMERA_GRAB_WHEN_EMPTY;
  config.fb_location = CAMERA_FB_IN_PSRAM;
  config.jpeg_quality = 12;
  config.fb_count = 1;

  // camera init
  esp_err_t err = esp_camera_init(&config);
  if (err != ESP_OK) {
    Serial.printf("Camera init failed with error 0x%x", err);
    return;
  }
  
  camera_sign = true;
  
  // Initialize the SD card
  if (!SD.begin(SD_PIN_CS)) {
    Serial.println("SD card initialization failed!");
    return;
  }

  uint8_t cardType = SD.cardType();

  // Determine if the type of SD card is available
  if(cardType == CARD_NONE){
    Serial.println("No SD card attached");
    return;
  }

  Serial.print("SD Card Type: ");
  if(cardType == CARD_MMC){
    Serial.println("MMC");
  } else if(cardType == CARD_SD){
    Serial.println("SDSC");
  } else if(cardType == CARD_SDHC){
    Serial.println("SDHC");
  } else {
    Serial.println("UNKNOWN");
  }
  
  sd_sign = true;

  Serial.println("Video will begin in one minute, please be ready.");
}

void loop() {
  // Camera & SD available, start taking video
  if (camera_sign && sd_sign) {
    // Get the current time
    unsigned long now = millis();

    //If it has been more than 1 minute since the last video capture, start capturing a new video
    if ((now - lastCaptureTime) >= 60000) {
      char filename[32];
      sprintf(filename, "/video%d.avi", imageCount);
      videoFile = SD.open(filename, FILE_WRITE);
      if (!videoFile) {
        Serial.println("Error opening video file!");
        return;
      }
      Serial.printf("Recording video：%s\n", filename);
      lastCaptureTime = now;
      
      // Start capturing video frames
      while ((millis() - lastCaptureTime) < captureDuration) {
        camera_fb_t *fb = esp_camera_fb_get();
        if (!fb) {
          Serial.println("Error getting framebuffer!");
          break;
        }
        videoFile.write(fb->buf, fb->len);
        esp_camera_fb_return(fb);
      }
      
      // Close the video file
      videoFile.close();
      Serial.printf("Video saved: %s\n", filename);
      imageCount++;

      Serial.println("Video will begin in one minute, please be ready.");

      // Wait for the remaining time of the minute
      delay(60000 - (millis() - lastCaptureTime));
    }
  }
}

Upload the code to XIAO ESP32S3 Sense, turn on the serial monitor, at this time please adjust the camera position to the object you want to record, after one minute, the orange LED on XIAO will start flashing and the recording will start and save to the microSD card.

note

Since the program does not involve settings such as encoding and frame rate, the video may open for only one second if there is no change in each frame of the recorded footage.

Program annotation​

The core and key in the procedure of recording video is to keep acquiring the photo stream for a continuous period of 10 seconds and write it to the microSD card continuously.

// Start capturing video frames
while ((millis() - lastCaptureTime) < captureDuration) {
  camera_fb_t *fb = esp_camera_fb_get();
  if (!fb) {
    Serial.println("Error getting framebuffer!");
    break;
  }
  videoFile.write(fb->buf, fb->len);
  esp_camera_fb_return(fb);
}

On top of that, we nest a layer of 1-minute wait judgments on the outside to ensure that the video starts every 1 minute.

//If it has been more than 1 minute since the last video capture, start capturing a new video
if ((now - lastCaptureTime) >= 60000) {

  ...

  delay(60000 - (millis() - lastCaptureTime));
}

Project II: Video Streaming​

At the end of this tutorial, let's show a video streaming project. This project allows you to see a live video stream on a web page created by XIAO ESP32S3 Sense, and you can change the display of the screen by setting some parameters.

You can find the complete program code and the required dependency files below this link.

If you are using the esp32 boards package with 2.0.x version on Arduino. Please download with:

Download the Code

If you are using the esp32 boards package with 3.0.x version on Arduino. Please download with:

Download the Code

Here is the Arduino program for this project.

#include "esp_camera.h"
#include "ESP32_OV5640_AF.h"   // OV5640 auto-focus library (OV5640 only)
#include <WiFi.h>

//
// WARNING!!! PSRAM IC required for UXGA resolution and high JPEG quality
//            Ensure ESP32 Wrover Module or other board with PSRAM is selected
//            Partial images will be transmitted if image exceeds buffer size
//
//            You must select partition scheme from the board menu that has at least 3MB APP space.
//            Face Recognition is DISABLED for ESP32 and ESP32-S2, because it takes up from 15
//            seconds to process single frame. Face Detection is ENABLED if PSRAM is enabled as well

// ===================
// Select camera model
// ===================
#define CAMERA_MODEL_XIAO_ESP32S3 // Has PSRAM
#include "camera_pins.h"

// ===========================
// Enter your WiFi credentials
// ===========================
const char *ssid = "**********";
const char *password = "**********";

void startCameraServer();
void setupLedFlash(int pin);

// --- OV5640 AF (OV5640 module only; fixed-focus modules such as the OV3660/NT99141 are skipped automatically) ---
OV5640 ov5640 = OV5640();
#define OV5640_AF_FOCUS_FRAMESIZE FRAMESIZE_SXGA  // AF focus evaluation requires >=1280x1024
#define OV5640_FOCUS_TIMEOUT_MS 8000
bool waitForOv5640Focus() {
  const uint32_t started = millis();
  uint8_t status = 0;
  while (millis() - started < OV5640_FOCUS_TIMEOUT_MS) {
    status = ov5640.getFWStatus();
    if (status == 0x10) { Serial.printf("OV5640 focus settled: FW_STATUS=0x%02X\n", status); return true; }
    delay(100);
  }
  Serial.printf("WARN: OV5640 focus not settled; last FW_STATUS=0x%02X\n", status);
  return false;
}

void setup() {
  Serial.begin(115200);
  Serial.setDebugOutput(true);
  Serial.println();

  camera_config_t config;
  config.ledc_channel = LEDC_CHANNEL_0;
  config.ledc_timer = LEDC_TIMER_0;
  config.pin_d0 = Y2_GPIO_NUM;
  config.pin_d1 = Y3_GPIO_NUM;
  config.pin_d2 = Y4_GPIO_NUM;
  config.pin_d3 = Y5_GPIO_NUM;
  config.pin_d4 = Y6_GPIO_NUM;
  config.pin_d5 = Y7_GPIO_NUM;
  config.pin_d6 = Y8_GPIO_NUM;
  config.pin_d7 = Y9_GPIO_NUM;
  config.pin_xclk = XCLK_GPIO_NUM;
  config.pin_pclk = PCLK_GPIO_NUM;
  config.pin_vsync = VSYNC_GPIO_NUM;
  config.pin_href = HREF_GPIO_NUM;
  config.pin_sccb_sda = SIOD_GPIO_NUM;
  config.pin_sccb_scl = SIOC_GPIO_NUM;
  config.pin_pwdn = PWDN_GPIO_NUM;
  config.pin_reset = RESET_GPIO_NUM;
  config.xclk_freq_hz = 20000000;
  config.frame_size = FRAMESIZE_UXGA;
  config.pixel_format = PIXFORMAT_JPEG;  // for streaming
  //config.pixel_format = PIXFORMAT_RGB565; // for face detection/recognition
  config.grab_mode = CAMERA_GRAB_WHEN_EMPTY;
  config.fb_location = CAMERA_FB_IN_PSRAM;
  config.jpeg_quality = 12;
  config.fb_count = 1;

  // if PSRAM IC present, init with UXGA resolution and higher JPEG quality
  //                      for larger pre-allocated frame buffer.
  if (config.pixel_format == PIXFORMAT_JPEG) {
    if (psramFound()) {
      config.jpeg_quality = 10;
      config.fb_count = 2;
      config.grab_mode = CAMERA_GRAB_LATEST;
    } else {
      // Limit the frame size when PSRAM is not available
      config.frame_size = FRAMESIZE_SVGA;
      config.fb_location = CAMERA_FB_IN_DRAM;
    }
  } else {
    // Best option for face detection/recognition
    config.frame_size = FRAMESIZE_240X240;
#if CONFIG_IDF_TARGET_ESP32S3
    config.fb_count = 2;
#endif
  }

#if defined(CAMERA_MODEL_ESP_EYE)
  pinMode(13, INPUT_PULLUP);
  pinMode(14, INPUT_PULLUP);
#endif

  // camera init
  esp_err_t err = esp_camera_init(&config);
  if (err != ESP_OK) {
    Serial.printf("Camera init failed with error 0x%x", err);
    return;
  }

  sensor_t *s = esp_camera_sensor_get();
  // --- OV5640 Heat Optimization: lower IO drive (0x302C bit[7:6] 11->00) ---
  if (s && s->id.PID == OV5640_PID) {
    s->set_reg(s, 0x302C, 0xC0, 0x00);
  }
  // --- OV5640 auto-focus init (OV5640 only; fixed-focus modules are skipped) ---
  if (s && s->id.PID == OV5640_PID) {
    Serial.println("OV5640 detected, initializing auto-focus...");
    if (s->set_framesize(s, OV5640_AF_FOCUS_FRAMESIZE) == 0 && ov5640.start(s)) {
      int r1 = ov5640.focusInit();
      int r2 = (r1 == 0) ? ov5640.autoFocusMode() : -1;
      Serial.printf("  focusInit=%d autoFocusMode=%d\n", r1, r2);
      if (r1 == 0 && r2 == 0) waitForOv5640Focus();
      s->set_framesize(s, config.frame_size);  // restore the capture framesize
    } else {
      Serial.println("WARN: OV5640 AF setup failed, skipped");
      s->set_framesize(s, config.frame_size);
    }
  }
  // initial sensors are flipped vertically and colors are a bit saturated
  if (s->id.PID == OV3660_PID) {
    s->set_vflip(s, 1);        // flip it back
    s->set_brightness(s, 1);   // up the brightness just a bit
    s->set_saturation(s, -2);  // lower the saturation
  }
  // drop down frame size for higher initial frame rate
  if (config.pixel_format == PIXFORMAT_JPEG) {
    s->set_framesize(s, FRAMESIZE_QVGA);
  }

#if defined(CAMERA_MODEL_M5STACK_WIDE) || defined(CAMERA_MODEL_M5STACK_ESP32CAM)
  s->set_vflip(s, 1);
  s->set_hmirror(s, 1);
#endif

#if defined(CAMERA_MODEL_ESP32S3_EYE)
  s->set_vflip(s, 1);
#endif

// Setup LED FLash if LED pin is defined in camera_pins.h
#if defined(LED_GPIO_NUM)
  setupLedFlash(LED_GPIO_NUM);
#endif

  WiFi.begin(ssid, password);
  WiFi.setSleep(false);

  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.println("WiFi connected");

  startCameraServer();

  Serial.print("Camera Ready! Use 'http://");
  Serial.print(WiFi.localIP());
  Serial.println("' to connect");
}

void loop() {
  // Do nothing. Everything is done in another task by the web server
  delay(10000);
}

Before uploading the program, you need to change the WiFi name and password in the code to your own. After uploading the program, if the XIAO ESP32C3 successfully connects to your WiFi, its IP address will be printed out.

caution

XIAO ESP32S3 will get very hot during long-running streaming — please be careful of burns. To reduce the temperature, apply the OV5640 Heat Optimization patch below.

tip

As you can see in the above figure, if you turn on the output of debug information, then you may see some of the chip kernel's debug information printed out in the serial monitor. For example [0;31mE (2947) MFN: Partition Not found[0m, please don't worry about it, it doesn't affect the running of the program.

Please open your browser, we recommend Edge or Google Chrome, and enter that IP address and you will see the configuration page for the video.

note

Please note that the device you are using the browser on needs to be on the same LAN as XIAO.

After configuring the video stream specifications you want to set up, click Start Stream at the bottom of the left toolbar and you will see the live feed of the camera.

Fortunately, ESP32 has also officially added face recognition to the program. You can experience the feature by turning on the button switch for face recognition with reduced picture quality.

tip

For performance reasons, the quality of the screen cannot be higher than CIF, otherwise the web page will pop up with an error when you turn on the switch for face recognition.

Oh, my big face is circled.

OV5640 AutoFocus​

Hadware Preparation​

OV5640 Camera for XIAO ESP32S3 Sense
Get One Now 🖱️

Software Preparation​

Method 1​

Special thanks to @Eric for providing the open source code

Download the Libraries

Code Example

#include "esp_camera.h"
#include <WiFi.h>
#include "ESP32_OV5640_AF.h"

#define CAMERA_MODEL_XIAO_ESP32S3 // Has PSRAM

#include "camera_pins.h"

const char* ssid = "";
const char* password = "";

void startCameraServer();
void setupLedFlash(int pin);
OV5640 ov5640 = OV5640();

void setup() {
  Serial.begin(115200);
  while(!Serial);
  Serial.setDebugOutput(true);
  Serial.println();

  camera_config_t config;
  config.ledc_channel = LEDC_CHANNEL_0;
  config.ledc_timer = LEDC_TIMER_0;
  config.pin_d0 = Y2_GPIO_NUM;
  config.pin_d1 = Y3_GPIO_NUM;
  config.pin_d2 = Y4_GPIO_NUM;
  config.pin_d3 = Y5_GPIO_NUM;
  config.pin_d4 = Y6_GPIO_NUM;
  config.pin_d5 = Y7_GPIO_NUM;
  config.pin_d6 = Y8_GPIO_NUM;
  config.pin_d7 = Y9_GPIO_NUM;
  config.pin_xclk = XCLK_GPIO_NUM;
  config.pin_pclk = PCLK_GPIO_NUM;
  config.pin_vsync = VSYNC_GPIO_NUM;
  config.pin_href = HREF_GPIO_NUM;
  config.pin_sscb_sda = SIOD_GPIO_NUM;
  config.pin_sscb_scl = SIOC_GPIO_NUM;
  config.pin_pwdn = PWDN_GPIO_NUM;
  config.pin_reset = RESET_GPIO_NUM;
  config.xclk_freq_hz = 20000000;
  config.frame_size = FRAMESIZE_UXGA;
  config.pixel_format = PIXFORMAT_JPEG; // for streaming
  //config.pixel_format = PIXFORMAT_RGB565; // for face detection/recognition
  config.grab_mode = CAMERA_GRAB_WHEN_EMPTY;
  config.fb_location = CAMERA_FB_IN_PSRAM;
  config.jpeg_quality = 12;
  config.fb_count = 1;
  
  // if PSRAM IC present, init with UXGA resolution and higher JPEG quality
  //                      for larger pre-allocated frame buffer.
  if(config.pixel_format == PIXFORMAT_JPEG){
    if(psramFound()){
      config.jpeg_quality = 10;
      config.fb_count = 2;
      config.grab_mode = CAMERA_GRAB_LATEST;
    } else {
      // Limit the frame size when PSRAM is not available
      config.frame_size = FRAMESIZE_SVGA;
      config.fb_location = CAMERA_FB_IN_DRAM;
    }
  } else {
    // Best option for face detection/recognition
    config.frame_size = FRAMESIZE_240X240;
#if CONFIG_IDF_TARGET_ESP32S3
    config.fb_count = 2;
#endif
  }

  // camera init
  esp_err_t err = esp_camera_init(&config);
  if (err != ESP_OK) {
    Serial.printf("Camera init failed with error 0x%x", err);
    return;
  }

  sensor_t * s = esp_camera_sensor_get();
  // --- OV5640 Heat Optimization: lower IO drive (0x302C bit[7:6] 11->00) ---
  if (s && s->id.PID == OV5640_PID) {
    s->set_reg(s, 0x302C, 0xC0, 0x00);
  }
  ov5640.start(s);

    if (ov5640.focusInit() == 0) {
    Serial.println("OV5640_Focus_Init Successful!");
  }

  if (ov5640.autoFocusMode() == 0) {
    Serial.println("OV5640_Auto_Focus Successful!");
  }

// Setup LED FLash if LED pin is defined in camera_pins.h
#if defined(LED_GPIO_NUM)
  setupLedFlash(LED_GPIO_NUM);
#endif

  WiFi.begin(ssid, password);
  WiFi.setSleep(false);

  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.println("WiFi connected");

  startCameraServer();

  Serial.print("Camera Ready! Use 'http://");
  Serial.print(WiFi.localIP());
  Serial.println("' to connect");
}

void loop() {
  uint8_t rc = ov5640.getFWStatus();
  Serial.printf("FW_STATUS = 0x%x\n", rc);

  if (rc == -1) {
    Serial.println("Check your OV5640");
  } else if (rc == FW_STATUS_S_FOCUSED) {
    Serial.println("Focused!");
  } else if (rc == FW_STATUS_S_FOCUSING) {
    Serial.println("Focusing!");
  }
}

Results Chart​

tip

The resolution needs to be above 1280*1024 to see the focusing effect, the screen will lag and you need to wait for the screen for a while when focusing.

Method 2​

tip

The resolution needs to be above 1600*1200 to see the focusing effect, the screen will lag and you need to wait for the screen for a while when focusing.

Download the following zip file and add it to Arudino

tip

The OV5640 in the libraries of method 1 and method 2 cannot exist at the same time

#include "esp_camera.h"
#include <WiFi.h>
#include "ESP32_OV5640_AF.h"

#define CAMERA_MODEL_XIAO_ESP32S3 // Has PSRAM

#include "camera_pins.h"

const char *ssid = "";
const char *password = "";

void startCameraServer();
void setupLedFlash(int pin);
OV5640 ov5640 = OV5640();

void setup() {
  Serial.begin(115200);

  camera_config_t config;
  config.ledc_channel = LEDC_CHANNEL_0;
  config.ledc_timer = LEDC_TIMER_0;
  config.pin_d0 = Y2_GPIO_NUM;
  config.pin_d1 = Y3_GPIO_NUM;
  config.pin_d2 = Y4_GPIO_NUM;
  config.pin_d3 = Y5_GPIO_NUM;
  config.pin_d4 = Y6_GPIO_NUM;
  config.pin_d5 = Y7_GPIO_NUM;
  config.pin_d6 = Y8_GPIO_NUM;
  config.pin_d7 = Y9_GPIO_NUM;
  config.pin_xclk = XCLK_GPIO_NUM;
  config.pin_pclk = PCLK_GPIO_NUM;
  config.pin_vsync = VSYNC_GPIO_NUM;
  config.pin_href = HREF_GPIO_NUM;
  config.pin_sscb_sda = SIOD_GPIO_NUM;
  config.pin_sscb_scl = SIOC_GPIO_NUM;
  config.pin_pwdn = PWDN_GPIO_NUM;
  config.pin_reset = RESET_GPIO_NUM;
  config.xclk_freq_hz = 20000000;
  config.frame_size = FRAMESIZE_UXGA;
  config.pixel_format = PIXFORMAT_JPEG;
  config.grab_mode = CAMERA_GRAB_LATEST;
  config.fb_location = CAMERA_FB_IN_PSRAM;
  config.jpeg_quality = 10;
  config.fb_count = 2;

  if(psramFound()){
    config.jpeg_quality = 10;
    config.fb_count = 2;
    config.grab_mode = CAMERA_GRAB_LATEST;
  } else {
    // Limit the frame size when PSRAM is not available
    config.frame_size = FRAMESIZE_SVGA;
    config.fb_location = CAMERA_FB_IN_DRAM;
  }

  esp_err_t err = esp_camera_init(&config);
  if (err != ESP_OK) {
    Serial.printf("Camera init failed with error 0x%x", err);
    return;
  }

  // auto focus
#if 1
  sensor_t* sensor = esp_camera_sensor_get();
  // --- OV5640 Heat Optimization: lower IO drive (0x302C bit[7:6] 11->00) ---
  if (sensor && sensor->id.PID == OV5640_PID) {
    sensor->set_reg(sensor, 0x302C, 0xC0, 0x00);
  }
  int ret = 0;
  ov5640.start(sensor);

  if (ov5640.focusInit() == 0) {
      Serial.println("OV5640_Focus_Init Successful!");
  } else {
      Serial.println("OV5640_Focus_Init Failed!");
  }

  ret = ov5640.autoFocusMode(false);
  if (ret == 0) {
    Serial.println("OV5640_Auto_Focus Successful!");
  } else {
    Serial.printf("OV5640_Auto_Focus Failed! - [%d]\n", ret);
  }
#endif

  WiFi.begin(ssid, password);
  WiFi.setSleep(false);

  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("\nWiFi connected");

  startCameraServer();

  Serial.printf("Camera Ready! Use 'http://%s' to connect\n", WiFi.localIP().toString().c_str());
}

void loop() {
  if (Serial.available()) {
    sensor_t* sensor = esp_camera_sensor_get();
    int ret = 0;

    switch (Serial.read()) {
      case 'b':
        ret = sensor->set_reg(sensor, 0x3022, 0xff, 0x03);
        Serial.printf("begin to auto focus - %d\n", ret);
        break;
      case 's':
        ret = sensor->set_reg(sensor, 0x3022, 0xff, 0x06);
        Serial.printf("focus stop here - %d\n", ret);
        break;
    }
  }

  uint8_t rc = ov5640.getFWStatus();
  Serial.printf("FW_STATUS = 0x%x\n", rc);

  if (rc == -1) {
    Serial.println("Check your OV5640");
  } else if (rc == FW_STATUS_S_FOCUSED) {
    Serial.println("Focused!");
  } else if (rc == FW_STATUS_S_FOCUSING) {
    Serial.println("Focusing!");
  } else {
  }

  delay(1000);
}
tip

Mode 1 is recommended because it has a more pronounced focusing effect than Mode 2 and provides a sharper image.

OV5640 Heat Optimization​

The OV5640 module heats up noticeably during long-running use. This section offers two ways to bring the temperature down: tuning the software parameters (Method 1), and lowering the IO drive capability through a register configuration (Method 2).

Method 1: Reduce heat by tuning parameters​

The example code in this Wiki is tuned for the best possible image quality — UXGA resolution (1600×1200) with double-buffered continuous frame grabbing in the photo examples, and a 240MHz CPU — so the board heats up noticeably during long runs. If you find the temperature too high, or you do not need that much resolution, you can lower the power consumption in the following ways, most of which do not affect the sharpness of single captures:

LeverWhat to changeHeat reductionAffects image quality?
CPU frequencyArduino IDE: Tools → CPU Frequency: 240MHz → 160MHz (or add setCpuFrequencyMhz(160); in setup())MediumNo (sensor imaging and JPEG encoding are done inside the OV5640, independent of the CPU)
Focus frequencySwitch continuous auto-focus (autoFocusMode) to periodically triggered single-shot focus, or lengthen the focus intervalSmallUnchanged for still scenes; moving scenes take longer to re-focus
Frame grabbingCAMERA_GRAB_LATEST → CAMERA_GRAB_WHEN_EMPTY (the camera fills available buffers, which reduces the continuous frame processing; it is not on-demand capture and does not put the sensor in standby)MediumThe frame you grab may be a few frames older than with CAMERA_GRAB_LATEST; sharpness is unaffected
ResolutionFRAMESIZE_UXGA (1600×1200) → FRAMESIZE_HD (1280×720) or SVGA (800×600)LargeFewer pixels, so less detail when enlarged (per-pixel sharpness is unchanged)
Double bufferingfb_count from 2 to 1 (together with the grab-mode change above)MediumNo

The best-value combination is to use a lower resolution and a single frame buffer to reduce capture-processing overhead — that is, one step down in resolution plus CAMERA_GRAB_WHEN_EMPTY with fb_count = 1. The temperature drops noticeably, with almost no impact on the sharpness of single captures. Note that CAMERA_GRAB_WHEN_EMPTY fills available buffers; it does not provide on-demand sensor capture or standby.

Method 2: Reduce power by lowering the IO drive capability​

tip

This section applies only to the OV5640 camera module. The OV2640 has no such register and is skipped automatically by the PID check in the code below.

When the OV5640 runs continuously for a long time — video streaming, live preview, or continuous auto-focus — the chip temperature can rise above 70°C. This is a hardware-level root cause and cannot be fully eliminated in software; it can only be mitigated.

Why the OV5640 heats up​

  1. Internal DVDD regulator drop: when the I/O supply (DOVDD) is 2.8V and the internal DVDD regulator is used, the large voltage drop itself dissipates significant power (noted in the official datasheet).
  2. Improper power-up sequence introduces leakage current and raises static power consumption.
  3. The datasheet strongly recommends cutting all power (including external DVDD) when the sensor is not in use, which confirms that OV5640 standby consumption is not ideal.

The only effective software lever is to lower the IO pin drive current to reduce dynamic power.

How it works — register 0x302C​

Register 0x302C (named DRIVE_CAPABILITY in the esp32-camera source) controls the drive capability of the IO pins (D0–D7, PCLK, VSYNC, HREF). The key field is Bit[7:6]:

Bit[7:6]Drive capability
114x (chip power-on default)
103x
012x
001x (weakest, lowest current, least heat)

Changing Bit[7:6] from 11 to 00 drops the drive capability from 4x to 1x. That is the entire principle — there is no other magic involved.

Code — add this right after esp_camera_init()​

// --- OV5640 Heat Optimization ---
// Lower IO drive capability via register 0x302C.
// Bit[7:6]: 11=4x (chip default) -> 00=1x (lowest current, least heat).
// Drops chip temperature from ~70°C+ to ~54°C during continuous streaming.
// OV5640 only; the PID check skips OV2640, which has no such register.
sensor_t * s = esp_camera_sensor_get();
if (s && s->id.PID == OV5640_PID) {
  s->set_reg(s, 0x302C, 0xC0, 0x00);
}
  • esp_camera_init(&config) — called as usual, the function is not modified. Internally it calls sensor.reset(), which writes a batch of default registers.
  • esp_camera_sensor_get() — gets the sensor_t * pointer needed to talk to the sensor.
  • if (s->id.PID == OV5640_PID) — only applies to the OV5640. All examples in this wiki are also compatible with the OV2640, and 0x302C is OV5640-specific, so it must not be written to an OV2640. The video-streaming example already uses the same if (s->id.PID == OV3660_PID) pattern.
  • s->set_reg(s, 0x302C, 0xC0, 0x00) — reads back 0x302C, clears only Bit[7:6] to 00 (mask 0xC0 = 0b11000000, value 0x00), and leaves the other bits at whatever the library set. This is safer than writing 0x03 directly — it does not depend on the library's default value and touches only the drive-capability bits.

Effect and risks​

  • Effect: under long-running continuous streaming, the OV5640 chip temperature drops from ~70°C+ to ~54°C (varies with ambient temperature and heat-sink).
  • Risk: lower drive capability slows signal edges, which raises the bar for PCB trace noise immunity; in extreme cases it may introduce image noise or sampling distortion. On the XIAO ESP32S3 Sense + OV5640 (heat-sink version) the impact is acceptable in testing.
  • Scope: OV5640 only; the OV2640 has no such register and is skipped by the PID check.

Troubleshooting​

Q1: When XIAO ESP32S3 Sense and Round Display are used together, do I have to cut the J3 pin? Which SD card slot can be used?​

A: In principle, you need to cut the J3 pin when XIAO ESP32S3 Sense is used together with Round Display to use microSD card. The reason is that there are pull-up resistors in the design of the circuit of both expansion boards, so theoretically, if two pull-up resistors work at the same time, then the SD card slot will not work properly. An error message of SD card mount failure will appear. Since the pull-up resistor on the Round Display cannot be blocked, you need to cut J3 on the sense expansion board to make sure only one pull-up resistor is working when the two are used together. This also determines that when both are used together, there is and only the SD card slot on the Round Display is active.

However, we need to thank engineer Mjrovai for the new method of using the microSD card slot on the XIAO ESP32S3 Sense at the same time, which is also possible at software level. We can refer to his methods and procedures.

Q2: The OV5640 gets very hot during long-running streaming. What should I do?​

A: This is expected for the OV5640 under continuous capture. Lower the IO drive capability via register 0x302C as described in OV5640 Heat Optimization — it drops the chip temperature from ~70°C+ to ~54°C. If you do not need high resolution, also lowering the frame size and switching to CAMERA_GRAB_WHEN_EMPTY further reduces power.

Tech Support & Product Discussion​

Thank you for choosing our products! We are here to provide you with different support to ensure that your experience with our products is as smooth as possible. We offer several communication channels to cater to different preferences and needs.

Loading Comments...