Problem Description

I can't upload a sketch on an ESP8266EX with ArduinoOTA library (using ArduinoIDE 1.8.13), which is on a PCB I designed (Crystal frequency: 40MHz and Flash size: 4MB --> Mfr. Part: HT25WD40ARZ and LCSC Part: C2928800).

The code used is the BasicOTA example given for the "Generic ESP8266 Module" boards. The code I want to upload is essentially the same.

The PCB is perfectly functional, and the serial upload works very well.
I first tried with version 3.1.2 of the board manager on the Arduino IDE, then with the latest version of git.
Plus, firewall is disabled.
In my research I saw that it could be a problem related to the size of the Flash memory, but I tried all sizes without success.

https://github.com/esp8266/Arduino/discussions/7955

When uploading serially I get this output:

Connecting....
    Chip is ESP8266EX
    Features: WiFi
    Crystal is 40MHz
    MAC: a4:e5:7c:27:00:93
    Uploading stub...
    Running stub...
    Stub running...
    Changing baud rate to 512000
    Changed.
    Configuring flash size...
    Auto-detected Flash size: 512KB
    Flash params set to 0x0300
    Compressed 320112 bytes to 231885...
    Writing at 0x00000000... (6 %)
    Writing at 0x00004000... (13 %)
    ...
    Writing at 0x00038000... (100 %)
    Wrote 320112 bytes (231885 compressed) at 0x00000000 in 4.9 seconds (effective 525.2 kbit/s)...
    Hash of data verified.
    
    Leaving...
    Hard resetting via RTS pin...

It's showing me a flash size of 512KB when I've designed the card with 4MB, which is odd enough, but it works.

MCVE Sketch

#include <ESP8266WiFi.h>
    #include <ESP8266mDNS.h>
    #include <WiFiUdp.h>
    #include <ArduinoOTA.h>
    
    #ifndef STASSID
    #define STASSID "your-ssid"
    #define STAPSK "your-password"
    #endif
    
    const char* ssid = STASSID;
    const char* password = STAPSK;
    
    void setup() {
      Serial.begin(115200);
      Serial.println("Booting");
      WiFi.mode(WIFI_STA);
      WiFi.begin(ssid, password);
      while (WiFi.waitForConnectResult() != WL_CONNECTED) {
        Serial.println("Connection Failed! Rebooting...");
        delay(5000);
        ESP.restart();
      }
    
      // Port defaults to 8266
      // ArduinoOTA.setPort(8266);
    
      // Hostname defaults to esp8266-[ChipID]
      // ArduinoOTA.setHostname("myesp8266");
    
      // No authentication by default
      // ArduinoOTA.setPassword("admin");
    
      // Password can be set with it's md5 value as well
      // MD5(admin) = 21232f297a57a5a743894a0e4a801fc3
      // ArduinoOTA.setPasswordHash("21232f297a57a5a743894a0e4a801fc3");
    
      ArduinoOTA.onStart([]() {
        String type;
        if (ArduinoOTA.getCommand() == U_FLASH) {
          type = "sketch";
        } else {  // U_FS
          type = "filesystem";
        }
    
        // NOTE: if updating FS this would be the place to unmount FS using FS.end()
        Serial.println("Start updating " + type);
      });
      ArduinoOTA.onEnd([]() {
        Serial.println("\nEnd");
      });
      ArduinoOTA.onProgress([](unsigned int progress, unsigned int total) {
        Serial.printf("Progress: %u%%\r", (progress / (total / 100)));
      });
      ArduinoOTA.onError([](ota_error_t error) {
        Serial.printf("Error[%u]: ", error);
        if (error == OTA_AUTH_ERROR) {
          Serial.println("Auth Failed");
        } else if (error == OTA_BEGIN_ERROR) {
          Serial.println("Begin Failed");
        } else if (error == OTA_CONNECT_ERROR) {
          Serial.println("Connect Failed");
        } else if (error == OTA_RECEIVE_ERROR) {
          Serial.println("Receive Failed");
        } else if (error == OTA_END_ERROR) {
          Serial.println("End Failed");
        }
      });
      ArduinoOTA.begin();
      Serial.println("Ready");
      Serial.print("IP address: ");
      Serial.println(WiFi.localIP());
    }
    
    void loop() {
      ArduinoOTA.handle();
    }

Debug Messages

I get the result on the ESP's serial port at update:

Start updating sketch
    Progress: 0%
    Progress: 0%
    Progress: 0%
    Error[4]: End Failed

and sometimes depending on the Flash memory selected (here: 512KB, FS: 32KB OTA: ~230KB):

Start updating sketch
    Progress: 0%
    Progress: 0%
    Progress: 0%
    Progress: 1%
    Error[4]: End Failed

This is what I get from the console of the IDE for the sketch I want to upload:

 . Variables and constants in RAM (global, static), used 29064 / 80192 bytes (36%)
    ║   SEGMENT  BYTES    DESCRIPTION
    ╠══ DATA     1496     initialized variables
    ╠══ RODATA   1392     constants       
    ╚══ BSS      26176    zeroed variables
    . Instruction RAM (IRAM_ATTR, ICACHE_RAM_ATTR), used 60543 / 65536 bytes (92%)
    ║   SEGMENT  BYTES    DESCRIPTION
    ╠══ ICACHE   32768    reserved space for flash instruction cache
    ╚══ IRAM     27775    code in IRAM   
    . Code in flash (default, ICACHE_FLASH_ATTR), used 287064 / 1048576 bytes (27%)
    ║   SEGMENT  BYTES    DESCRIPTION
    ╚══ IROM     287064   code in flash   
    
    ...
    
    An error occurred while uploading the sketch
    Uploading.................................................
    15:08:09 [ERROR]: Error Uploading

Basic Infos

• I have tested that the issue is present in current master branch (aka latest git).
• I have searched the issue tracker for a similar issue.
• I have filled out all fields below.

Platform

• Hardware: [ESP8266EX]
• Core Version: [latest git hash]
• Development Env: [Arduino IDE 1.8.13]
• Operating System: [Windows]

Settings in IDE

• Module: [Generic ESP8266 Module]
• Flash Mode: [DOUT]
• Flash Size: [4MB]
• lwip Variant: [v2 Lower Memory]
• Reset Method: [dtr]
• Flash Frequency: [40Mhz]
• CPU Frequency: [160MHz]
• Upload Using: [OTA]
• Upload Speed: [115200]

 

You need to have enough memory available for the current program plus the program you are trying to upload. If it is reporting 512k and attempting to upload 287064 with probably roughly the same size already in memory it's going to fail. I had a similar problem with a 1M unit I have and it really doesn't give any indication why it failed. After looking at the numbers it made sense... the new program was too big to fit in the memory.

When you update by OTA it writes the new program in flash not being used by the current program and when it reboots it jumps to the new starting point and ignores the old program still in memory.

My guess is you'll have to figure out why it's not reporting the memory correctly and get that working right first.