esp-web-tools: Arduino 2.0.4, PlatformIO 5.1.0, LittleFS is corrupted

Hi all, as soon as I update Arduino Core from 2.0.3 to 2.0.4 and PlatformIO from 5.0.0 to 5.1.0 my Little FS is corrupted after flashing my firmware with esp web tools.

this is my simple json:

{
   "name":"Glow Worm Luciferin FULL",
   "version":"5.6.3",
   "new_install_improv_wait_time":40,
   "builds":[
      {
         "chipFamily":"ESP32",
         "parts":[
            {
               "path":"GlowWormLuciferinFULL_ESP32_firmware-factory.bin",
               "offset":0
            }
         ]
      },
      {
         "chipFamily":"ESP8266",
         "parts":[
            {
               "path":"GlowWormLuciferinFULL_ESP8266_firmware.bin",
               "offset":0
            }
         ]
      }
   ]
}

and this is the script that I use to create a factory bin.

def esp32_create_factory_bin(source, target, env):
    print("Generating factory bin for genuine esp units")
    #offset = 0x1000
    offset = 0x0
    new_file_name = env.subst("$BUILD_DIR/${PROGNAME}-factory.bin")
    sections = env.subst(env.get('FLASH_EXTRA_IMAGES'))
    new_file = open(new_file_name,"wb")
    for section in sections:
        sect_adr,sect_file = section.split(" ",1)
        source = open(sect_file,"rb")
        new_file.seek(int(sect_adr,0)-offset)
        new_file.write(source.read());
        source.close()

    firmware = open(env.subst("$BUILD_DIR/${PROGNAME}.bin"),"rb")
    new_file.seek(0x10000-offset)
    new_file.write(firmware.read())
    new_file.close()
    firmware.close()

env.AddPostAction("$BUILD_DIR/${PROGNAME}.bin", esp32_create_factory_bin)

It worked well before but now it is broken… As soon as I flash the firmware with Arduino core 2.0.4, LittleFS got corrupted. It works well with Arduino core 2.0.3. I think that this can be related to flash mode (QIO is now needed for some boards), is there a way to specify the flash mode with esp web tools?

Any suggestions will be really appreciated 😃

Thanks Davide

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 30 (20 by maintainers)

Most upvoted comments

I see a lot of beautiful minds on this issue, except for mine obviously. xD

I reduced the Jason2866 script to be more “compliant” with smaller projects like mine.

Import("env")

env = DefaultEnvironment()
platform = env.PioPlatform()

from genericpath import exists
import sys
from os.path import join

sys.path.append(join(platform.get_package_dir("tool-esptoolpy")))
import esptool

FRAMEWORK_DIR = platform.get_package_dir("framework-arduinoespressif32")
variants_dir = join(FRAMEWORK_DIR, "variants", "luciferin")

def esp32_create_combined_bin(source, target, env):
    #print("Generating combined binary for serial flashing")
    # factory_offset = -1      # error code value - currently unused
    app_offset = 0x10000     # default value for "old" scheme
    fs_offset = -1           # error code value
    new_file_name = env.subst("$BUILD_DIR/${PROGNAME}-factory.bin")
    sections = env.subst(env.get("FLASH_EXTRA_IMAGES"))
    firmware_name = env.subst("$BUILD_DIR/${PROGNAME}.bin")
    chip = env.get("BOARD_MCU")
    flash_size = env.BoardConfig().get("upload.flash_size", "4MB")
    flash_freq = env.BoardConfig().get("build.f_flash", "40000000L")
    flash_freq = str(flash_freq).replace("L", "")
    flash_freq = str(int(int(flash_freq) / 1000000)) + "m"
    flash_mode = env.BoardConfig().get("build.flash_mode", "dout")
    if flash_mode == "qio":
        flash_mode = "dio"
    elif flash_mode == "qout":
        flash_mode = "dout"
    cmd = [
        "--chip",
        chip,
        "merge_bin",
        "-o",
        new_file_name,
        "--flash_mode",
        flash_mode,
        "--flash_freq",
        flash_freq,
        "--flash_size",
        flash_size,
    ]

    print("    Offset | File")
    for section in sections:
        sect_adr, sect_file = section.split(" ", 1)
        print(f" -  {sect_adr} | {sect_file}")
        cmd += [sect_adr, sect_file]

    # "main" firmware to app0 - mandatory
    print(f" - {hex(app_offset)} | {firmware_name}")
    cmd += [hex(app_offset), firmware_name]

    if(fs_offset != -1):
        fs_bin = join(env.subst("$BUILD_DIR"),"littlefs.bin")
        if exists(fs_bin):
            before_reset = env.BoardConfig().get("upload.before_reset", "default_reset")
            after_reset = env.BoardConfig().get("upload.after_reset", "hard_reset")
            print(f" - {hex(fs_offset)}| {fs_bin}")
            cmd += [hex(fs_offset), fs_bin]
            env.Replace(
                UPLOADERFLAGS=[
                    "--chip", chip,
                    "--port", '"$UPLOAD_PORT"',
                    "--baud", "$UPLOAD_SPEED",
                    "--before", before_reset,
                    "--after", after_reset,
                    "write_flash", "-z",
                    "--flash_mode", "${__get_board_flash_mode(__env__)}",
                    "--flash_freq", "${__get_board_f_flash(__env__)}",
                    "--flash_size", flash_size
                ],
                UPLOADCMD='"$PYTHONEXE" "$UPLOADER" $UPLOADERFLAGS ' + " ".join(cmd[7:])
            )
            print("Will use custom upload command for flashing operation to add file system defined for this build target.")

    # print('Using esptool.py arguments: %s' % ' '.join(cmd))

    esptool.main(cmd)

env.AddPostAction("$BUILD_DIR/${PROGNAME}.bin", esp32_create_combined_bin)

It now works awesome!

I’m honored to be helped by you guys! Bye!

sblantipodi on Aug 10, 2022

Can you open an issue on esptool.js to track that, and note what errors you see.

Will do, back on PC not until tomorrow.

Jason2866 on Aug 10, 2022

@TD-er yes. It should follow the same logic as esptool.py. Espressif wants to keep the esptool.js port 1:1 the same as esptool.py (including how functions are broken out etc).

Great! Then I can think of a few more feature requests 😉

TD-er on Aug 9, 2022

@TD-er yes. It should follow the same logic as esptool.py. Espressif wants to keep the esptool.js port 1:1 the same as esptool.py (including how functions are broken out etc).

balloob on Aug 9, 2022