Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

When digital ROI goes beyond image size #3043

Open
EwaldvanGeffen opened this issue Apr 20, 2024 · 1 comment
Open

When digital ROI goes beyond image size #3043

EwaldvanGeffen opened this issue Apr 20, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@EwaldvanGeffen
Copy link

EwaldvanGeffen commented Apr 20, 2024 •
edited
Loading

The Problem

In my case when my digit ROI delta Y exceeded beyond the bottom of the demo picture (so the bottom red horizontal box isn't visible because its outside the image). The image in image recognition for this digit got corrupted. I had other smaller sized digit ROI that would still function and not have visual corruption.

I think you need to limit digitroi.x digitroi.deltaX <= image.x and similar for y so it cannot go out of bounds.

Version

15.7

Logfile

[0d00h00m00s] 2024-04-20T00:14:15 <INF> [MAIN] =================================================
[0d00h00m00s] 2024-04-20T00:14:15 <INF> [MAIN] ==================== Start ======================
[0d00h00m00s] 2024-04-20T00:14:15 <INF> [MAIN] =================================================
[0d00h00m00s] 2024-04-20T00:14:15 <INF> [MAIN] PSRAM size: 8388608 byte (8MB / 64MBit)
[0d00h00m00s] 2024-04-20T00:14:15 <INF> [MAIN] Total heap: 4380199 byte
[0d00h00m04s] 2024-04-20T00:14:20 <INF> [MAIN] Camera info: PID: 0x26, VER: 0x42, MIDL: 0x7f, MIDH: 0xa2
[0d00h00m04s] 2024-04-20T00:14:20 <INF> [SDCARD] Basic R/W check started...
[0d00h00m04s] 2024-04-20T00:14:20 <INF> [SDCARD] Basic R/W check successful
[0d00h00m04s] 2024-04-20T00:14:20 <INF> [SNTP] TimeServer not defined, using default: pool.ntp.org
[0d00h00m04s] 2024-04-20T00:14:20 <INF> [SNTP] Configuring NTP Client...
[0d00h00m04s] 2024-04-20T02:14:20 <INF> [SNTP] Time zone set to CET-1CEST,M3.5.0,M10.5.0/3
[0d00h00m04s] 2024-04-20T02:14:20 <INF> [SNTP] time zone:  0200 Delta to UTC: 7200 seconds
[0d00h00m04s] 2024-04-20T02:14:20 <INF> [SNTP] Time is already set: 2024-04-20 02:14:20
[0d00h00m04s] 2024-04-20T02:14:20 <INF> [MAIN] CPU frequency: 160 MHz
[0d00h00m05s] 2024-04-20T02:14:20 <INF> [SDCARD] Folder/file presence check started...
[0d00h00m05s] 2024-04-20T02:14:20 <INF> [SDCARD] Folder/file presence check successful
[0d00h00m05s] 2024-04-20T02:14:20 <INF> [MAIN] Tag: 'v15.7.0', Release: v15.7.0 (Commit: 0d0b018 ), Date/Time: 2024-02-17 00:15, Web UI: Release: v15.7.0 (Commit: 0d0b018 )
[0d00h00m05s] 2024-04-20T02:14:20 <INF> [MAIN] Reset reason: Via esp_restart
[0d00h00m05s] 2024-04-20T02:14:20 <INF> [WLANINI] SSID: m2m
[0d00h00m05s] 2024-04-20T02:14:20 <INF> [WLANINI] Password: XXXXXXXX
[0d00h00m05s] 2024-04-20T02:14:20 <INF> [WLANINI] RSSIThreshold: -75
[0d00h00m05s] 2024-04-20T02:14:20 <INF> [MAIN] WLAN config loaded, init WIFI...
[0d00h00m05s] 2024-04-20T02:14:20 <INF> [WIFI] Automatic interface config --> Use DHCP service
[0d00h00m05s] 2024-04-20T02:14:21 <INF> [WIFI] Set hostname to: iotatedge
[0d00h00m05s] 2024-04-20T02:14:21 <INF> [WIFI] Init successful
[0d00h00m07s] 2024-04-20T02:14:23 <INF> [WIFI] Connected to: wifi, RSSI: -68
[0d00h00m08s] 2024-04-20T02:14:24 <INF> [WIFI] Assigned IP: 10.20.1.300
[0d00h00m12s] 2024-04-20T02:14:27 <INF> [SNTP] Time is synced with NTP Server pool.ntp.org: 2024-04-20 02:14:27
[0d00h00m13s] 2024-04-20T02:14:29 <INF> [MAIN] Device info: CPU cores: 2, Chip revision: 301
[0d00h00m13s] 2024-04-20T02:14:29 <INF> [MAIN] SD card info: Name: SA16G, Capacity: 15193MB, Free: 15167MB
[0d00h00m15s] 2024-04-20T02:14:31 <INF> [MAIN] Initialization completed successfully
[0d00h00m17s] 2024-04-20T02:14:33 <INF> [LOGFILE] Set log level to DEBUG
[0d00h00m17s] 2024-04-20T02:14:33 <INF> [MAINCTRL] Starting Flow...
[0d00h00m17s] 2024-04-20T02:14:33 <DBG> [MAINCTRL] ----------------------------------------------------------------
[0d00h00m17s] 2024-04-20T02:14:33 <INF> [MAINCTRL] Round #1 started
[0d00h00m17s] 2024-04-20T02:14:33 <DBG> [FLOWCTRL] Status: Take Image (02:14:33)
[0d00h00m17s] 2024-04-20T02:14:33 <DBG> [PSRAM] Init shared memory for step 'Take Image' (STBI buffers)
[0d00h00m21s] 2024-04-20T02:14:37 <DBG> [PSRAM] Allocated 18456 bytes in PSRAM for 'STBI'
[0d00h00m21s] 2024-04-20T02:14:37 <DBG> [PSRAM] Freeing memory in PSRAM used for 'STBI'...
[0d00h00m21s] 2024-04-20T02:14:37 <DBG> [PSRAM] Allocated 18456 bytes in PSRAM for 'STBI'
[0d00h00m21s] 2024-04-20T02:14:37 <DBG> [PSRAM] Allocating memory (307215 bytes) for STBI (use shared memory in PSRAM)...
[0d00h00m21s] 2024-04-20T02:14:37 <DBG> [PSRAM] Allocating memory (153615 bytes) for STBI (use shared memory in PSRAM)...
[0d00h00m21s] 2024-04-20T02:14:37 <DBG> [PSRAM] Allocating memory (153615 bytes) for STBI (use shared memory in PSRAM)...
[0d00h00m21s] 2024-04-20T02:14:37 <DBG> [PSRAM] Allocated 643 bytes in PSRAM for 'STBI'
[0d00h00m21s] 2024-04-20T02:14:37 <DBG> [PSRAM] Allocated 643 bytes in PSRAM for 'STBI'
[0d00h00m21s] 2024-04-20T02:14:37 <DBG> [PSRAM] Allocated 643 bytes in PSRAM for 'STBI'
[0d00h00m21s] 2024-04-20T02:14:37 <DBG> [PSRAM] Allocating memory (921601 bytes) for STBI (use shared memory in PSRAM)...
[0d00h00m22s] 2024-04-20T02:14:38 <DBG> [PSRAM] Part of shared memory used for STBI (PSRAM, part of shared memory) is free again
[0d00h00m22s] 2024-04-20T02:14:38 <DBG> [PSRAM] Freeing memory in PSRAM used for 'STBI'...
[0d00h00m22s] 2024-04-20T02:14:38 <DBG> [PSRAM] Part of shared memory used for STBI (PSRAM, part of shared memory) is free again
[0d00h00m22s] 2024-04-20T02:14:38 <DBG> [PSRAM] Freeing memory in PSRAM used for 'STBI'...
[0d00h00m22s] 2024-04-20T02:14:38 <DBG> [PSRAM] Part of shared memory used for STBI (PSRAM, part of shared memory) is free again
[0d00h00m22s] 2024-04-20T02:14:38 <DBG> [PSRAM] Freeing memory in PSRAM used for 'STBI'...
[0d00h00m22s] 2024-04-20T02:14:38 <DBG> [PSRAM] Freeing memory in PSRAM used for 'STBI'...
[0d00h00m24s] 2024-04-20T02:14:40 <DBG> [C IMG BASIS] Not freeing (zwImage as there was never PSRAM allocated for it)
[0d00h00m24s] 2024-04-20T02:14:40 <DBG> [PSRAM] Deinit shared memory for step 'Take Image' (STBI buffers)
[0d00h00m24s] 2024-04-20T02:14:40 <DBG> [FLOWCTRL] Status: Aligning (02:14:40)
[0d00h00m25s] 2024-04-20T02:14:41 <DBG> [PSRAM] Allocating tmpImage (921600 bytes, use shared memory in PSRAM)...
[0d00h00m27s] 2024-04-20T02:14:43 <DBG> [PSRAM] Allocated 18456 bytes in PSRAM for 'STBI'
[0d00h00m27s] 2024-04-20T02:14:43 <DBG> [PSRAM] Freeing memory in PSRAM used for 'STBI'...
[0d00h00m27s] 2024-04-20T02:14:43 <DBG> [PSRAM] Allocated 18456 bytes in PSRAM for 'STBI'
[0d00h00m27s] 2024-04-20T02:14:43 <DBG> [PSRAM] Allocated 17935 bytes in PSRAM for 'STBI'
[0d00h00m27s] 2024-04-20T02:14:43 <DBG> [PSRAM] Allocated 4495 bytes in PSRAM for 'STBI'
[0d00h00m27s] 2024-04-20T02:14:43 <DBG> [PSRAM] Allocated 4495 bytes in PSRAM for 'STBI'
[0d00h00m27s] 2024-04-20T02:14:43 <DBG> [PSRAM] Allocated 105 bytes in PSRAM for 'STBI'
[0d00h00m27s] 2024-04-20T02:14:43 <DBG> [PSRAM] Allocated 105 bytes in PSRAM for 'STBI'
[0d00h00m27s] 2024-04-20T02:14:43 <DBG> [PSRAM] Allocated 105 bytes in PSRAM for 'STBI'
[0d00h00m27s] 2024-04-20T02:14:43 <DBG> [PSRAM] Allocated 44983 bytes in PSRAM for 'STBI'
[0d00h00m27s] 2024-04-20T02:14:43 <DBG> [PSRAM] Freeing memory in PSRAM used for 'STBI'...
[0d00h00m27s] 2024-04-20T02:14:43 <DBG> [PSRAM] Freeing memory in PSRAM used for 'STBI'...
[0d00h00m27s] 2024-04-20T02:14:43 <DBG> [PSRAM] Freeing memory in PSRAM used for 'STBI'...
[0d00h00m27s] 2024-04-20T02:14:43 <DBG> [PSRAM] Freeing memory in PSRAM used for 'STBI'...
[0d00h00m27s] 2024-04-20T02:14:43 <DBG> [PSRAM] Freeing memory in PSRAM used for 'STBI'...
[0d00h00m27s] 2024-04-20T02:14:43 <DBG> [PSRAM] Freeing memory in PSRAM used for 'STBI'...
[0d00h00m27s] 2024-04-20T02:14:43 <DBG> [PSRAM] Freeing memory in PSRAM used for 'STBI'...
[0d00h00m39s] 2024-04-20T02:14:54 <DBG> [OTA FILE] download_get_handler
[0d00h00m39s] 2024-04-20T02:14:55 <DBG> [MAIN SERVER] info_get_handler
[0d00h00m56s] 2024-04-20T02:15:11 <DBG> [PSRAM] Freeing memory in PSRAM used for 'STBI'...
[0d00h00m56s] 2024-04-20T02:15:11 <DBG> [PSRAM] Allocated 18456 bytes in PSRAM for 'STBI'
[0d00h00m56s] 2024-04-20T02:15:11 <DBG> [PSRAM] Freeing memory in PSRAM used for 'STBI'...
[0d00h00m56s] 2024-04-20T02:15:11 <DBG> [PSRAM] Allocated 18456 bytes in PSRAM for 'STBI'
[0d00h00m56s] 2024-04-20T02:15:11 <DBG> [PSRAM] Allocated 5135 bytes in PSRAM for 'STBI'
[0d00h00m56s] 2024-04-20T02:15:11 <DBG> [PSRAM] Allocated 1295 bytes in PSRAM for 'STBI'
[0d00h00m56s] 2024-04-20T02:15:11 <DBG> [PSRAM] Allocated 1295 bytes in PSRAM for 'STBI'
[0d00h00m56s] 2024-04-20T02:15:11 <DBG> [PSRAM] Allocated 78 bytes in PSRAM for 'STBI'
[0d00h00m56s] 2024-04-20T02:15:11 <DBG> [PSRAM] Allocated 78 bytes in PSRAM for 'STBI'
[0d00h00m56s] 2024-04-20T02:15:11 <DBG> [PSRAM] Allocated 78 bytes in PSRAM for 'STBI'
[0d00h00m56s] 2024-04-20T02:15:11 <DBG> [PSRAM] Allocated 11701 bytes in PSRAM for 'STBI'
[0d00h00m56s] 2024-04-20T02:15:11 <DBG> [PSRAM] Freeing memory in PSRAM used for 'STBI'...
[0d00h00m56s] 2024-04-20T02:15:11 <DBG> [PSRAM] Freeing memory in PSRAM used for 'STBI'...
[0d00h00m56s] 2024-04-20T02:15:11 <DBG> [PSRAM] Freeing memory in PSRAM used for 'STBI'...
[0d00h00m56s] 2024-04-20T02:15:11 <DBG> [PSRAM] Freeing memory in PSRAM used for 'STBI'...
[0d00h00m56s] 2024-04-20T02:15:11 <DBG> [PSRAM] Freeing memory in PSRAM used for 'STBI'...
[0d00h00m56s] 2024-04-20T02:15:12 <DBG> [PSRAM] Freeing memory in PSRAM used for 'STBI'...
[0d00h00m56s] 2024-04-20T02:15:12 <DBG> [PSRAM] Freeing memory in PSRAM used for 'STBI'...
[0d00h01m07s] 2024-04-20T02:15:23 <DBG> [OTA FILE] download_get_handler
[0d00h01m07s] 2024-04-20T02:15:23 <DBG> [MAIN SERVER] info_get_handler
[0d00h01m08s] 2024-04-20T02:15:23 <DBG> [PSRAM] Freeing memory in PSRAM used for 'STBI'...
[0d00h01m13s] 2024-04-20T02:15:29 <DBG> [PSRAM] Shared memory used for tmpImage (PSRAM, part of shared memory) is free again
[0d00h01m13s] 2024-04-20T02:15:29 <DBG> [FLOWCTRL] Status: Digitalization of ROIs (02:15:29)
[0d00h01m14s] 2024-04-20T02:15:30 <DBG> [OTA FILE] download_get_handler
[0d00h01m14s] 2024-04-20T02:15:30 <DBG> [CNN] doFlow after alignment
[0d00h01m14s] 2024-04-20T02:15:30 <DBG> [PSRAM] Allocating Tensor Arena (819200 bytes, use shared memory in PSRAM)...
[0d00h01m14s] 2024-04-20T02:15:30 <DBG> [TFLITE] CTfLiteClass::LoadModel
[0d00h01m14s] 2024-04-20T02:15:30 <DBG> [TFLITE] CTfLiteClass::ReadFileToModel: /sdcard/config/dig-class11_1700_s2.tflite
[0d00h01m14s] 2024-04-20T02:15:30 <DBG> [TFLITE] Loading Model /sdcard/config/dig-class11_1700_s2.tflite /size: 356488 bytes...
[0d00h01m14s] 2024-04-20T02:15:30 <DBG> [PSRAM] Allocating Model memory (1363148 bytes, use shared memory in PSRAM)...
[0d00h01m15s] 2024-04-20T02:15:31 <DBG> [MAIN SERVER] info_get_handler
[0d00h01m15s] 2024-04-20T02:15:31 <DBG> [TFLITE] CTfLiteClass::MakeAllocate
[0d00h01m15s] 2024-04-20T02:15:31 <DBG> [CNN] Processing Number 'main'
[0d00h01m15s] 2024-04-20T02:15:31 <DBG> [CNN] ROI #0 - TfLite
[0d00h01m15s] 2024-04-20T02:15:31 <DBG> [CNN] CNN Type: Digital
[0d00h01m16s] 2024-04-20T02:15:31 <DBG> [CNN] ROI #1 - TfLite
[0d00h01m16s] 2024-04-20T02:15:31 <DBG> [CNN] CNN Type: Digital
[0d00h01m16s] 2024-04-20T02:15:32 <DBG> [CNN] ROI #2 - TfLite
[0d00h01m16s] 2024-04-20T02:15:32 <DBG> [CNN] CNN Type: Digital
[0d00h01m17s] 2024-04-20T02:15:32 <DBG> [CNN] ROI #3 - TfLite
[0d00h01m17s] 2024-04-20T02:15:32 <DBG> [CNN] CNN Type: Digital
[0d00h01m17s] 2024-04-20T02:15:33 <DBG> [CNN] ROI #4 - TfLite
[0d00h01m17s] 2024-04-20T02:15:33 <DBG> [CNN] CNN Type: Digital
[0d00h01m18s] 2024-04-20T02:15:33 <DBG> [CNN] ROI #5 - TfLite
[0d00h01m18s] 2024-04-20T02:15:33 <DBG> [CNN] CNN Type: Digital
[0d00h01m18s] 2024-04-20T02:15:34 <DBG> [PSRAM] Shared memory used for Tensor Arena and model (PSRAM, part of shared memory) is free again
[0d00h01m18s] 2024-04-20T02:15:34 <DBG> [FLOWCTRL] Status: Post-Processing (02:15:34)
[0d00h01m18s] 2024-04-20T02:15:34 <DBG> [CNN] getReadout _analog=0, _extendedResolution=0, prev=-1
[0d00h01m18s] 2024-04-20T02:15:34 <DBG> [POSTPROC] handleAllowNegativeRate for device: main
[0d00h01m18s] 2024-04-20T02:15:34 <INF> [POSTPROC] main: Raw: 245.044, Value: 245.044, Status: no error
[0d00h01m18s] 2024-04-20T02:15:34 <INF> [MAINCTRL] Round #1 completed (61 seconds)
[0d00h01m18s] 2024-04-20T02:15:34 <DBG> [MAINCTRL] CPU Temperature: 53°C
[0d00h01m18s] 2024-04-20T02:15:34 <DBG> [MAINCTRL] WIFI Signal (RSSI): -68dBm
[0d00h01m20s] 2024-04-20T02:15:36 <DBG> [OTA FILE] download_get_handler
[0d00h01m20s] 2024-04-20T02:15:36 <DBG> [MAIN SERVER] info_get_handler
[0d00h03m42s] 2024-04-20T02:17:58 <DBG> [OTA FILE] download_get_handler
[0d00h03m42s] 2024-04-20T02:17:58 <DBG> [OTA FILE] download_get_handler
[0d00h05m17s] 2024-04-20T02:19:33 <DBG> [MAINCTRL] ----------------------------------------------------------------

Expected Behavior

no image corrutption

Screenshots

No response

Additional Context

15.7 has nice image processing features in terms of autoexposure, b/w and for me zoom. It is bad in converting configs from older versions, because model files dissappear? settings change and the wizard needs to be completed before it'll ever take a picture. But once you do each step individually and check the config file and reboot it works out. Not perfect, I think you need unit tests or something.
@EwaldvanGeffen EwaldvanGeffen added the bug Something isn't working label Apr 20, 2024
@SybexX
Copy link
Collaborator

SybexX commented Apr 26, 2024

try the current rolling, the zoom is better/expanded there.
https://github.com/jomjol/AI-on-the-edge-device/actions/runs/8811491971

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@SybexX @EwaldvanGeffen